Why are there dummy modules in sys.modules?

Posted on

Question :

Why are there dummy modules in sys.modules?

Importing the standard “logging” module pollutes sys.modules with a bunch of dummy entries:

Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on win32
>>> import sys
>>> import logging
>>> sorted(x for x in sys.modules.keys() if 'log' in x)
['logging', 'logging.atexit', 'logging.cStringIO', 'logging.codecs', 
'logging.os', 'logging.string', 'logging.sys', 'logging.thread', 
'logging.threading', 'logging.time', 'logging.traceback', 'logging.types']

# and perhaps even more surprising:
>>> import traceback
>>> traceback is sys.modules['logging.traceback']
False
>>> sys.modules['logging.traceback'] is None
True

So importing this package puts extra names into sys.modules, except that they are not modules, just references to None. Other modules (e.g. xml.dom and encodings) have this issue as well. Why?

Edit: Building on bobince’s answer, there are pages describing the origin (see section “Dummy Entries in sys.modules”) and future of the feature.

Answer #1:

None values in sys.modules are cached failures of relative lookups.

So when you’re in package foo and you import sys, Python looks first for a foo.sys module, and if that fails goes to the top-level sys module. To avoid having to check the filesystem for foo/sys.py again on further relative imports, it stores None in the sys.modules to flag that the module didn’t exist and a subsequent import shouldn’t look there again, but go straight to the loaded sys.

This is a cPython implementation detail you can’t usefully rely on, but you will need to know it if you’re doing nasty magic import/reload hacking.

It happens to all packages, not just logging. For example, import xml.dom and see xml.dom.xml in the module list as it tries to import xml from inside xml.dom.

As Python moves towards absolute import this ugliness will happen less.

Answered By: bobince

Leave a Reply

Your email address will not be published.