Does python optimize modules when they are imported multiple times?

Posted on

Question :

Does python optimize modules when they are imported multiple times?

If a large module is loaded by some submodule of your code, is there any benefit to referencing the module from that namespace instead of importing it again?

For example:
I have a module MyLib, which makes extensive use of ReallyBigLib. If I have code that imports MyLib, should I dig the module out like so

import MyLib
ReallyBigLib = MyLib.SomeModule.ReallyBigLib

or just

import MyLib
import ReallyBigLib
Asked By: JimB

||

Answer #1:

Python modules could be considered as singletons… no matter how many times you import them they get initialized only once, so it’s better to do:

import MyLib
import ReallyBigLib

Relevant documentation on the import statement:

https://docs.python.org/2/reference/simple_stmts.html#the-import-statement

Once the name of the module is known (unless otherwise specified, the term “module” will refer to both packages and modules), searching for the module or package can begin. The first place checked is sys.modules, the cache of all modules that have been imported previously. If the module is found there then it is used in step (2) of import.

The imported modules are cached in sys.modules:

This is a dictionary that maps module names to modules which have already been loaded. This can be manipulated to force reloading of modules and other tricks. Note that removing a module from this dictionary is not the same as calling reload() on the corresponding module object.

Answered By: Toni Ruža

Answer #2:

As others have pointed out, Python maintains an internal list of all modules that have been imported. When you import a module for the first time, the module (a script) is executed in its own namespace until the end, the internal list is updated, and execution of continues after the import statement.

Try this code:

   # module/file a.py
   print "Hello from a.py!"
   import b

   # module/file b.py
   print "Hello from b.py!"
   import a

There is no loop: there is only a cache lookup.

>>> import b
Hello from b.py!
Hello from a.py!
>>> import a
>>>

One of the beauties of Python is how everything devolves to executing a script in a namespace.

Answered By: Charles Merriam

Answer #3:

It makes no substantial difference. If the big module has already been loaded, the second import in your second example does nothing except adding ‘ReallyBigLib’ to the current namespace.

Answer #4:

WARNING: Python does not guarantee that module will not be initialized twice.
I’ve stubled upon such issue. See discussion:
http://code.djangoproject.com/ticket/8193

Answered By: Munhitsu

Answer #5:

The internal registry of imported modules is the sys.modules dictionary, which maps module names to module objects. You can look there to see all the modules that are currently imported.

You can also pull some useful tricks (if you need to) by monkeying with sys.modules – for example adding your own objects as pseudo-modules which can be imported by other modules.

Answered By: babbageclunk

Answer #6:

It is the same performancewise. There is no JIT compiler in Python yet.

Answered By: Marko

Leave a Reply

Your email address will not be published.