Have you ever come across circular imports in Python? Well, it’s a very common code smell that indicates something’s wrong with the design or structure.
Circular Import Example
How does circular import occur? This import error usually occurs when two or more modules depending on each other try to import before fully initializing.
Let’s say we have two modules: module_1.py and module_2.py.
from module_2 import ModY
class ModX:
mody_obj = ModY()
from module_1 import ModX
class ModY:
modx_obj = ModX()
In the above code snippets, both module_1 and module_2 are mutually dependent on each other.
The initialization of mody_obj in module_1 depends on module_2 and the initialization of modx_obj in module_2 depends on module_1.
This is what we call a circular dependency. Both modules will stuck in the import loops while attempting to load each other.
If we run module_1.py, we’ll get the following traceback.
Traceback (most recent call last):
File "module_1.py", line 1, in <module>
from module_2 import ModY
File "module_2.py", line 1, in <module>
from module_1 import ModX
File "module_1.py", line 1, in <module>
from module_2 import ModY
ImportError: cannot import name 'ModY' from partially initialized module 'module_2' (most likely due to a circular import)
This error explains the situation of circular import. When the program attempted to import ModY from module_2, at that time module_2 wasn’t fully initialized (due to another import statement that attempts to import ModX from module_1).
How to fix circular imports in Python? There are different ways to get rid of circular imports in Python.
Fix Circular Imports in Python
Move code into a common file
We can move the code into a common file to avoid import errors and then try to import the modules from that file.
class ModX:
pass
class ModY:
pass
In the above code snippet, we moved the classes ModX and ModY into a common file (main.py).
from main import ModY
class Mod_X:
mody_obj = ModY()
from main import ModX
class Mod_Y:
modx_obj = ModX()
Now, module_1 and module_2 import the classes from main which fixes the circular import situation.
There is a problem with this approach, sometimes the codebase is so large that it becomes risky to move the code into another file.
Move the import to the end of the module
We can shift the import statement at the end of the module. This will give time to fully initialize the module before importing another module.
class ModX:
pass
from module_2 import ModY
class Mod_X:
mody_obj = ModY()
class ModY:
pass
from module_1 import ModX
Importing module within the class/function scope
Importing modules within the class or function scope can avoid circular imports. This allows the module to be imported only when the class or function is invoked. It’s relevant when we want to minimize memory use.
class ModX:
pass
class Mod_X:
from module_2 import ModY
mody_obj = ModY()
class ModY:
pass
class Mod_Y:
from module_1 import ModX
modx_obj = ModX()
We moved the import statements within classes Mod_X and Mod_Y scope in module_1 and module_2 respectively.
If we run either module_1 or module_2, we’ll not get a circular import error. But, this approach makes the class accessible only within the class’s scope, so we can’t leverage the import globally.
Using module name/alias
Using the module name or just an alias like this solves the problem. This allows both modules to load fully by deferring circular dependency until runtime.
import module_2 as m2
class ModX:
def __init__(self):
self.mody_obj = m2.ModY()
import module_1 as m1
class ModY:
def __init__(self):
self.modx_obj = m1.ModX()
Using importlib library
We can also use the importlib library to import the modules dynamically.
import importlib
class ModX:
def __init__(self):
m2 = importlib.import_module('module_2')
self.mody_obj = m2.ModY()
import importlib
class ModY:
def __init__(self):
m1 = importlib.import_module('module_1')
self.mody_obj = m1.ModX()
Circular Imports in Python Packages
Usually, circular imports come from modules within the same package. In complex projects, the directory structure is also complex, with packages within packages.
These packages and sub-packages contain __init__.py files to provide easier access to modules. That’s where sometimes arises circular dependencies among modules unintentionally.
We have the following directory structure.
root_dir/
|- mainpkg/
|---- modpkg_x/
|-------- __init__.py
|-------- module_1.py
|-------- module_1_1.py
|---- modpkg_y/
|-------- __init__.py
|-------- module_2.py
|---- __init__.py
|- main.py
We have a package mainpkg and a main.py file. We have two sub-packages modpkg_x and modpkg_y within mainpkg.
Here’s what each Python file within modpkg_x and modpkg_y looks like.
mainpkg/modpkg_x/__init__.py
from .module_1 import ModX
from .module_1_1 import ModA
This file imports both classes (ModX and ModA) from module_1 and module_1_1.
mainpkg/modpkg_x/module_1.py
from ..modpkg_y.module_2 import ModY
class ModX:
mody_obj = ModY()
The module_1 imports a class ModY from module_2.
mainpkg/modpkg_x/module_1_1.py
class ModA:
pass
The module_1_1 imports nothing. It is not dependent on any module.
mainpkg/modpkg_y/__init__.py
from .module_2 import ModY
This file imports the class ModY from module_2.
mainpkg/modpkg_y/module_2.py
from ..modpkg_x.module_1_1 import ModA
class ModY:
moda_obj = ModA()
The module_2 imports a class ModA from the module_1_1.
We have the following code within the main.py file.
root_dir/main.py
from mainpkg.modpkg_y.module_2 import ModY
def mody():
y_obj = ModY()
mody()
The main file imports a class ModY from module_2. This file is dependent on module_2.
If we visualize the import cycle here, it would look like the following ignoring the __init__.py files within the modpkg_x and modpkg_y.
We can see that the main file depends on module_2, module_1 also depends on module_2 and module_2 depends on module_1_1. There is no import cycle.
But you know, modules depend on their __init__.py file, so the __init__.py file initializes first, and modules are re-imported.
This is what the import cycle looks like now.
This made module_1_1 depend on module_1, which is a fake dependency.
If this is the case, empty the sub-packages __init__.py files and using a separate __init__.py file can help by centralizing imports at the package level.
root_dir/
|- mainpkg/
|---- modpkg_x/
|-------- __init__.py
|-------- module_1.py
|-------- module_1_1.py
|---- modpkg_y/
|-------- __init__.py
|-------- module_2.py
|---- subpkg/
|-------- __init__.py
|---- __init__.py
|- main.py
In this structure, we added another sub-package subpkg within mainpkg.
mainpkg/subpkg/__init__.py
from ..modpkg_x.module_1 import ModX
from ..modpkg_x.module_1_1 import ModA
from ..modpkg_y.module_2 import ModY
This will allow internal modules to import from a single source, reducing the need for cross-imports.
Now we can update the import statement within the main.py file.
root_dir/main.py
from mainpkg.subpkg import ModY
def mody():
y_obj = ModY()
mody()
This solves the problem of circular dependency between the modules within the same package.
Conclusion
Circular dependency or import in Python is a code smell which is an indication of serious re-structuring and refactoring of the code.
You can try any of these above-mentioned ways to avoid circular dependency in Python.
That’s all for now.
Keep Coding✌✌.
