Creating conftest.py hierarchies for monorepos
Scaling pytest across a multi-package monorepo introduces architectural complexity that flat test configurations cannot sustain. As test suites grow, implicit fixture resolution, unpredictable plugin loading, and collection bloat frequently degrade CI reliability and developer velocity. Establishing a deterministic conftest.py hierarchy is not merely an organizational preference; it is a foundational requirement for maintaining strict scope boundaries, preventing cross-package leakage, and enabling parallelized execution. This guide details production-grade strategies for architecting, debugging, and validating conftest.py hierarchies in modern Python monorepos.
Monorepo Test Architecture: Why conftest.py Hierarchies Fail at Scale
When scaling pytest across a monorepo, developers frequently encounter unpredictable fixture resolution and silent collection failures. Understanding how pytest traverses directories and loads implicit plugins is foundational to avoiding architectural debt. The Advanced Pytest Architecture & Configuration framework provides the necessary mental model for mapping collection boundaries to package structures.
At its core, pytest does not rely on Python's standard import system for test discovery. Instead, it executes an upward directory traversal starting from each collected test file. During this traversal, pytest identifies conftest.py files and registers them as implicit plugins. These plugins are loaded into the FixtureManager in Last-In-First-Out (LIFO) order, meaning the nearest conftest.py to the test file takes precedence over parent directories. While this mechanism simplifies local test execution, it becomes a liability in monorepos where multiple services share overlapping fixture names or where sys.path manipulation interferes with collection boundaries.
Collection bloat typically originates from flat conftest.py structures. When a single root configuration file declares dozens of session-scoped fixtures, pytest eagerly evaluates them for every collected test, regardless of whether the test actually requests them. This eager evaluation compounds with autouse=True fixtures, triggering expensive database migrations, network stubs, or environment variable injections across unrelated packages. Furthermore, namespace package discovery (PEP 420) introduces subtle pitfalls: directories lacking __init__.py may be silently skipped during traversal, breaking expected fixture inheritance chains and causing FixtureLookupError exceptions that only manifest in CI environments.
The interaction between testpaths in pyproject.toml and sys.path resolution further complicates matters. If testpaths is misconfigured, pytest may inadvertently collect production modules, triggering import-time side effects or registering fixtures in unintended scopes. Without explicit boundary enforcement, monorepo test suites devolve into tightly coupled dependency graphs where modifying a single fixture in one package triggers cascading failures across unrelated services.
Designing the Hierarchy: Root, Domain, and Module Scopes
A resilient monorepo testing strategy requires explicit boundary enforcement. By delegating responsibilities across root, domain, and module tiers, teams can prevent cross-package fixture collisions. Detailed implementation patterns for this tiered approach are documented in Managing Conftest Hierarchies, which outlines scope delegation and safe override mechanics.
The three-tier delegation model maps directly to pytest's fixture scoping system:
- Root Tier (
/tests/conftest.py): Handles global infrastructure, CI environment detection, and cross-cutting concerns. Fixtures here should be strictlysession-scoped and lazily initialized. This tier is the appropriate location forpytest_addoptionhooks, global logging configuration, and baselinepytest_pluginsdeclarations. - Domain/Service Tier (
/tests/<package>/conftest.py): Encapsulates service-specific mocks, database schemas, and API client factories. Fixtures here typically operate atmoduleorclassscope. This tier enforces package isolation by overriding root fixtures with domain-specific implementations and applying marker-based filtering to prevent accidental cross-domain execution. - Module/Edge-Case Tier (
/tests/<package>/test_<module>/conftest.py): Reserved for highly localized parametrization, temporary directory overrides, and test-specific state resets. These fixtures should remainfunction-scoped and avoidautouse=Trueunless strictly necessary for cleanup.
Strict scope isolation relies on explicit pytest_plugins registration rather than implicit Python imports. Using import statements to share fixtures across packages bypasses pytest's internal dependency graph, causing fixtures to be registered in the wrong collection phase and breaking request.node introspection. Instead, declare shared fixture modules as strings in pytest_plugins = ["tests.shared.fixtures.db"]. This ensures pytest loads them as proper plugins with correct hook execution order.
Additionally, configure testpaths in pyproject.toml to restrict collection to dedicated test directories. Never place __init__.py files inside test directories. Their presence converts test folders into Python packages, altering sys.path resolution and causing pytest to treat conftest.py files as regular modules rather than implicit plugins. This distinction is critical for maintaining predictable pytest fixture scope isolation across large codebases.
Implementation: Minimal Reproducible Hierarchy
The following directory structure demonstrates a production-ready hierarchy. Each conftest.py file is scoped to its logical boundary, with explicit plugin registration replacing implicit import leakage.
monorepo/
├── pyproject.toml
├── src/
│ ├── auth/
│ └── billing/
└── tests/
├── conftest.py # Root: Global CI & Plugin Registration
├── shared/
│ └── fixtures/
│ ├── db.py
│ └── network.py
├── auth/
│ ├── conftest.py # Domain: Auth-specific overrides
│ └── test_sessions.py
└── billing/
├── conftest.py # Domain: Billing-specific overrides
└── test_invoices.py
Root conftest.py: Global Infrastructure & Plugin Registration
The root configuration establishes the baseline environment and registers shared fixture modules. It avoids heavy initialization, deferring expensive operations until explicitly requested.
# tests/conftest.py
import pytest
from pathlib import Path
# Explicit plugin registration replaces cross-package imports
pytest_plugins = ["tests.shared.fixtures.db", "tests.shared.fixtures.network"]
def pytest_addoption(parser):
"""Register CLI options for environment-specific toggles."""
parser.addoption(
"--env",
default="local",
choices=["local", "ci", "staging"],
help="Target environment for test execution"
)
@pytest.fixture(scope="session", autouse=True)
def global_config(request):
"""Session-scoped configuration resolver.
Uses request.config.getoption() to inject environment context.
"""
env = request.config.getoption("--env")
return {
"env": env,
"db_url": f"sqlite:///test_{env}.db",
"mock_network": env == "ci"
}
Domain conftest.py: Service-Specific Overrides & Isolation
Domain-level configurations override root fixtures when necessary and enforce package boundaries using markers and autouse guards.
# tests/auth/conftest.py
import pytest
@pytest.fixture(scope="module")
def service_client(global_config):
"""Domain-scoped client factory.
Inherits global_config but applies auth-specific initialization.
"""
return create_auth_client(global_config["db_url"])
@pytest.fixture(autouse=True)
def enforce_domain_boundary(request):
"""Prevents cross-domain test execution without explicit markers.
Uses request.node.keywords for marker introspection.
"""
if "cross_domain" not in request.node.keywords:
yield
else:
pytest.skip("Cross-domain tests require explicit @pytest.mark.cross_domain")
Key implementation principles:
- Placement outside
src/: Allconftest.pyfiles reside in thetests/directory to prevent production import pollution. pytest_pluginssyntax: Always use list format. String concatenation or dynamic assignment breaks plugin discovery.- Fixture override mechanics: Child
conftest.pyfiles automatically override parent fixtures with identical names. Use@pytest.fixture(autouse=True)cautiously to avoid unintended teardown side effects. - Environment toggles:
request.config.getoption()enables deterministic behavior across local development and CI pipelines without modifying test code.
Edge-Case Resolution: Cross-Package Fixture Collisions & Namespace Packages
When sibling packages define identically named fixtures, pytest resolves them based on collection order, leading to non-deterministic test behavior. Use pytest --fixtures --verbose to trace definition paths and enforce explicit naming conventions.
Fixture Name Resolution Order
Pytest's FixtureManager builds a dependency graph during collection. If two conftest.py files in the same directory tree define @pytest.fixture(name="db_session"), the nearest file to the test file wins. However, if tests are collected in parallel or via glob patterns, the resolution order becomes unpredictable. This manifests as intermittent AssertionError failures where tests expect a mocked database but receive a production connection.
Resolution Strategy:
- Prefix fixtures with domain identifiers (e.g.,
auth_db_session,billing_db_session). - Use
@pytest.fixture(name="explicit_name")to enforce uniqueness. - Implement a CI linter to scan for duplicate fixture definitions before merge.
Diagnostic Script: Fixture Collision Detector
Automated collision detection prevents hierarchy drift. The following AST-based scanner identifies duplicate fixture names across the monorepo without executing tests.
# scripts/scan_fixture_collisions.py
import ast
import sys
from pathlib import Path
def scan_conftest_fixtures(root: Path) -> dict[str, list[str]]:
"""Scan all conftest.py files and report duplicate fixture names."""
fixtures: dict[str, list[str]] = {}
for conf in root.rglob("conftest.py"):
try:
tree = ast.parse(conf.read_text())
except SyntaxError:
continue
for node in ast.walk(tree):
if isinstance(node, ast.FunctionDef):
for dec in node.decorator_list:
# Handle @pytest.fixture and @pytest.fixture(name="...")
if isinstance(dec, ast.Call) and hasattr(dec.func, "attr") and dec.func.attr == "fixture":
# Check for explicit name kwarg
name_kwarg = next((kw.value.value for kw in dec.keywords if kw.arg == "name"), None)
fixture_name = name_kwarg or node.name
fixtures.setdefault(fixture_name, []).append(str(conf.relative_to(root)))
elif isinstance(dec, ast.Name) and dec.id == "fixture":
fixtures.setdefault(node.name, []).append(str(conf.relative_to(root)))
return {k: v for k, v in fixtures.items() if len(v) > 1}
if __name__ == "__main__":
root = Path(sys.argv[1] if len(sys.argv) > 1 else ".")
collisions = scan_conftest_fixtures(root)
if collisions:
print("️ Fixture collisions detected:")
for name, paths in collisions.items():
print(f" {name}: {', '.join(paths)}")
sys.exit(1)
print("✅ No fixture collisions found.")
PEP 420 Namespace Package Pitfalls
Directories without __init__.py are treated as implicit namespace packages. While beneficial for source code, they disrupt pytest's upward traversal. If a parent directory lacks __init__.py, pytest may skip conftest.py files entirely, breaking fixture inheritance.
Resolution: Add empty __init__.py to test directories containing conftest.py, or explicitly configure pythonpath = ["."] in pyproject.toml. Alternatively, use pytest --ignore=src to force explicit test path resolution and bypass namespace package ambiguity.
Advanced Control: Custom Hooks & Collection Modification
For large-scale monorepos, static conftest.py files often require runtime augmentation. Hooking into pytest_collection_modifyitems allows teams to filter, reorder, or inject markers without modifying test files.
Dynamic Test Filtering & Marker Injection
The pytest_collection_modifyitems hook receives the complete list of collected Item objects before execution. This enables package-aware filtering, which is essential for CI pipelines running targeted test suites.
# tests/conftest.py (advanced hook integration)
import pytest
def pytest_collection_modifyitems(config, items):
"""Filter tests based on --package CLI option or inject markers."""
target_pkg = config.getoption("--package", default=None)
if not target_pkg:
return
skip_marker = pytest.mark.skip(reason=f"Excluded: --package={target_pkg}")
for item in items:
# Check if test belongs to target package path
if target_pkg not in str(item.fspath):
item.add_marker(skip_marker)
Early Plugin Bootstrapping via pytest_configure
The pytest_configure hook executes before collection begins, making it ideal for dynamic pytest_plugins assignment or environment validation. However, modifying pytest_plugins at this stage requires caution to avoid hook recursion.
def pytest_configure(config):
"""Validate environment and conditionally register plugins."""
env = config.getoption("--env")
if env == "staging":
# Dynamically register staging-specific fixtures
config.pluginmanager.register(StagingNetworkMock(), "staging_network")
# Prevent recursive hook execution
if not hasattr(config, "_hierarchy_validated"):
validate_hierarchy_integrity(config)
config._hierarchy_validated = True
Avoiding Hook Recursion & Stack Overflow
Pytest's hook execution follows a strict chain. Modifying plugin state or calling config.pluginmanager.register() during collection can trigger infinite recursion if not guarded. Always use configuration flags (config._hierarchy_validated) to ensure hooks execute exactly once. Additionally, avoid calling pytest.main() or re-invoking collection inside hooks, as this resets the internal state machine and corrupts fixture caches.
Validation & Profiling: Ensuring Hierarchy Integrity
Continuous validation prevents hierarchy drift. Integrate pytest --trace-config into CI pipelines to audit plugin loading sequences and enforce deterministic collection across environments.
Plugin Loading Audit
pytest --trace-config outputs the exact order in which conftest.py files and plugins are registered. Scan this output for unexpected duplicates or out-of-order loading. In CI, pipe the output to a grep filter to assert that only expected plugins are active:
pytest --trace-config 2>&1 | grep -E "conftest|plugin" > /tmp/plugin_audit.log
# CI assertion: Verify no duplicate conftest paths
if grep -q "duplicate" /tmp/plugin_audit.log; then
echo "❌ Plugin loading collision detected"
exit 1
fi
Fixture Overhead Profiling
Use pytest --durations=10 to identify the slowest fixtures. Combine with pytest --trace-config to verify conftest loading order. For granular measurement, wrap session-scoped fixtures with timing decorators:
import time
import pytest
from functools import wraps
def profiled_fixture(func):
@wraps(func)
def wrapper(*args, **kwargs):
start = time.perf_counter()
result = yield from func(*args, **kwargs)
duration = time.perf_counter() - start
if duration > 2.0:
pytest._log.warning(f"Fixture {func.__name__} took {duration:.2f}s")
return result
return wrapper
CI Pipeline Validation Scripts
Automate hierarchy checks using pre-commit hooks and CI gates:
- Pre-commit: Run the AST collision detector and enforce
testpathsconfiguration. - CI Gate: Execute
pytest --collect-only --strict-markersto verify marker consistency. - Profiling Gate: Fail builds if
--durationsoutput exceeds baseline thresholds by >15%.
These automated checks ensure that conftest.py best practices python developers rely on remain enforced as the monorepo scales.
Common Pitfalls & Resolution Matrix
| Pitfall | Symptom | Diagnosis | Resolution |
|---|---|---|---|
| Fixture Name Shadowing | Tests in package B unexpectedly use package A's fixture implementation. | pytest --fixtures --verbose shows identical names in sibling conftest.py files. | Prefix fixtures with domain identifiers or use @pytest.fixture(name="explicit_name"). Implement CI linter. |
conftest.py Inside src/ | Pytest collects production code; fixtures leak into runtime imports. | Check testpaths in pyproject.toml. Run pytest --collect-only to verify tree. | Move all conftest.py to tests/. Use norecursedirs = src. Remove __init__.py from test dirs. |
pytest_plugins Misconfiguration | Plugins fail silently or raise ModuleNotFoundError. | Enable pytest --trace-config. Check for string concatenation instead of list assignment. | Always use pytest_plugins = ["pkg.module"]. Validate paths with importlib.util.find_spec(). |
| Namespace Package Discovery Failure | Pytest skips conftest.py in directories lacking __init__.py. | Run python -c "import sys; print(sys.path)". Verify pytest's collection root. | Add __init__.py to test directories or configure pythonpath in pyproject.toml. |
Frequently Asked Questions
Can I safely share fixtures across completely unrelated monorepo packages?
Yes, but only through a centralized tests/shared/ directory referenced via pytest_plugins in each package's conftest.py. Avoid placing shared fixtures in the root conftest.py unless they are truly global. Use explicit imports or plugin registration to maintain traceability and prevent implicit scope leakage.
Why does pytest ignore my root conftest.py when running tests from a subdirectory?
Pytest only loads conftest.py files that are ancestors of the collected test files. If you run `pytest services/auth/tests