feat: lazy imports google auth by hebaalazzeh · Pull Request #17679 · googleapis/google-cloud-python

hebaalazzeh · 2026-07-09T05:09:19Z

Description

This PR initiates the rollout of our PEP 0810 lazy-loading architecture to google-auth, starting with the transport module.

By applying Python 3.15's native explicit lazy imports directly inside our transport wrappers (requests.py, urllib3.py, and grpc.py), we defer the eager parsing and compilation of the heavy third-party networking libraries (requests, urllib3, and grpc). This acts as the first step in addressing the high initialization latency and peak memory footprints we're seeing on Serverless cold-starts.

Older Python runtimes (3.14 and below) safely ignore the __lazy_modules__ set and fall back to standard eager execution, meaning this introduces zero backwards compatibility risk.

Code Review

This pull request introduces a __lazy_modules__ set in packages/google-auth/google/auth/transport/__init__.py to support lazy loading of transport modules. The review feedback correctly identifies an incorrect module name (_aiohttp_requests instead of aiohttp_requests) that would prevent lazy importing in Python 3.15+.

parthea

LGTM but holding off on formal approval until we have tests. We should assert that on Python 3.15+ the modules are indeed absent from sys.modules until accessed, and that on pre-3.15 environments they fallback cleanly to standard eager imports

import sys
import pytest

# List of modules we expect to be lazy
LAZY_MODULES = [
    "google.auth.transport.requests",
    "google.auth.transport.urllib3",
    "google.auth.transport.grpc",
]

def clean_sys_modules():
    """Helper to ensure we start with a clean slate for import testing."""
    for mod in LAZY_MODULES:
        sys.modules.pop(mod, None)

@pytest.mark.skipif(sys.version_info < (3, 15), reason="PEP 810 requires Python 3.15+")
def test_lazy_imports_on_python_315():
    clean_sys_modules()
    
    # 1. Import the transport package
    import google.auth.transport
    
    # 2. Assert that none of the lazy modules have been eagerly loaded into sys.modules
    for mod in LAZY_MODULES:
        assert mod not in sys.modules
        
    # 3. Access an attribute to trigger reification
    from google.auth.transport import requests
    _ = requests.__name__  # Trigger first-use reification
    
    # 4. Assert that the module has now been reified and loaded
    assert "google.auth.transport.requests" in sys.modules


@pytest.mark.skipif(sys.version_info >= (3, 15), reason="Testing fallback behavior on < 3.15")
def test_fallback_eager_imports_pre_315():
    clean_sys_modules()
    
    # On older Python, __lazy_modules__ is safely ignored, meaning they should eager-load
    import google.auth.transport
    
    for mod in LAZY_MODULES:
        assert mod in sys.modules

parthea · 2026-07-09T22:05:19Z

I see the tests are failing. We also need to add from google.auth.transport import requests to the test file

parthea · 2026-07-09T22:15:01Z

Disregard my last comment.

Under PEP 810, __lazy_modules__ is a module-local opt-in . It only intercepts import statements written inside the module where it is defined

We should only add __lazy_modules__ to files where we also have the same import statements

From https://docs.python.org/3.15/reference/simple_stmts.html#lazy-imports

Any regular (non-lazy) import statement at module scope whose target appears in lazy_modules is treated as a lazy import, exactly as if the lazy keyword had been used.

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

…ization

…ution

…it__ to fully address parthea's feedback

hebaalazzeh · 2026-07-10T07:25:30Z

addressed all comments

parthea · 2026-07-13T17:32:45Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces lazy import support for Python 3.15+ by defining __lazy_modules__ in several transport modules (grpc.py, requests.py, and urllib3.py) and adding corresponding integration tests. However, the feedback highlights that the lazy loading mechanism is currently defeated in grpc.py due to inheriting from grpc.AuthMetadataPlugin at the module level, and in urllib3.py due to a module-level version check on urllib3.__version__.

parthea

Please can you address the feedback from GCA?

hebaalazzeh · 2026-07-13T18:17:38Z

\gemini

parthea · 2026-07-13T18:20:36Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces lazy loading support for heavy transport modules (grpc, requests, and urllib3) to optimize import times, particularly targeting Python 3.15+ (PEP 810), with fallback behavior for older versions. Key changes include defining __lazy_modules__, decoupling classes from eager parent imports, and adding a test suite to verify the lazy loading behavior. Feedback focuses on optimizing a performance bottleneck in urllib3.py's dynamic attribute resolution, addressing a bypassed ImportError check in requests.py due to deferred imports, and adding explanatory comments in grpc.py to prevent accidental regressions.

hebaalazzeh · 2026-07-13T19:07:54Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces support for PEP 0810 (Explicit Lazy Imports) in Python 3.15+ across the grpc, requests, and urllib3 transports. This is achieved by defining lazy_modules, removing eager base class inheritance, and lazily resolving attributes (such as RequestMethods in AuthorizedHttp). A test suite is also added to verify this behavior. The reviewer suggests improving the getattr implementation in AuthorizedHttp by dynamically checking for attributes on the underlying RequestMethods class rather than hardcoding specific method names, and lazily importing version to avoid eagerly loading the packaging module.

ohmayr · 2026-07-13T23:13:49Z

+class AuthorizedHttp(object):
    """A urllib3 HTTP class with credentials.


Changing AuthorizedHttp from subclassing urllib3.request.RequestMethods (or urllib3._request_methods.RequestMethods) to object is a breaking change across all Python versions (3.9–3.15+). Any downstream code that checks isinstance(authed_http, urllib3.request.RequestMethods) (or urllib3.PoolManager) will now return False.

ohmayr · 2026-07-13T23:14:35Z

+    def __getattr__(self, name):
+        if name in ("request", "request_encode_url", "request_encode_body"):
+            import types
+
+            if AuthorizedHttp._request_methods_class is None:
+                if version.parse(urllib3.__version__) >= version.parse("2.0.0"):
+                    AuthorizedHttp._request_methods_class = (
+                        urllib3._request_methods.RequestMethods
+                    )
+                else:
+                    AuthorizedHttp._request_methods_class = (
+                        urllib3.request.RequestMethods
+                    )
+
+            method = getattr(AuthorizedHttp._request_methods_class, name)
+            bound_method = types.MethodType(method, self)
+            setattr(self, name, bound_method)


If we revert class AuthorizedHttp(object): back to inheriting from RequestMethods to avoid the breaking isinstance() change, this __getattr__ method and the _request_methods_class lookup won't be needed and should be removed to keep the implementation simple and safe.

ohmayr · 2026-07-13T23:15:09Z

+# Inheriting from object instead of grpc.AuthMetadataPlugin is intentional.
+# This prevents eagerly loading grpc at module import time, preserving lazy loading support.
+class AuthMetadataPlugin(object):


Similar to AuthorizedHttp in urllib3.py, changing AuthMetadataPlugin from subclassing grpc.AuthMetadataPlugin to object introduces a breaking change across all Python versions for any downstream code checking isinstance(plugin, grpc.AuthMetadataPlugin).

ohmayr · 2026-07-13T23:17:09Z

+__lazy_modules__: Set[str] = {"requests", "requests.adapters", "requests.exceptions"}
+


should this be declared before the import statements for it to actually work?

ohmayr · 2026-07-13T23:18:41Z

            :class:`requests.exceptions.Timeout`.
    """

    def __init__(self, timeout, timeout_error_type=requests.exceptions.Timeout):


Default parameter expressions in function signatures (requests.exceptions.Timeout) are evaluated when the function is defined at module import time. This immediately reifies requests.exceptions (and requests) when requests.py is imported.

ohmayr · 2026-07-13T23:21:12Z

@@ -0,0 +1,77 @@
+# Copyright 2026 Google LLC


We should remove test_lazy_imports.py entirely rather than maintaining subprocess-based tests:

Testing Python's Internal implementation: __lazy_modules__ is a standard declarative hint for PEP 0810. We don't need to maintain custom subprocess unit tests to verify Python 3.15's built-in module reification behavior.

hebaalazzeh marked this pull request as ready for review July 9, 2026 05:09

hebaalazzeh requested review from a team as code owners July 9, 2026 05:09

gemini-code-assist Bot reviewed Jul 9, 2026

View reviewed changes

Comment thread packages/google-auth/google/auth/transport/__init__.py Outdated

parthea reviewed Jul 9, 2026

View reviewed changes

parthea assigned hebaalazzeh Jul 9, 2026

parthea mentioned this pull request Jul 9, 2026

feat: lazy imports gapic v1 #17673

Open

parthea reviewed Jul 9, 2026

View reviewed changes

Comment thread packages/google-auth/google/auth/transport/__init__.py Outdated

hebaalazzeh requested a review from a team as a code owner July 10, 2026 01:33

hebaalazzeh and others added 13 commits July 10, 2026 06:40

feat(auth): Implement PEP 0810 lazy loading in transport

e646bcd

fix: use hardcoded module names instead of __name__

bfc403f

Update packages/google-auth/google/auth/transport/__init__.py

8e528e9

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

test: add lazy imports fallback and tests for PEP 810

af31035

fix: resolve linting errors and add license header

72548f9

chore: fix test_lazy_imports.py to properly trigger module re-initial…

e982bc7

…ization

chore: fix flake8 lint errors related to import ordering

bc4ae32

chore: fix lint and formatting issues

f398718

fix: update __lazy_modules__ targets to match import statements

bb5c39c

fix: typo aiohttp_requests to _aiohttp_requests

a3a85a0

fix: add type hint to __lazy_modules__

68eca51

chore: remove accidentally committed scratch files

24ac419

chore: fix import order in __init__.py to satisfy flake8

eef11dc

hebaalazzeh force-pushed the feature/lazy-imports-google-auth branch from 6e6fe6e to eef11dc Compare July 10, 2026 06:43

hebaalazzeh added 5 commits July 10, 2026 06:48

test: rewrite lazy imports test using subprocess to prevent test poll…

73ac7e8

…ution

chore: run black on test_lazy_imports.py

b528945

refactor: apply PEP 810 directly to transport modules instead of __in…

cfde494

…it__ to fully address parthea's feedback

chore: run black on transport modules

bcb18a4

fix: flake8 import order errors for typing.Set

3eb6523

hebaalazzeh added 2 commits July 10, 2026 07:15

fix: remaining flake8 errors in urllib3.py

35e8bdc

chore: revert DEFAULT_PYTHON_VERSION to 3.14 in noxfile.py

5503252

parthea assigned parthea and unassigned hebaalazzeh Jul 13, 2026

gemini-code-assist Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread packages/google-auth/google/auth/transport/grpc.py

Comment thread packages/google-auth/google/auth/transport/urllib3.py

parthea reviewed Jul 13, 2026

View reviewed changes

parthea assigned hebaalazzeh and unassigned parthea Jul 13, 2026

fix: defer urllib3 and grpc eager loading in transport modules

c4a53ce

gemini-code-assist Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread packages/google-auth/google/auth/transport/urllib3.py

Comment thread packages/google-auth/google/auth/transport/requests.py

Comment thread packages/google-auth/google/auth/transport/grpc.py

hebaalazzeh added 3 commits July 13, 2026 18:23

chore: add PEP 0810 comment above __lazy_modules__

94ba2c2

fix: address code review comments for lazy imports

a1e2d61

fix: address lint issue in requests.py import order

7ae442e

gemini-code-assist Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread packages/google-auth/google/auth/transport/urllib3.py

ohmayr reviewed Jul 13, 2026

View reviewed changes

		class AuthorizedHttp(object):
		"""A urllib3 HTTP class with credentials.

		__lazy_modules__: Set[str] = {"requests", "requests.adapters", "requests.exceptions"}

Uh oh!

Conversation

hebaalazzeh commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Documents

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

parthea left a comment

Choose a reason for hiding this comment

Uh oh!

parthea commented Jul 9, 2026

Uh oh!

parthea commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

hebaalazzeh commented Jul 10, 2026

Uh oh!

parthea commented Jul 13, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

parthea left a comment

Choose a reason for hiding this comment

Uh oh!

hebaalazzeh commented Jul 13, 2026

Uh oh!

parthea commented Jul 13, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hebaalazzeh commented Jul 13, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

ohmayr Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

ohmayr Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

ohmayr Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

ohmayr Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

ohmayr Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

ohmayr Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hebaalazzeh commented Jul 9, 2026 •

edited

Loading

parthea commented Jul 9, 2026 •

edited

Loading