Add python 3.13 support #6172
Open
Add python 3.13 support #6172
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds comprehensive Python 3.13 support to MediaPipe: ## Core Changes - Add Python 3.13 to hermetic Python configuration in WORKSPACE - Update requirements to Python 3.13 compatible versions: - JAX 0.5.3 (required for Python 3.13) - protobuf 5.29.5 (maintain v5.x compatibility) - scipy 1.16.3 - Add requirements_lock_3_13.txt for reproducible builds ## Build System - Update setup.py to version 0.10.14 - Fix module imports in setup.py (__init__.py generation) - Configure system OpenCV 4.12 for macOS (via Homebrew) - Update OPENCV_SO_VERSION to 4.12 in third_party/BUILD - Add macOS linker flags (-undefined dynamic_lookup) for Python extensions ## Code Fixes - Fix absl::MutexLock pointer/reference issue in calculator_graph.cc - Fix TextEmbedderResult import in tasks/python/text/__init__.py - Fix GestureRecognizerResult import in tasks/python/vision/__init__.py - Remove holistic_landmarker references (deleted in upstream commit 0c6369d) ## Dependencies - Disable FFmpeg to avoid compatibility issues with FFmpeg 8.0+ - Disable internet-dependent OpenCV modules (gapi, ADE, ObSensor) - Document OpenCV 4 setup in third_party/opencv_macos.BUILD ## CI/CD - Add GitHub Actions workflow for building wheels - Support Python 3.9, 3.10, 3.11, 3.12, 3.13 - Automatic version generation for dev builds - Wheel artifacts retained for 90 days ## Build Artifacts - Update .gitignore for Python build artifacts and build.log Tested: Successfully builds Python 3.13 wheel on macOS ARM64 Build time: ~11 minutes with system OpenCV
Switch to bazelisk for automatic Bazel version management. Bazelisk reads .bazelversion and downloads the correct Bazel version, avoiding version conflicts and ensuring consistency with local builds.
Regenerated all requirements lock files using pip-compile for consistency
across Python 3.9, 3.10, 3.11, 3.12, and 3.13.
Key changes:
1. opencv-contrib-python: 4.10.0.84 -> 4.12.0.88 (Python 3.13 compatibility)
- Version 4.10.0.84 did not have Python 3.13 wheels, causing builds
to fail when trying to build from source due to numpy incompatibility
- Version 4.12.0.88 provides cp37-abi3 wheels compatible with Python 3.7+
including Python 3.13
2. protobuf: 4.25.5 -> 5.29.5 (for Python 3.10, 3.11, 3.12)
- Ensures compatibility with protobuf@29 (v5.29.x) compiler from Homebrew
- All lock files now use consistent protobuf version 5.29.5
- Updated requirements.txt lower bound to >=5.29.3
The CI tests were failing with "AttributeError: module 'mediapipe' has no attribute '__version__'" because Python was importing from the source directory instead of the installed package in site-packages. When running tests from within the source directory, Python's import system prioritizes the local mediapipe/ directory over the installed package. The source directory's __init__.py only contains the copyright header, while the installed package's __init__.py has __version__ appended by setup.py during the build. Solution: Change to /tmp before running import tests to ensure Python imports from the installed package rather than the source directory.
Since all Python versions (3.9-3.13) now have lock files, we can remove the conditional check and directly use the lock files.
- Removed incorrectly named lock files with dots (3.9, 3.10, etc.) - Added requirements_lock_3_9.txt with correct underscore naming - Updated CI workflow to convert python-version format from dots to underscores when loading lock files (e.g., "3.9" -> "3_9") All lock files now follow the consistent naming pattern: requirements_lock_3_9.txt, requirements_lock_3_10.txt, etc.
Changed caching strategy to follow GitHub Actions best practices: - Cache only Homebrew package downloads (~//Library/Caches/Homebrew) instead of installed Cellar directories - Add HOMEBREW_NO_AUTO_UPDATE=1 for faster installs - Remove manual brew link command (no longer needed) This fixes the protobuf@29 linking issue where packages restored from Cellar cache were not properly linked.
Changed __version__ from '0.10.14' to 'dev' as a temporary workaround. The CI workflow already updates this value during the build process using sed to inject the proper version.
This file is replaced by version-specific lock files: requirements_lock_3_9.txt, requirements_lock_3_10.txt, etc.
Fixed references from requirements_lock.txt to requirements_lock_3_9.txt: - Updated python_init_repositories to use requirements_lock_3_9.txt for Python 3.9 - Updated mediapipe_pip_deps to use requirements_lock_3_9.txt - Updated third_party/BUILD exports_files This fixes the build error where dist/ directory was not created because the build couldn't find requirements_lock.txt.
- Updated requirements.txt to use sentencepiece>=0.2.1 for Python 3.13 (0.2.0 fails to build on Python 3.13 due to CMake compatibility issues) - Regenerated requirements_lock_3_13.txt with correct dependencies: - sentencepiece==0.2.1 - opencv-contrib-python==4.12.0.88 - numpy==2.2.6 (Python 3.13 supports numpy 2.x)
Reverted cmake build configuration changes in third_party/BUILD: - OPENCV_SHARED_LIBS = True (instead of False) - OPENCV_SO_VERSION = "3.4" (instead of "4.12") - Removed gapi, ADE, ObSensor, FFmpeg disable flags - Removed out_include_dir = "include/opencv4" - Fixed dylib version pattern (removed .0 suffix) These changes were for building OpenCV from source, but we use the system OpenCV from Homebrew on macOS, so the source build configuration is not used for our wheel builds.
Reverted the opencv http_archive in WORKSPACE from version 4.10.0 back to 3.4.11. This archive is only used when building OpenCV from source, which we don't do for macOS wheel builds (we use system OpenCV from Homebrew via macos_opencv repository). Kept the macos_opencv path as /opt/homebrew/Cellar which is correct for Apple Silicon Macs.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
fangchenli
force-pushed
the
add-python-3.13-support
branch
3 times, most recently
from
fbfb503 to
e72329c
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cannot build wheel for Python 3.9 since amatchstatement was introduced here daa7190#diff-bbe623d0da30bd71b65bf323f986a65b8246295527ea972ff092b1110258139eR85.The GHA job demonstrates it works for Python 3.9 - 3.13 on ARM macOS.
close #5943, close #5708, close #5812, close #5965, close #5985
The changes in this PR are probably beyond the scope of the community, and it likely won't get merged. Just leave it here as a reference.