Fix: Enable manylinux2014 builds for Google Colab compatibility#6
Closed
andylizf wants to merge 22 commits into
Closed
Fix: Enable manylinux2014 builds for Google Colab compatibility#6andylizf wants to merge 22 commits into
andylizf wants to merge 22 commits into
Conversation
- Add Python_FIND_VIRTUALENV hints to pyproject.toml for CMake - Create standardized cibuildwheel workflow using manylinux_2_28 - Simplify system dependency installation - Add global cibuildwheel configuration in root pyproject.toml - Create streamlined test workflow for manylinux compatibility
- Remove Windows support as not needed - Move Python finding hints to cibuildwheel environment variables - Keep pyproject.toml clean to avoid breaking normal builds - Target manylinux_2_28 for better Colab compatibility
- Switch from manylinux_2_28 to manylinux2014 (provides manylinux_2_17) - This should produce wheels compatible with manylinux_2_35_x86_64 requirement - Update package manager from dnf to yum for CentOS 7 - Use cmake3 with symlink for compatibility
- Add libomp for macOS to fix OpenMP linking error - Add python3-devel for Linux (though may not be needed in manylinux) - Install numpy before building to satisfy faiss CMake requirements - Add before-build configuration to cibuildwheel
- Modify faiss CMakeLists.txt to try both FindPython and FindPython3 - Add scikit-build configuration to help with Python detection - Simplify Linux environment variables in cibuildwheel - Add CMake Python detection before faiss configuration
- Set FAISS_ENABLE_PYTHON to OFF since we use our own Cython bindings - This avoids the CMake Python finding issues in manylinux environments - Simplify CMakeLists.txt by removing unnecessary Python finding logic - Keep swig installation for other potential uses
- cmake.verbose is not allowed when minimum-version is set to 0.10 or higher - This was causing build failures in cibuildwheel
- Re-enable FAISS_ENABLE_PYTHON since we need the Python bindings - Use Development.Module component for better compatibility - Pass Python information to Faiss through CMake cache variables - Add CMAKE_PREFIX_PATH= to help CMake find Python in manylinux
- Use Development.Module instead of Development component - Find Python in main Faiss CMakeLists.txt before python subdirectory - Add debug output to trace Python variable passing - Set Python_FIND_VIRTUALENV=ONLY for Faiss
- Set MACOSX_DEPLOYMENT_TARGET=11.0 for macOS builds - Add pyzmq to test requirements to use pre-built wheels - Configure deployment target in both workflow and pyproject.toml - Skip ARM64 tests on GitHub Actions to avoid cross-compilation issues
- Cap pyzmq version to <27 for manylinux2014 compatibility - Pre-install pyzmq binary wheel before tests using CIBW_BEFORE_TEST - Force pip to use only binary wheels with --only-binary :all:
- Remove macOS from build matrix to simplify CI - Keep macOS configurations for future reference but they won't be used - This speeds up CI and focuses on the primary target (Colab/Linux)
- Add explicit Python finding logic to DiskANN CMakeLists.txt - Change Development to Development.Module to avoid Embed requirement - Pass Python variables through CMake cache to submodules
- Change from src/third_party/DiskANN to third_party/DiskANN - This was causing CMake to fail finding the subdirectory
- Replace all Chinese comments with English - Ensure code is internationalization-friendly
- Remove duplicate ci-cibuildwheel.yml workflow to avoid confusion - Fix DiskANN CMakeLists.txt to support both standard and manylinux builds - Try Development.Module first (manylinux), fallback to Development (standard) - Keep build-reusable.yml and build-cibuildwheel.yml as separate workflows - They build different wheel types and are selected at release time
- Point to fix/python-finding-compatibility branch - Support both Development and Development.Module for Python finding - Fixes compatibility with both standard and manylinux builds
andylizf
force-pushed
the
fix/manylinux-compatibility
branch
from
July 25, 2025 20:33
9e20c1b to
02672c0
Compare
- Use manylinux_2_35 (GLIBC 2.35) instead of manylinux2014 (GLIBC 2.17) - Still compatible with Google Colab (requires ≤2.35) - Benefits: newer toolchain, better performance, modern C++ features - Switch from yum to dnf package manager - Remove pyzmq version cap as manylinux_2_35 has newer ZeroMQ - Update documentation to reflect the change
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes the wheel building process to support Google Colab environments by using manylinux2014 instead of manylinux_2_28.
Key Changes
1. manylinux2014 Compatibility
2. Python Finding Improvements
3. pyzmq Compatibility
4. FileNotFoundError Fix
5. CI/Build Improvements
Testing
Notes
Fixes issues with Google Colab compatibility reported by users.