Is your feature request related to a problem? Please describe.
When cuDF enables Arrow Parquet, cpp/cmake/thirdparty/get_arrow.cmake selects Thrift_SOURCE BUNDLED. Arrow then downloads the Thrift source archive during its nested CMake configure. That download currently depends on Arrow’s default upstream URLs unless every caller constructs and supplies the complete versioned ARROW_THRIFT_URL itself.
This makes cuDF builds unnecessarily vulnerable to upstream availability and prevents CI systems from configuring a stable authenticated mirror using only a mirror base URL. rapidsai/cudf#23409 is an example of an affected PR: its build was blocked while Arrow’s bundled Thrift archive could not be fetched reliably.
Describe the solution you would like
Add generic mirror plumbing alongside cuDF’s Arrow dependency setup:
- Accept an environment variable such as
ARROW_THRIFT_MIRROR_URL containing the mirror base URL.
- During the Arrow project configure, read
ARROW_THRIFT_BUILD_VERSION from Arrow’s thirdparty/versions.txt and derive the versioned archive URL, for example ${ARROW_THRIFT_MIRROR_URL}/thrift/<version>/thrift-<version>.tar.gz.
- Set Arrow’s supported
ARROW_THRIFT_URL override only when the caller has not already supplied ARROW_THRIFT_URL, so the explicit full URL remains authoritative.
- Leave existing behavior unchanged when no mirror variable is defined.
A CMake project hook (for example, a helper installed through CMAKE_PROJECT_arrow_INCLUDE from get_arrow.cmake) can run after Arrow establishes its source directory and before it resolves bundled dependencies.
Authentication remains the responsibility of each build environment: callers can configure CMAKE_NETRC and CMAKE_NETRC_FILE as appropriate. cuDF would own only the reusable Arrow/Thrift URL construction.
Describe alternatives you have considered
- Have every downstream repository parse Arrow’s version file and install its own project hook. This duplicates Arrow-specific dependency logic and can drift across cuDF consumers.
- Hard-code the complete Thrift URL in CI. This duplicates the Thrift version and must be updated whenever Arrow changes it.
- Rely only on Arrow’s public fallback URLs. This does not address transient upstream failures or authenticated enterprise mirrors.
Additional context
The Java JNI build enables Arrow Parquet through cuDF’s get_arrow.cmake, so placing the generic mechanism there would cover cuDF itself and downstream builds that configure java/src/main/native.
Is your feature request related to a problem? Please describe.
When cuDF enables Arrow Parquet,
cpp/cmake/thirdparty/get_arrow.cmakeselectsThrift_SOURCE BUNDLED. Arrow then downloads the Thrift source archive during its nested CMake configure. That download currently depends on Arrow’s default upstream URLs unless every caller constructs and supplies the complete versionedARROW_THRIFT_URLitself.This makes cuDF builds unnecessarily vulnerable to upstream availability and prevents CI systems from configuring a stable authenticated mirror using only a mirror base URL. rapidsai/cudf#23409 is an example of an affected PR: its build was blocked while Arrow’s bundled Thrift archive could not be fetched reliably.
Describe the solution you would like
Add generic mirror plumbing alongside cuDF’s Arrow dependency setup:
ARROW_THRIFT_MIRROR_URLcontaining the mirror base URL.ARROW_THRIFT_BUILD_VERSIONfrom Arrow’sthirdparty/versions.txtand derive the versioned archive URL, for example${ARROW_THRIFT_MIRROR_URL}/thrift/<version>/thrift-<version>.tar.gz.ARROW_THRIFT_URLoverride only when the caller has not already suppliedARROW_THRIFT_URL, so the explicit full URL remains authoritative.A CMake project hook (for example, a helper installed through
CMAKE_PROJECT_arrow_INCLUDEfromget_arrow.cmake) can run after Arrow establishes its source directory and before it resolves bundled dependencies.Authentication remains the responsibility of each build environment: callers can configure
CMAKE_NETRCandCMAKE_NETRC_FILEas appropriate. cuDF would own only the reusable Arrow/Thrift URL construction.Describe alternatives you have considered
Additional context
The Java JNI build enables Arrow Parquet through cuDF’s
get_arrow.cmake, so placing the generic mechanism there would cover cuDF itself and downstream builds that configurejava/src/main/native.