Skip to content

Commit 1b5d48f

Browse files
sunyabpixar-oss
sunyab
authored and
pixar-oss
committed
build: Install Python modules to standard site-packages directory
By default the build now installs Python modules to the standard site-packages directory for the version of Python used to build OpenUSD. This directory varies depending on platform and version. Typical locations are: Linux/MacOS: - <inst>/lib/pythonX.Y/site-packages - <inst>/lib64/pythonX.Y/site-packages Windows: - <inst>/Lib/site-packages Users can override this by setting the PXR_PYTHON_INSTALL_DIR parameter to either an absolute path or a path relative to CMAKE_INSTALL_PREFIX when running CMake. This is also available when running build_usd.py via the --python-install-dir option. Setting this to "lib/python" will restore the previous install layout. Docs and tutorials were updated to reflect this change. Info about the install directory and PYTHONPATH setup were centralized in the "Python" section in BUILDING.md, with other docs updated to refer there. Fixes PixarAnimationStudios#30 (Internal change: 2414602)
1 parent a539ac6 commit 1b5d48f

13 files changed

Lines changed: 194 additions & 59 deletions

File tree

‎.github/workflows/pypi.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ jobs:
8484
# PXR_PY_UNDEFINED_DYNAMIC_LOOKUP flag will ensure that we never try to
8585
# link against this library anyway, so it doesn't matter.
8686
docker exec usdmanylinux ${{ matrix.PYTHON.INTERPRETER }} -c "import pathlib,sysconfig; pathlib.Path(sysconfig.get_config_var('LIBDIR'), sysconfig.get_config_var('LDLIBRARY')).touch()"
87-
docker exec usdmanylinux ${{ matrix.PYTHON.INTERPRETER }} build_scripts/build_usd.py --build-monolithic --build-args USD,"-DPXR_PY_UNDEFINED_DYNAMIC_LOOKUP=ON -DPXR_BUILD_USD_TOOLS=OFF -DPXR_BUILD_EXEC=OFF -DPXR_INSTALL_LOCATION=../pxr/pluginfo" --no-materialx --no-imaging --no-examples --no-tutorials --build /opt/USD/gen/build --src /opt/USD/gen/src /opt/USD/inst -v
87+
docker exec usdmanylinux ${{ matrix.PYTHON.INTERPRETER }} build_scripts/build_usd.py --build-monolithic --build-args USD,"-DPXR_PY_UNDEFINED_DYNAMIC_LOOKUP=ON -DPXR_BUILD_USD_TOOLS=OFF -DPXR_BUILD_EXEC=OFF -DPXR_INSTALL_LOCATION=../pxr/pluginfo -DPXR_PYTHON_INSTALL_DIR=lib/python" --no-materialx --no-imaging --no-examples --no-tutorials --build /opt/USD/gen/build --src /opt/USD/gen/src /opt/USD/inst -v
8888
- name: Creating packaging directory
8989
run: |
9090
docker exec usdmanylinux mkdir ./packaging
@@ -153,7 +153,7 @@ jobs:
153153
- name: Build USD
154154
run: |
155155
sudo xcode-select -s /Applications/Xcode_16.app/Contents/Developer
156-
${{ matrix.PYTHON.INTERPRETER }} build_scripts/build_usd.py --build-monolithic --build-args USD,"-DPXR_PY_UNDEFINED_DYNAMIC_LOOKUP=ON -DPXR_BUILD_USD_TOOLS=OFF -DPXR_BUILD_EXEC=OFF -DPXR_INSTALL_LOCATION=../pluginfo" --no-materialx --no-imaging --no-examples --no-tutorials --generator Xcode --build-target universal --build $GITHUB_WORKSPACE/USDgen/build --src $GITHUB_WORKSPACE/USDgen/src $GITHUB_WORKSPACE/USDinst -v
156+
${{ matrix.PYTHON.INTERPRETER }} build_scripts/build_usd.py --build-monolithic --build-args USD,"-DPXR_PY_UNDEFINED_DYNAMIC_LOOKUP=ON -DPXR_BUILD_USD_TOOLS=OFF -DPXR_BUILD_EXEC=OFF -DPXR_INSTALL_LOCATION=../pluginfo -DPXR_PYTHON_INSTALL_DIR=lib/python" --no-materialx --no-imaging --no-examples --no-tutorials --generator Xcode --build-target universal --build $GITHUB_WORKSPACE/USDgen/build --src $GITHUB_WORKSPACE/USDgen/src $GITHUB_WORKSPACE/USDinst -v
157157
- name: Packaging USD
158158
run: |
159159
pwd
@@ -229,7 +229,7 @@ jobs:
229229
call C:\"Program Files (x86)"\"Microsoft Visual Studio"\2022\Enterprise\VC\Auxiliary\Build\vcvars64.bat
230230
set BOOST_ROOT=
231231
python --version
232-
python build_scripts/build_usd.py --build-monolithic --build-args USD,"-DPXR_ENABLE_PRECOMPILED_HEADERS=OFF -DPXR_PY_UNDEFINED_DYNAMIC_LOOKUP=ON -DPXR_BUILD_USD_TOOLS=OFF -DPXR_BUILD_EXEC=OFF -DPXR_INSTALL_LOCATION=../pxr/pluginfo" --no-materialx --no-imaging --no-examples --no-tutorials --build USDgen/build --src USDgen/src USDinst -v
232+
python build_scripts/build_usd.py --build-monolithic --build-args USD,"-DPXR_ENABLE_PRECOMPILED_HEADERS=OFF -DPXR_PY_UNDEFINED_DYNAMIC_LOOKUP=ON -DPXR_BUILD_USD_TOOLS=OFF -DPXR_BUILD_EXEC=OFF -DPXR_INSTALL_LOCATION=../pxr/pluginfo -DPXR_PYTHON_INSTALL_DIR=lib/python" --no-materialx --no-imaging --no-examples --no-tutorials --build USDgen/build --src USDgen/src USDinst -v
233233
shell: cmd
234234
- name: Packaging USD
235235
run: |

‎BUILDING.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,32 @@ Please refer to [VERSIONS.md](VERSIONS.md) for supported Python versions.
134134
Support for Python can optionally be disabled by specifying the cmake flag
135135
`PXR_ENABLE_PYTHON_SUPPORT=FALSE`.
136136

137+
By default, Python bindings are installed into the standard Python site-packages
138+
directory under `CMAKE_INSTALL_PREFIX`. This directory varies depending on the
139+
platform and version of Python used to build USD. Typical locations are:
140+
141+
- Linux/MacOS:
142+
- `<prefix>/lib/pythonX.Y/site-packages`
143+
- `<prefix>/lib64/pythonX.Y/site-packages`
144+
145+
- Windows:
146+
- `<prefix>/Lib/site-packages`
147+
148+
To install into a custom directory instead, set `PXR_PYTHON_INSTALL_DIR` to a
149+
path relative to `CMAKE_INSTALL_PREFIX` or to an absolute path. For example,
150+
to restore the layout used by older USD releases:
151+
152+
```
153+
-DPXR_PYTHON_INSTALL_DIR=lib/python
154+
```
155+
156+
Python must be set up to discover the bindings in order to use them and the
157+
components that depend on them. This is typically done by adding the install
158+
directory to the `PYTHONPATH` environment variable. Note that with the default
159+
settings, no additional configuration is required if USD is built into a
160+
virtualenv (i.e., `CMAKE_INSTALL_PREFIX` is set to the virtual environment's
161+
root directory).
162+
137163
##### OpenGL
138164

139165
Support for OpenGL can optionally be disabled by specifying the cmake flag

‎build_scripts/build_usd.py‎

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -399,8 +399,21 @@ def AppendCXX11ABIArg(buildFlag, context, buildArgs):
399399
buildArgs.append('{flag}="{flags}"'.format(
400400
flag=buildFlag, flags=" ".join(cxxFlags)))
401401

402+
def GetCMakeCacheValue(buildDir, variable):
403+
"""Return the value of a CMake cache variable by querying the build
404+
directory with 'cmake -N -LA <buildDir>', or None on failure."""
405+
output = GetCommandOutput(
406+
"cmake -N -LA {}".format(shlex.quote(buildDir)))
407+
if output is None:
408+
return None
409+
for line in output.splitlines():
410+
m = re.match(r"{}(?::[A-Z]+)?=(.*)".format(re.escape(variable)), line)
411+
if m:
412+
return m.group(1).strip()
413+
return None
414+
402415
def RunCMake(context, force, extraArgs = None, installDir = None):
403-
"""Invoke CMake to configure, build, and install a library whose
416+
"""Invoke CMake to configure, build, and install a library whose
404417
source code is located in the current working directory."""
405418
# Create a directory for out-of-source builds in the build directory
406419
# using the name of the current working directory.
@@ -1757,6 +1770,10 @@ def InstallUSD(context, force, buildArgs):
17571770
if context.buildPython:
17581771
extraArgs.append('-DPXR_ENABLE_PYTHON_SUPPORT=ON')
17591772

1773+
if context.pythonInstallDir is not None:
1774+
extraArgs.append('-DPXR_PYTHON_INSTALL_DIR={}'
1775+
.format(context.pythonInstallDir))
1776+
17601777
# Many people on Windows may not have Python libraries with debug
17611778
# symbols (denoted by a '_d') installed. This is the common
17621779
# case when a user installs Python from the official download
@@ -2212,6 +2229,12 @@ def InstallUSD(context, force, buildArgs):
22122229
"Don't define Boost Python Debug if your Python "
22132230
"library comes with Debugging symbols.")
22142231

2232+
group.add_argument("--python-install-dir", type=str,
2233+
dest="python_install_dir", default=None,
2234+
help=("Directory to install USD Python bindings, relative "
2235+
"to the install prefix or absolute. Defaults to the "
2236+
"Python site-packages directory."))
2237+
22152238
(NO_IMAGING, IMAGING, USD_IMAGING) = (0, 1, 2)
22162239

22172240
group = parser.add_argument_group(title="Imaging and USD Imaging Options")
@@ -2436,9 +2459,11 @@ def __init__(self, args):
24362459

24372460
# Optional components
24382461
self.buildTests = (args.build_tests and not embedded)
2439-
self.buildPython = (args.build_python and
2440-
not embedded and
2462+
self.buildPython = (args.build_python and
2463+
not embedded and
24412464
not self.targetWasm)
2465+
2466+
self.pythonInstallDir = args.python_install_dir
24422467
self.buildExamples = (args.build_examples and
24432468
not embedded)
24442469
self.buildTutorials = (args.build_tutorials and
@@ -2951,8 +2976,13 @@ def FormatBuildArguments(buildArgs):
29512976
sys.exit(1)
29522977

29532978
# Done. Print out a final status message.
2979+
usdCMakeBuildDir = os.path.join(context.buildDir,
2980+
os.path.basename(context.usdSrcDir))
2981+
pythonInstallDir = GetCMakeCacheValue(usdCMakeBuildDir, "PXR_PYTHON_INSTALL_DIR")
29542982
requiredInPythonPath = set([
2955-
os.path.join(context.usdInstDir, "lib", "python")
2983+
os.path.join(context.usdInstDir, pythonInstallDir)
2984+
if pythonInstallDir is not None else
2985+
"<unknown: could not read PXR_PYTHON_INSTALL_DIR from CMake cache>"
29562986
])
29572987
requiredInPythonPath.update(extraPythonPaths)
29582988

‎cmake/defaults/Options.cmake‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,14 @@ set(PXR_INSTALL_LOCATION ""
105105
"Intended final location for plugin resource files."
106106
)
107107

108+
set(PXR_PYTHON_INSTALL_DIR ""
109+
CACHE
110+
STRING
111+
"Directory for installing Python bindings (relative to \
112+
CMAKE_INSTALL_PREFIX or absolute). Defaults to the Python \
113+
site-packages directory."
114+
)
115+
108116
set(PXR_OVERRIDE_PLUGINPATH_NAME ""
109117
CACHE
110118
STRING
@@ -270,4 +278,4 @@ set(PXR_COMPILER_CACHE_NAME "ccache"
270278
CACHE
271279
STRING
272280
"The name of the compiler cache program to use"
273-
)
281+
)

‎cmake/defaults/Packages.cmake‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,37 @@ if(PXR_ENABLE_PYTHON_SUPPORT)
7979
# USD builds only work with Python3
8080
setup_python_package(Python3)
8181

82+
# Compute the default Python bindings install directory from the Python
83+
# interpreter if the user has not provided an explicit value. Note that
84+
# we intentionally do not use Python3_SITEARCH since that may be an
85+
# absolute path pointing outside of our install prefix. We should allow
86+
# such paths only if they are given by the user.
87+
if(NOT PXR_PYTHON_INSTALL_DIR)
88+
execute_process(
89+
COMMAND "${PYTHON_EXECUTABLE}"
90+
"${PROJECT_SOURCE_DIR}/cmake/macros/getPythonInstallDir.py"
91+
OUTPUT_STRIP_TRAILING_WHITESPACE
92+
OUTPUT_VARIABLE _pxr_default_python_install_dir
93+
RESULT_VARIABLE _pxr_sysconfig_rc
94+
)
95+
if(_pxr_sysconfig_rc EQUAL 0 AND _pxr_default_python_install_dir)
96+
set(PXR_PYTHON_INSTALL_DIR "${_pxr_default_python_install_dir}"
97+
CACHE STRING
98+
"Directory for installing Python bindings." FORCE)
99+
else()
100+
set(PXR_PYTHON_INSTALL_DIR "lib/python"
101+
CACHE STRING
102+
"Directory for installing Python bindings." FORCE)
103+
message(WARNING
104+
"Could not detect Python site-packages path,"
105+
"defaulting PXR_PYTHON_INSTALL_DIR to lib/python")
106+
endif()
107+
unset(_pxr_default_python_install_dir)
108+
unset(_pxr_sysconfig_rc)
109+
endif()
110+
111+
message(STATUS "Installing Python bindings to ${PXR_PYTHON_INSTALL_DIR}")
112+
82113
# --Jinja2
83114
find_package(Jinja2)
84115
else()

‎cmake/macros/Private.cmake‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -180,7 +180,7 @@ function(_install_python LIBRARY_NAME)
180180
${ARGN}
181181
)
182182

183-
set(libPythonPrefix lib/python)
183+
set(libPythonPrefix "${PXR_PYTHON_INSTALL_DIR}")
184184
_get_python_module_name(${LIBRARY_NAME} LIBRARY_INSTALLNAME)
185185

186186
set(files_copied "")
@@ -219,10 +219,11 @@ function(_install_python LIBRARY_NAME)
219219
list(APPEND filesToInstall ${CMAKE_CURRENT_SOURCE_DIR}/${file})
220220
endif()
221221

222-
# Note that we always install under lib/python/pxr, even if we are in
223-
# the third_party project. This means the import will always look like
224-
# 'from pxr import X'. We need to do this per-loop iteration because
225-
# the installDest may be different due to the presence of subdirs.
222+
# Note that we always install under ${PXR_PYTHON_INSTALL_DIR}/pxr, even
223+
# if we are in the third_party project. This means the import will
224+
# always look like 'from pxr import X'. We need to do this per-loop
225+
# iteration because the installDest may be different due to the
226+
# presence of subdirs.
226227
install(
227228
FILES ${filesToInstall}
228229
DESTINATION "${installDest}"
@@ -394,7 +395,7 @@ function(_install_pyside_ui_files LIBRARY_NAME)
394395
FOLDER "${folder}"
395396
)
396397

397-
set(libPythonPrefix lib/python)
398+
set(libPythonPrefix "${PXR_PYTHON_INSTALL_DIR}")
398399
_get_python_module_name(${LIBRARY_NAME} LIBRARY_INSTALLNAME)
399400

400401
install(
@@ -1024,7 +1025,7 @@ function(_pxr_python_module NAME)
10241025
# 'from pxr import X'. Additionally, python libraries always install
10251026
# into the default lib install, not into the third_party subdirectory
10261027
# or similar.
1027-
set(libInstallPrefix "lib/python/pxr/${pyModuleName}")
1028+
set(libInstallPrefix "${PXR_PYTHON_INSTALL_DIR}/pxr/${pyModuleName}")
10281029

10291030
# Python modules need to be able to access their corresponding
10301031
# wrapped library and the install lib directory.

‎cmake/macros/Public.cmake‎

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -478,7 +478,7 @@ function(pxr_setup_python)
478478
string(REPLACE ";" ", " pyModulesStr "${converted}")
479479

480480
# Install a pxr __init__.py with an appropriate __all__
481-
_get_install_dir(lib/python/pxr installPrefix)
481+
_get_install_dir(${PXR_PYTHON_INSTALL_DIR}/pxr installPrefix)
482482

483483
file(WRITE "${CMAKE_CURRENT_BINARY_DIR}/generated_modules_init.py"
484484
"__all__ = [${pyModulesStr}]\n")
@@ -1027,7 +1027,11 @@ function(pxr_register_test TEST_NAME)
10271027
# Ensure that Python imports the Python files built by this build.
10281028
# On Windows convert backslash to slash and don't change semicolons
10291029
# to colons.
1030-
set(_testPythonPath "${CMAKE_INSTALL_PREFIX}/lib/python;$ENV{PYTHONPATH}")
1030+
if(IS_ABSOLUTE "${PXR_PYTHON_INSTALL_DIR}")
1031+
set(_testPythonPath "${PXR_PYTHON_INSTALL_DIR};$ENV{PYTHONPATH}")
1032+
else()
1033+
set(_testPythonPath "${CMAKE_INSTALL_PREFIX}/${PXR_PYTHON_INSTALL_DIR};$ENV{PYTHONPATH}")
1034+
endif()
10311035
if(WIN32)
10321036
string(REGEX REPLACE "\\\\" "/" _testPythonPath "${_testPythonPath}")
10331037
else()
@@ -1354,15 +1358,22 @@ endfunction() # pxr_tests_prologue
13541358

13551359
function(pxr_build_python_documentation)
13561360
set(BUILT_XML_DOCS "${PROJECT_BINARY_DIR}/docs/doxy_xml")
1357-
set(CONVERT_DOXYGEN_TO_PYTHON_DOCS_SCRIPT
1361+
set(CONVERT_DOXYGEN_TO_PYTHON_DOCS_SCRIPT
13581362
"${PROJECT_SOURCE_DIR}/docs/python/convertDoxygen.py")
1359-
set(INSTALL_PYTHON_PXR_ROOT "${CMAKE_INSTALL_PREFIX}/lib/python/pxr")
1363+
1364+
# Compute the absolute path to the Python bindings install directory.
1365+
if(IS_ABSOLUTE "${PXR_PYTHON_INSTALL_DIR}")
1366+
set(_pythonInstallDirAbs "${PXR_PYTHON_INSTALL_DIR}")
1367+
else()
1368+
set(_pythonInstallDirAbs "${CMAKE_INSTALL_PREFIX}/${PXR_PYTHON_INSTALL_DIR}")
1369+
endif()
1370+
set(INSTALL_PYTHON_PXR_ROOT "${_pythonInstallDirAbs}/pxr")
13601371

13611372
# Get the list of pxr python modules and run a install command for each
13621373
get_property(pxrPythonModules GLOBAL PROPERTY PXR_PYTHON_MODULES)
13631374
# Create string of module names, joined with ","
13641375
string(REPLACE ";" "," pxrPythonModulesStr "${pxrPythonModules}")
1365-
# Run convertDoxygen on the module list, setting PYTHONPATH
1376+
# Run convertDoxygen on the module list, setting PYTHONPATH
13661377
# to the install path for the USD Python modules
13671378
if (WIN32)
13681379
set(DLL_PATH_FLAG "--dllPath \"${CMAKE_INSTALL_PREFIX}/lib;${CMAKE_INSTALL_PREFIX}/bin;${CMAKE_INSTALL_PREFIX}/plugin/usd;${CMAKE_INSTALL_PREFIX}/share/usd/examples/plugin\"")
@@ -1376,7 +1387,7 @@ function(pxr_build_python_documentation)
13761387
COMMAND ${PYTHON_EXECUTABLE} ${CONVERT_DOXYGEN_TO_PYTHON_DOCS_SCRIPT} \
13771388
--package pxr --module ${pxrPythonModulesStr} \
13781389
--inputIndex ${BUILT_XML_DOCS}/index.xml \
1379-
--pythonPath ${CMAKE_INSTALL_PREFIX}/lib/python \
1390+
--pythonPath ${_pythonInstallDirAbs} \
13801391
${DLL_PATH_FLAG} \
13811392
--output ${INSTALL_PYTHON_PXR_ROOT})
13821393
if (NOT \${convert_doxygen_return_code} EQUAL \"0\")

‎cmake/macros/getPythonInstallDir.py‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
#
2+
# Copyright 2026 Pixar
3+
#
4+
# Licensed under the terms set forth in the LICENSE.txt file available at
5+
# https://openusd.org/license.
6+
#
7+
8+
# Prints the platform-specific Python site-packages directory where the USD
9+
# Python modules will be installed by default. This is a relative path,
10+
# e.g. "lib/python3.9/site-packages", that will be anchored to the library
11+
# install prefix.
12+
13+
import pathlib
14+
import sys
15+
import sysconfig
16+
17+
sitePackagePath = None
18+
19+
# Use the "platlib" path relative to sys.exec_prefix. These are where
20+
# packages that are not pure Python are meant to be installed and
21+
# should handle most cases, including installing into a virtualenv.
22+
#
23+
# For reference:
24+
# https://packaging.python.org/en/latest/specifications/binary-distribution-format/#what-s-the-deal-with-purelib-vs-platlib
25+
# https://discuss.python.org/t/understanding-site-packages-directories/12959
26+
try:
27+
sitePackagePath = pathlib.PurePath(
28+
sysconfig.get_path("platlib")).relative_to(sys.exec_prefix)
29+
except ValueError:
30+
# If the "platlib" path is not relative to sys.exec_prefix, fallback
31+
# to "lib/pythonX.Y/site-packages".
32+
#
33+
# One place this shows up is on MacOS when using the Python interpreter
34+
# installed with the Xcode Command Line Tools. Note that this Python
35+
# isn't intended for general development.
36+
major, minor = sys.version_info.major, sys.version_info.minor
37+
sitePackagePath = pathlib.PurePath(
38+
f"lib/python{major}.{minor}/site-packages")
39+
40+
# Ensure we have a relative path; we should never have an absolute path
41+
# here since by default we want to install the modules somewhere within
42+
# the USD install prefix.
43+
if sitePackagePath.is_absolute():
44+
sys.exit(f"ERROR: Expected relative path, got {sitePackagePath}")
45+
46+
print(sitePackagePath)

‎docs/plugins_renderman.rst‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -68,9 +68,6 @@ variables. We'll refer to the install location of your USD build with
6868
* - Environment Variable
6969
- Meaning
7070
- Value
71-
* - :envvar:`PYTHONPATH`
72-
- This is a path list which Python uses to find modules.
73-
- :filename:`$PYTHONPATH:<inst>/lib/python`
7471
* - :envvar:`RMANTREE`
7572
- Directory where Pixar's RenderMan is installed
7673
- Required

0 commit comments

Comments
 (0)