How to selectively install packages?

[g-kartik]

I need to install av package. It needs to be installed in an intermediate docker container having a specific version of ffmpeg without fetching binaries. How to tell sync command to install just this package and exclude others and vice-versa? I think it does not support as of now. As a workaround, I added the av pacakge under optional-dependencies group pyav and tried installation using command pdm sync -G pyav --no-default --without dev, however it also installs packages fromdev group. Can't we just install packages from a single group exclusively?

[frostming]

This is a bug, the command should work

You can run pdm sync -G pyav --no-default --prod to achieve this now.

[g-kartik]

Yes it works. However I couldn't understand the command. --no-default should nullify --prod, isn't it? What does it do exactly?

[frostming]

--prod is a negative flag, rather than positive, it means to unselect the whole dev-dependencies groups, adding --no-default means no groups are selected.

[g-kartik]

Got it. Thanks.
However what is your take on this topic? Should users be allowed to selectively install packages from any group and also exclude them?

[g-kartik]

Hey,
The installation of av fails when passing --no-isolation flag. Is this error related to pdm?

ERROR [build-image-av 6/6] RUN --mount=type=cache,target=/home/root/.cache/pdm     pdm sync -G pyav --no-isolation --prod --no-default --no-editable --verbose                     20.9s
------                                                                                                                                                                                       
 > [build-image-av 6/6] RUN --mount=type=cache,target=/home/root/.cache/pdm     pdm sync -G pyav --no-isolation --prod --no-default --no-editable --verbose:                                 
2.599 WARNING: Project requires a python version of >=3.8.10, The virtualenv is being created for you as it cannot be matched to the right version.                                          
2.599 INFO: python.use_venv is on, creating a virtualenv for this project...                                                                                                                 
2.639 Run command: ['/usr/bin/python3', '-m', 'virtualenv', '/opt/.venv', '-p', '/usr/bin/python3', '--prompt=opt-3.8', '--no-pip', '--no-setuptools', '--no-wheel']                         
2.806 created virtual environment CPython3.8.10.final.0-64 in 47ms                                                                                                                           
2.806   creator CPython3Posix(dest=/opt/.venv, clear=False, no_vcs_ignore=False, global=False)                                                                                               
2.806   activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator                                                                          
2.830 Virtualenv is created successfully at /opt/.venv                                                                                                                                       
2.912 STATUS: Resolving packages from lockfile...                                                                                                                                            
3.104 Synchronizing working set with resolved packages: 1 to add, 0 to update, 0 to remove                                                                                                   
3.104                                                                                                                                                                                        
20.19 unearth.preparer: Downloading <Link https://files.pythonhosted.org/packages/08/bf/83b16b2ef4f414e1a127d484e1617d14c2e09c2b258dd6df4780a9dbb4d7/av-8.0.2.tar.gz (from https://pypi.org/simple/av/)> (2.3 MB)
20.41 pdm.termui: Preparing environment(Non-isolated mode) for PEP 517 build...
20.41 pdm.termui: Saving wheel to cache: <Link https://files.pythonhosted.org/packages/08/bf/83b16b2ef4f414e1a127d484e1617d14c2e09c2b258dd6df4780a9dbb4d7/av-8.0.2.tar.gz (from https://pypi.org/simple/av/)>
20.41 pdm.termui: Running PEP 517 backend to build a wheel for <Link https://files.pythonhosted.org/packages/08/bf/83b16b2ef4f414e1a127d484e1617d14c2e09c2b258dd6df4780a9dbb4d7/av-8.0.2.tar.gz (from https://pypi.org/simple/av/)>
20.48   ✖ Install av 8.0.2 failed
20.48 pdm.termui: Error occurs: 
20.48 Traceback (most recent call last):
20.48   File "/usr/local/lib/python3.8/dist-packages/pdm/builders/base.py", line 84, in wrapper
20.48     return func(*args, **kwargs)
20.48   File "/usr/local/lib/python3.8/dist-packages/pdm/builders/wheel.py", line 26, in build
20.48     filename = self._hook.build_wheel(out_dir, self.config_settings, metadata_directory)
20.48   File "/usr/local/lib/python3.8/dist-packages/pyproject_hooks/_impl.py", line 256, in build_wheel
20.48     return self._call_hook(
20.48   File "/usr/local/lib/python3.8/dist-packages/pyproject_hooks/_impl.py", line 402, in _call_hook
20.48     raise BackendUnavailable(
20.48 pyproject_hooks._impl.BackendUnavailable: Cannot import 'setuptools.build_meta'
20.48 
20.48 The above exception was the direct cause of the following exception:
20.48 
20.48 Traceback (most recent call last):
20.48   File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
20.48     result = self.fn(*self.args, **self.kwargs)
20.48   File "/usr/local/lib/python3.8/dist-packages/pdm/installers/synchronizers.py", line 230, in install_candidate
20.48     self.manager.install(can)
20.48   File "/usr/local/lib/python3.8/dist-packages/pdm/installers/manager.py", line 33, in install
20.48     prepared.build(),
20.48   File "/usr/local/lib/python3.8/dist-packages/pdm/models/candidates.py", line 402, in build
20.48     self._cached = Path(builder.build(build_dir, metadata_directory=self._metadata_dir))
20.48   File "/usr/local/lib/python3.8/dist-packages/pdm/builders/base.py", line 86, in wrapper
20.48     raise BuildError(str(e)) from e
20.48 pdm.exceptions.BuildError: Cannot import 'setuptools.build_meta'
20.48 Retry failed jobs
20.48 pdm.termui: Preparing environment(Non-isolated mode) for PEP 517 build...
20.48 pdm.termui: Saving wheel to cache: <Link https://files.pythonhosted.org/packages/08/bf/83b16b2ef4f414e1a127d484e1617d14c2e09c2b258dd6df4780a9dbb4d7/av-8.0.2.tar.gz (from https://pypi.org/simple/av/)>
20.48 pdm.termui: Running PEP 517 backend to build a wheel for <Link https://files.pythonhosted.org/packages/08/bf/83b16b2ef4f414e1a127d484e1617d14c2e09c2b258dd6df4780a9dbb4d7/av-8.0.2.tar.gz (from https://pypi.org/simple/av/)>
20.57   ✖ Install av 8.0.2 failed
20.57 pdm.termui: Error occurs: 
20.57 Traceback (most recent call last):
20.57   File "/usr/local/lib/python3.8/dist-packages/pdm/builders/base.py", line 84, in wrapper
20.57     return func(*args, **kwargs)
20.57   File "/usr/local/lib/python3.8/dist-packages/pdm/builders/wheel.py", line 26, in build
20.57     filename = self._hook.build_wheel(out_dir, self.config_settings, metadata_directory)
20.57   File "/usr/local/lib/python3.8/dist-packages/pyproject_hooks/_impl.py", line 256, in build_wheel
20.57     return self._call_hook(
20.57   File "/usr/local/lib/python3.8/dist-packages/pyproject_hooks/_impl.py", line 402, in _call_hook
20.57     raise BackendUnavailable(
20.57 pyproject_hooks._impl.BackendUnavailable: Cannot import 'setuptools.build_meta'
20.57 
20.57 The above exception was the direct cause of the following exception:
20.57 
20.57 Traceback (most recent call last):
20.57   File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
20.57     result = self.fn(*self.args, **self.kwargs)
20.57   File "/usr/local/lib/python3.8/dist-packages/pdm/installers/synchronizers.py", line 230, in install_candidate
20.57     self.manager.install(can)
20.57   File "/usr/local/lib/python3.8/dist-packages/pdm/installers/manager.py", line 33, in install
20.57     prepared.build(),
20.57   File "/usr/local/lib/python3.8/dist-packages/pdm/models/candidates.py", line 402, in build
20.57     self._cached = Path(builder.build(build_dir, metadata_directory=self._metadata_dir))
20.57   File "/usr/local/lib/python3.8/dist-packages/pdm/builders/base.py", line 86, in wrapper
20.57     raise BuildError(str(e)) from e
20.57 pdm.exceptions.BuildError: Cannot import 'setuptools.build_meta'
20.60 
20.61 Traceback (most recent call last):
20.61   File "/usr/local/bin/pdm", line 8, in <module>
20.61     sys.exit(main())
20.61   File "/usr/local/lib/python3.8/dist-packages/pdm/core.py", line 346, in main
20.61     return core.main(args or sys.argv[1:])
20.61   File "/usr/local/lib/python3.8/dist-packages/pdm/core.py", line 264, in main
20.61     raise cast(Exception, err).with_traceback(traceback) from None
20.61   File "/usr/local/lib/python3.8/dist-packages/pdm/core.py", line 259, in main
20.61     self.handle(project, options)
20.61   File "/usr/local/lib/python3.8/dist-packages/pdm/core.py", line 195, in handle
20.61     command.handle(project, options)
20.61   File "/usr/local/lib/python3.8/dist-packages/pdm/cli/commands/sync.py", line 44, in handle
20.61     actions.do_sync(
20.61   File "/usr/local/lib/python3.8/dist-packages/pdm/cli/actions.py", line 330, in do_sync
20.61     synchronizer.synchronize()
20.61   File "/usr/local/lib/python3.8/dist-packages/pdm/installers/synchronizers.py", line 429, in synchronize
20.61     raise InstallationError("Some package operations are not complete yet")
20.61 pdm.exceptions.InstallationError: Some package operations are not complete yet
------

 1 warning found (use --debug to expand):
 - UndefinedVar: Usage of undefined variable '$no_proxy' (line 74)
Dockerfile:59
--------------------
  58 |     WORKDIR /opt
  59 | >>> RUN --mount=type=cache,target=/home/root/.cache/pdm \
  60 | >>>     pdm sync -G pyav --no-isolation --prod --no-default --no-editable --verbose
  61 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c pdm sync -G pyav --no-isolation --prod --no-default --no-editable --verbose" did not complete successfully: exit code: 1

[frostming]

Is this error related to pdm?

No, learn about the effect of --no-isolation please.