ResolvePackageNotFound error in Ubuntu/Linux

[urlicht]

Not sure if it's a PyPlot-specific problem (could be caused by PyCall.jl or matplotlib itself I suppose)

Upon a fresh install of the latest release of PyCall (setting ENV["PYTHON"]="" to use a separate Julia miniconda environment), it fails to install certain packages, including matplotlib. Trying to install PyPlot.jl (or even manually activating the environment and performing conda install matplotlib) returns the following error:

Solving environment: unsuccessful initial attempt using frozen solve. Retrying with flexible solve.
Solving environment: unsuccessful attempt using repodata from current_repodata.json, retrying with next repodata source.

ResolvePackageNotFound: 
  - conda==23.1.0

Reproducible across multiple Ubuntu machines running 18.04, 20.04, or 22.04

[a2ray]

Yes, I was absolutely unable to register a package update on GitHub/Registrator because of this. My solution was install matplolib through Conda.pip and then Conda.add, before pointing PyCall to Conda's python. It works now, see here for my solution within a build script. This needs to remain an open issue IMHO.

[Alexander-Barth]

Here is a reproducer using docker:

docker run -it   julia:1.9 /bin/bash
julia --eval 'using Pkg; Pkg.add("PyPlot");'
julia --eval 'using PyPlot'

I am wondering if it also affects non-Linux systems.

Full session

$ docker run -it   julia:1.9 /bin/bash
Unable to find image 'julia:1.9' locally
1.9: Pulling from library/julia
648e0aadf75a: Pull complete 
f396ccff5916: Pull complete 
5e8906fb6194: Pull complete 
f4f5ee390bc1: Pull complete 
Digest: sha256:fcb2a336582c4da45bd8a3ba97cc84f0556b89fbd103cda9ab305a591785c4ae
Status: Downloaded newer image for julia:1.9
root@30d69add978b:/# julia --eval 'using Pkg; Pkg.add("PyPlot");'
  Installing known registries into `~/.julia`
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
   Installed FixedPointNumbers ─ v0.8.4
   Installed JSON ────────────── v0.21.4
   Installed Preferences ─────── v1.4.0
   Installed PyCall ──────────── v1.96.1
   Installed ColorTypes ──────── v0.11.4
   Installed Parsers ─────────── v2.7.2
   Installed Conda ───────────── v1.9.1
   Installed PrecompileTools ─── v1.1.2
   Installed VersionParsing ──── v1.3.0
   Installed PyPlot ──────────── v2.11.1
   Installed Reexport ────────── v1.2.2
   Installed LaTeXStrings ────── v1.3.0
   Installed MacroTools ──────── v0.5.10
   Installed Colors ──────────── v0.12.10
    Updating `~/.julia/environments/v1.9/Project.toml`
  [d330b81b] + PyPlot v2.11.1
    Updating `~/.julia/environments/v1.9/Manifest.toml`
  [3da002f7] + ColorTypes v0.11.4
  [5ae59095] + Colors v0.12.10
  [682c06a0] + JSON v0.21.4
  [b964fa9f] + LaTeXStrings v1.3.0
  [1914dd2f] + MacroTools v0.5.10
  [69de0a69] + Parsers v2.7.2
  [aea7be01] + PrecompileTools v1.1.2
  [21216c6a] + Preferences v1.4.0
  [438e738f] + PyCall v1.96.1
  [d330b81b] + PyPlot v2.11.1
  [189a3867] + Reexport v1.2.2
  [81def892] + VersionParsing v1.3.0
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [b77e0a4c] + InteractiveUtils
  [b27032c2] + LibCURL v0.6.3
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.9.2
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics v1.9.0
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v1.0.5+0
  [deac9b47] + LibCURL_jll v7.84.0+0
  [29816b5a] + LibSSH2_jll v1.10.2+0
  [c8ffd9c3] + MbedTLS_jll v2.28.2+0
  [14a3606d] + MozillaCACerts_jll v2022.10.11
  [4536629a] + OpenBLAS_jll v0.3.21+4
  [bea87d4a] + SuiteSparse_jll v5.10.1+6
  [83775a58] + Zlib_jll v1.2.13+0
  [8e850b90] + libblastrampoline_jll v5.8.0+0
  [8e850ede] + nghttp2_jll v1.48.0+0
  [3f19e933] + p7zip_jll v17.4.0+0
    Building Conda ─→ `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/8c86e48c0db1564a1d49548d3515ced5d604c408/build.log`
    Building PyCall → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/43d304ac6f0354755f1d60730ece8c499980f7ba/build.log`
Precompiling project...
  15 dependencies successfully precompiled in 42 seconds. 5 already precompiled.
root@30d69add978b:/# julia --eval 'using PyPlot'
[ Info: Installing matplotlib via the Conda matplotlib package...
[ Info: Running `conda install -y matplotlib` in root environment
Collecting package metadata (current_repodata.json): done
Solving environment: unsuccessful initial attempt using frozen solve. Retrying with flexible solve.
Solving environment: unsuccessful attempt using repodata from current_repodata.json, retrying with next repodata source.

ResolvePackageNotFound: 
  - conda==23.1.0

ERROR: InitError: failed process: Process(setenv(`/root/.julia/conda/3/x86_64/bin/conda install -y matplotlib`,["SHLVL=1", "PATH=/usr/local/julia/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "PWD=/", "CONDA_PREFIX=/root/.julia/conda/3/x86_64", "JULIA_PATH=/usr/local/julia", "JULIA_GPG=3673DF529D9049477F76B37566E3C7DC03D6E495", "_=/usr/local/julia/bin/julia", "OPENBLAS_DEFAULT_NUM_THREADS=1", "JULIA_VERSION=1.9.2", "HOME=/root", "TERM=xterm", "CONDARC=/root/.julia/conda/3/x86_64/condarc-julia.yml", "OPENBLAS_MAIN_FREE=1", "HOSTNAME=30d69add978b", "PYTHONIOENCODING=UTF-8"]), ProcessExited(1)) [1]

Stacktrace:
  [1] pipeline_error
    @ ./process.jl:565 [inlined]
  [2] run(::Cmd; wait::Bool)
    @ Base ./process.jl:480
  [3] run
    @ ./process.jl:477 [inlined]
  [4] runconda(args::Cmd, env::String)
    @ Conda ~/.julia/packages/Conda/2lg2O/src/Conda.jl:128
  [5] add(pkg::String, env::String; channel::String, satisfied_skip_solve::Bool, args::Cmd)
    @ Conda ~/.julia/packages/Conda/2lg2O/src/Conda.jl:256
  [6] add (repeats 2 times)
    @ ~/.julia/packages/Conda/2lg2O/src/Conda.jl:239 [inlined]
  [7] pyimport_conda
    @ ~/.julia/packages/PyCall/ilqDX/src/PyCall.jl:721
  [8] pyimport_conda
    @ ~/.julia/packages/PyCall/ilqDX/src/PyCall.jl:715 [inlined]
  [9] __init__()
    @ PyPlot ~/.julia/packages/PyPlot/H01LC/src/init.jl:173
 [10] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1115
 [11] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
    @ Base ./loading.jl:1061
 [12] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
    @ Base ./loading.jl:1506
 [13] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1783
 [14] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1660
 [15] macro expansion
    @ ./loading.jl:1648 [inlined]
 [16] macro expansion
    @ ./lock.jl:267 [inlined]
 [17] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1611
during initialization of module PyPlot

caused by: PyError (PyImport_ImportModule

The Python package matplotlib could not be imported by pyimport. Usually this means
that you did not install matplotlib in the Python version being used by PyCall.

PyCall is currently configured to use the Julia-specific Python distribution
installed by the Conda.jl package.  To install the matplotlib module, you can
use `pyimport_conda("matplotlib", PKG)`, where PKG is the Anaconda
package that contains the module matplotlib, or alternatively you can use the
Conda package directly (via `using Conda` followed by `Conda.add` etcetera).

Alternatively, if you want to use a different Python distribution on your
system, such as a system-wide Python (as opposed to the Julia-specific Python),
you can re-configure PyCall with that Python.   As explained in the PyCall
documentation, set ENV["PYTHON"] to the path/name of the python executable
you want to use, run Pkg.build("PyCall"), and re-launch Julia.

) <class 'ModuleNotFoundError'>
ModuleNotFoundError("No module named 'matplotlib'")

Stacktrace:
  [1] pyimport(name::String)
    @ PyCall ~/.julia/packages/PyCall/ilqDX/src/PyCall.jl:558
  [2] pyimport_conda
    @ ~/.julia/packages/PyCall/ilqDX/src/PyCall.jl:716
  [3] pyimport_conda
    @ ~/.julia/packages/PyCall/ilqDX/src/PyCall.jl:715 [inlined]
  [4] __init__()
    @ PyPlot ~/.julia/packages/PyPlot/H01LC/src/init.jl:173
  [5] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1115
  [6] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
    @ Base ./loading.jl:1061
  [7] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
    @ Base ./loading.jl:1506
  [8] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1783
  [9] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1660
 [10] macro expansion
    @ ./loading.jl:1648 [inlined]
 [11] macro expansion
    @ ./lock.jl:267 [inlined]
 [12] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1611

Conda info

root@30d69add978b:/# /root/.julia/conda/3/x86_64/bin/conda info

     active environment : None
       user config file : /root/.condarc
 populated config files : /root/.julia/conda/3/x86_64/.condarc
          conda version : 23.7.2
    conda-build version : not installed
         python version : 3.10.12.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.36=0
                          __linux=5.15.0=0
                          __unix=0=0
       base environment : /root/.julia/conda/3/x86_64  (writable)
      conda av data dir : /root/.julia/conda/3/x86_64/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /root/.julia/conda/3/x86_64/pkgs
                          /root/.conda/pkgs
       envs directories : /root/.julia/conda/3/x86_64/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/23.7.2 requests/2.31.0 CPython/3.10.12 Linux/5.15.0-47-generic debian/12 glibc/2.36
                UID:GID : 0:0
             netrc file : None
           offline mode : False

[Alexander-Barth]

Other workarounds:

• first downgrading to conda==23.1.0, then installing matplotlib (which surprisingly installs conda 23.7.2)
• first install mamba, then install matplotlib using mamba.

As it is likely a conda problem, I posted this question of StackOverflow:
https://stackoverflow.com/questions/76817732/installing-matplotlib-with-conda-fails-with-conda-exceptions-resolvepackagenotfo