Cython sanity check uses PKG_CONFIG_PATH while python.find_installation().dependency() does not

[jorisvandenbossche]

Describe the bug

I noticed that building pandas from source in a conda environment with the latest meson 1.11 stopped working (and downgrading just meson to 1.10 made it work), with a ERROR: Compiler cython cannot compile programs. error.

Looking at the logs, it seems that cython is now failing in the sanity check step, because it finds system Python instead of the Python inside the conda environment.

Relevant part of meson-log.txt

Detecting compiler via: `cython -V` -> 0
stdout:
Cython version 3.2.4
-----------
stderr:
Cython version 3.2.4
-----------
Sanity check transpiler command line: cython --fast-fail -o sanity_check_for_cython.c sanity_check_for_cython.pyx -3
Sanity check transpiler stdout:

-----
Sanity check transpiler stderr:

-----
Pkg-config binary missing from cross or native file, or env var undefined.
Trying a default Pkg-config fallback at pkg-config
Found pkg-config: YES (/usr/bin/pkg-config) 0.29.2
Determining dependency 'python3' with pkg-config executable '/usr/bin/pkg-config'
env[PKG_CONFIG_PATH]: 
env[PKG_CONFIG]: /usr/bin/pkg-config
-----------
Called: `/usr/bin/pkg-config --modversion python3` -> 0
stdout:
3.10
-----------
env[PKG_CONFIG_PATH]: 
env[PKG_CONFIG]: /usr/bin/pkg-config
-----------
Called: `/usr/bin/pkg-config --cflags python3` -> 0
stdout:
-I/usr/include/python3.10 -I/usr/include/x86_64-linux-gnu/python3.10
-----------
env[PKG_CONFIG_ALLOW_SYSTEM_LIBS]: 1
env[PKG_CONFIG_PATH]: 
env[PKG_CONFIG]: /usr/bin/pkg-config
-----------
Called: `/usr/bin/pkg-config --libs python3` -> 0
env[PKG_CONFIG_PATH]: 
env[PKG_CONFIG]: /usr/bin/pkg-config
-----------
Called: `/usr/bin/pkg-config --libs python3` -> 0
Run-time dependency python3 found: YES 3.10
checking compilation of transpiled source:
Sanity check compiler command line: /home/joris/scipy/repos/test-meson-cython/.pixi/envs/default/bin/x86_64-conda-linux-gnu-cc -D_FILE_OFFSET_BITS=64 -o sanity_check_for_cython.so sanity_check_for_cython.c -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/joris/scipy/repos/test-meson-cython/.pixi/envs/default/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/joris/scipy/repos/test-meson-cython/.pixi/envs/default/include -D_FILE_OFFSET_BITS=64 -fPIC -I/usr/include/python3.10 -I/usr/include/x86_64-linux-gnu/python3.10 -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/home/joris/scipy/repos/test-meson-cython/.pixi/envs/default/lib -Wl,-rpath-link,/home/joris/scipy/repos/test-meson-cython/.pixi/envs/default/lib -L/home/joris/scipy/repos/test-meson-cython/.pixi/envs/default/lib -shared -Wl,--allow-shlib-undefined
Sanity check compile stdout:

-----
Sanity check compile stderr:
In file included from /usr/include/python3.10/Python.h:8,
                 from sanity_check_for_cython.c:19:
/usr/include/python3.10/pyconfig.h:3:12: fatal error: x86_64-linux-gnu/python3.10/pyconfig.h: No such file or directory
    3 | #  include <x86_64-linux-gnu/python3.10/pyconfig.h>
      |            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

-----

../meson.build:1:0: ERROR: Compiler cython cannot compile programs.

Things that seem to fix it (but so were not needed for older versions of meson):

• Explicitly setting PKG_CONFIG_PATH=$CONDA_PREFIX/lib/pkgconfig
• Installing pkg-config in the local conda environment

To Reproduce

Minimal meson.build file:

project(
  'test-cython',
  ['c', 'cython'],
  version: '0.1.0',
)

py = import('python').find_installation(pure: false)

and then running meson setup . build/ fails with meson 1.11 but works with 1.10, if running this inside a conda env (with the c compiler also getting detected from the conda env)

Expected behavior
That this keeps working as with meson 1.10 (i.e. works out of the box in a conda env without having to set env variables manually)

system parameters

• Native build
• Ubuntu 22.04
• Python 3.13 (but not limited to this version)
• what meson --version: 1.11.1

[jorisvandenbossche]

This might be related to #15092 (cc @dcbaker) which replaced the previous light-weight cython check with a more elaborate sanity check.

Also, with the above meson.build file, it is quickly reproducible using pixi, which can quickly set up conda envs on the fly:

$ pixi exec --with meson=1.11.1 --with meson-python --with python=3.13 --with cython --with c-compiler meson setup . build111
The Meson build system
Version: 1.11.1
Source dir: /home/joris/scipy/repos/test-meson-cython
Build dir: /home/joris/scipy/repos/test-meson-cython/build111
Build type: native build
Project name: test-cython
Project version: 0.1.0
C compiler for the host machine: /home/joris/.cache/rattler/cache/cached-envs-v0/meson-f46c55a9fd5ef7c8/bin/x86_64-conda-linux-gnu-cc (gcc 14.3.0 "x86_64-conda-linux-gnu-cc (conda-forge gcc 14.3.0-18) 14.3.0")
C linker for the host machine: /home/joris/.cache/rattler/cache/cached-envs-v0/meson-f46c55a9fd5ef7c8/bin/x86_64-conda-linux-gnu-cc ld.bfd 2.45.1

meson.build:1:0: ERROR: Compiler cython cannot compile programs.

vs

$ pixi exec --with meson=1.10.0 --with meson-python --with python=3.13 --with cython --with c-compiler meson setup . build110
The Meson build system
Version: 1.10.0
Source dir: /home/joris/scipy/repos/test-meson-cython
Build dir: /home/joris/scipy/repos/test-meson-cython/build110
Build type: native build
Project name: test-cython
Project version: 0.1.0
C compiler for the host machine: /home/joris/.cache/rattler/cache/cached-envs-v0/meson-4e5056ada20d421e/bin/x86_64-conda-linux-gnu-cc (gcc 14.3.0 "x86_64-conda-linux-gnu-cc (conda-forge gcc 14.3.0-18) 14.3.0")
C linker for the host machine: /home/joris/.cache/rattler/cache/cached-envs-v0/meson-4e5056ada20d421e/bin/x86_64-conda-linux-gnu-cc ld.bfd 2.45.1
Cython compiler for the host machine: cython (cython 3.2.4)
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python3 found: YES (/home/joris/.cache/rattler/cache/cached-envs-v0/meson-4e5056ada20d421e/bin/python3.13)
Build targets in project: 0

Found ninja-1.13.2 at /home/joris/.cache/rattler/cache/cached-envs-v0/meson-4e5056ada20d421e/bin/ninja

[dcbaker]

Yeah, the previous implementation of the "Cython" sanity check was actually the C sanity check (so not actually testing the output of Cython at all), so I'm not surprised that some cases that previously succeeded are now failing. I will look more and see if this is something that can be resolved.

[dcbaker]

I'm going to admit I'm confused about how this is working at all, if pkg-config is returning the system headers in the check, isn't it also returning the system headers when you compile the transpiled cython sources in your build. so even if we stubbed out the sanity check it should still fail to build?

[eli-schwartz]

import('python').find_installation().dependency()

is not guaranteed to be the default python3 installed to /usr.

Myself, I'm surprised that the python3 installed to /usr had Python.h but didn't have x86_64-linux-gnu/python3.10/pyconfig.h. Although that's not really the issue here.

[bonzini]

Can this be closed? Based on pandas-dev/pandas#65228, it seems like the problem was in the image.

[dcbaker]

I'm not sure. I guess the question is whether this constitutes a regression that previous our sanity check didn't do anything and the project configured correctly, or whether that is a regression. I tend to think It's an annoying regression, but maybe as a project we think it's fine?

[bonzini]

The question is why it failed. The linked pandas bug has this in root cause:

The CI conda env uses Python 3.11, but PKG_CONFIG_PATH was empty, so pkg-config fell back to system Python 3.12. The latest runner image has /usr/include/python3.12/pyconfig.h referencing a multiarch header (x86_64-linux-gnu/python3.12/pyconfig.h) that doesn't exist.

There are two layers of bugs here, each of which is worth catching:

• The ubuntu-24.04 runner image updated on 2026-04-14 (ubuntu24/20260413.86) has broken system Python 3.12 headers — the multiarch pyconfig.h is missing, causing the meson Cython sanity check to fail with Compiler cython cannot compile programs
• meson wasn't even finding the desired Python 3.11 headers, and having the broken system ones was actually a stroke of luck, because the system headers were newer and not necessarily 3.11-compatible?

Perhaps Meson could improve how Python headers are found, but the new Cython sanity check seems fine? If so, this would be a new feature and not something for 1.11.2.

[dcbaker]

I guess the question is: if those bugs are worth fixing for 1.12, should we add a warning for 1.11 that things may be broken and allow the sanity check to be skipped?