Put a safeguard on `spk convert pip` to not install python-X if X already exists

[dcookspi]

This is a request for feature to block packages from being converted with the convert tool. Specifically for a safeguard on spk convert pip to not install python-X if X already exists in SPK.

This has nearly bitten us (SPI) a couple of times. The example spk package is opencolorio, which exists in our repo and provides valid python bindings. The problem package is the pypi opencolorio that is converted to python-opencolorio by spk convert pip ... typically as part of a dependency of another top-level python packages that's being converted. python-opencolorio provides conflicting opencolorio bindings and this may lead you subtle issues that we want to avoid (based on assumptions of names and capabilities that are not correct). Another example of this is openimageio.

So we'd like a way to prevent this from occuring throughspk convert pip ... commands. At the moment, someone converting a top-level package might not be aware of all the dependencies it is also converting and whether conflicting ones like this should be published into spk.

This could be by checking for spk package names before converting packages or dependencies, or having config for blocklist of know culprits for a site (in case they don't want a blanket ban on this kind of shard name suffixes).

[lgritz]

Yeah, my concern is mainly that because no connection between X and python-X is understood by the system, you could very easily get environments instantiated that include both and they are incompatible.

I'm not especially opinionated about how we fix this, but the simple approach would be if spk convert pip would notice if X already exists, print a scary warning, and request explicit confirmation before creating a python-X (and the other way around if python-X exists and then somebody tries to create X). That still places some burden on people's judgment when they see the warning.

Maybe a slightly more sophisticated approach would be some way for X to explicitly declare that it provides a python module, and for the solver to just know that X and python-X provide overlapping components and are therefore incompatible (while not necessarily disallowing the existence of both packages, only constraining that they can't both be instantiated in the same environment).

[rydrman]

From the meeting today:

• the most appropriate solution here seems to introduce a blocklist for spk-convert-pip with an explicit error message in addition to a matching embedded package so that spk-convert-pip will use the internal version to satisfy the python dependency where possible