The current reward service uses Python venv for different models; there are several issues:
- Unconfigurable Python version. Python venv does not support different Python versions. Some model like OCR require old python and Torch, a version-configurable Python is needed for our further support of other models.
- Improper Python import path. The RewardProcessHandler has a fallback logic for venv path, which breaks the independence of the model's environment. Some libraries, like flash-attn or op with cuda will raise errors.
We need a better virtual environment management for the reward service, uv is a good choice.
The current reward service uses Python venv for different models; there are several issues:
We need a better virtual environment management for the reward service,
uvis a good choice.