Skip to content

Better virtual envrionment isolation for reward service #495

Description

@Dinghow

The current reward service uses Python venv for different models; there are several issues:

  1. Unconfigurable Python version. Python venv does not support different Python versions. Some model like OCR require old python and Torch, a version-configurable Python is needed for our further support of other models.
  2. Improper Python import path. The RewardProcessHandler has a fallback logic for venv path, which breaks the independence of the model's environment. Some libraries, like flash-attn or op with cuda will raise errors.

We need a better virtual environment management for the reward service, uv is a good choice.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions