Optional EvalPort interop for QualityClassifier scores

[adhabnr-ux]

Hi NeMo Curator team — I maintain EvalPort, an open, framework-agnostic JSON spec for portable LLM eval datasets and results (a TestCase/Suite/ResultSet schema with a validator, so a dataset or a graded run can move between tools without hand-writing a converter each time). Opening this as an issue per CONTRIBUTING.md, since it's a small, concrete proposal rather than a larger design change.

I installed nemo-curator and read the real classifier stage rather than guessing (nemo_curator/stages/text/classifiers/quality.py):

class QualityClassifier(DistributedDataClassifier):
    def __init__(
        self,
        label_field: str = "quality_pred",
        score_field: str | None = None,
        text_field: str = "text",
        filter_by: list[str] | None = None,
        ...
    ): ...

Running this stage over a dataset writes a label_field (default "quality_pred") and, when set, a score_field (the model's probability) onto every row, keyed off text_field. That's a direct match for EvalPort's other half: each row of text is a TestCase, and the quality_pred/probability pair QualityClassifier attaches to it is exactly the shape of an EvalPort Grader result — a per-TestCase label plus a numeric score — so a curated dataset's classifier output could become a ResultSet without inventing field names.

Two ways I could see this landing, and I don't have a strong preference:

1. A standalone nemo-curator-openeval-adapter package in the EvalPort repo, depending on nemo-curator as a normal dependency. Zero footprint on this repo.
2. A small optional module inside this repo if you'd rather it live here.

Either way, real tests would validate against EvalPort's actual JSON Schema, not a mock.

Let me know which direction you'd prefer, or if this isn't a fit for your roadmap right now — no worries either way.