Skip to content

Commit 3e6c909

Browse files
authored
feat(experiments): Add new flags to Experiment entity (#1371)
* Add new flags to Experiment entity Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com> * use show_evaluations_over_time Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com> --------- Signed-off-by: shanaiabuggy <59746633+shanaiabuggy@users.noreply.github.com>
1 parent e1093e9 commit 3e6c909

32 files changed

Lines changed: 1459 additions & 134 deletions

File tree

docs/cli/reference.mdx

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6046,8 +6046,10 @@ nemo experiments create [OPTIONS] [NAME]
60466046
* `--default-sort`: Default sort for this experiment's evaluations list, as a `sort`-param string: a comma-separated, ordered list of fields where the first is the primary sort and the rest break ties (leading '-' on a field = descending), e.g. '-evaluators.reward.mean,cost_usd.mean'. Defaults to '-created_at'. Accepts any field the evaluations list `sort` param does; clients apply it as the list `sort` param.
60476047
* `--description`: Human-readable purpose of the experiment.
60486048
* `--insight-id`: Reference to an external insight that seeded this experiment, if any.
6049+
* `--is-favorite`: Whether this Experiment is marked as a favorite. Defaults to false on create; omit on update to preserve the existing value.
60496050
* `--metadata`: Free-form producer metadata for the experiment. (JSON string)
60506051
* `--pareto`: Default X/Y metrics for a group's cost-vs-accuracy Pareto view. Metric ids use the same vocabulary as the evaluations list sort/filter fields — `cost_usd`, `latency_ms`, or `evaluators.<name>`. Defaults to cost (x) vs latency (y): both exist for every group, so the chart always has something to render before anyone customizes it. (JSON string)
6052+
* `--show-evaluations-over-time`: Whether Studio should display this Experiment's Evaluation results over time. Defaults to false on create; omit on update to preserve the existing value.
60516053
* `--summary`: Human- or agent-authored summary of the experiment's findings.
60526054
* `--exist-ok`: Do not raise an error if the resource already exists. Returns the existing resource.
60536055

@@ -6110,10 +6112,13 @@ nemo experiments list [OPTIONS]
61106112
JSON-only fields:
61116113
metadata: dict[str, str]
61126114

6113-
Filter experiments by name, insight_id, is_deleted, or a metadata key/value (filter[metadata.\<key>]=\<value>). Pass is_deleted=true to return only soft-deleted experiments; omit to see only live ones.
6115+
Filter experiments by name, insight_id, is_favorite, show_evaluations_over_time, baseline_evaluation_name, is_deleted, or a metadata key/value (filter[metadata.\<key>]=\<value>). Pass is_deleted=true to return only soft-deleted experiments; omit to see only live ones.
6116+
* `--filter.baseline-evaluation-name`
61146117
* `--filter.insight-id`
61156118
* `--filter.is-deleted`
6119+
* `--filter.is-favorite`
61166120
* `--filter.name`
6121+
* `--filter.show-evaluations-over-time`
61176122

61186123
**Help:**
61196124

@@ -6181,11 +6186,14 @@ nemo experiments update [OPTIONS] PATH_NAME
61816186

61826187
* `--workspace`
61836188
* `--body-name`: Workspace-unique experiment name.
6189+
* `--baseline-evaluation-name`: Name of this Experiment's baseline Evaluation. The Evaluation must already be a live member of the Experiment. Set null to clear the selected baseline.
61846190
* `--default-sort`: Default sort for this experiment's evaluations list, as a `sort`-param string: a comma-separated, ordered list of fields where the first is the primary sort and the rest break ties (leading '-' on a field = descending), e.g. '-evaluators.reward.mean,cost_usd.mean'. Defaults to '-created_at'. Accepts any field the evaluations list `sort` param does; clients apply it as the list `sort` param.
61856191
* `--description`: Human-readable purpose of the experiment.
61866192
* `--insight-id`: Reference to an external insight that seeded this experiment, if any.
6193+
* `--is-favorite`: Whether this Experiment is marked as a favorite. Defaults to false on create; omit on update to preserve the existing value.
61876194
* `--metadata`: Free-form producer metadata for the experiment. (JSON string)
61886195
* `--pareto`: Default X/Y metrics for a group's cost-vs-accuracy Pareto view. Metric ids use the same vocabulary as the evaluations list sort/filter fields — `cost_usd`, `latency_ms`, or `evaluators.<name>`. Defaults to cost (x) vs latency (y): both exist for every group, so the chart always has something to render before anyone customizes it. (JSON string)
6196+
* `--show-evaluations-over-time`: Whether Studio should display this Experiment's Evaluation results over time. Defaults to false on create; omit on update to preserve the existing value.
61896197
* `--summary`: Human- or agent-authored summary of the experiment's findings.
61906198

61916199
**Help:**
@@ -6811,7 +6819,7 @@ nemo intake ingest spans create [OPTIONS]
68116819
**Options:**
68126820

68136821
* `--workspace`
6814-
* `--source`: (required)
6822+
* `--source`: Stable name for the source trace store, such as `langsmith` or `mlflow`.
68156823
* `--spans`: JSON string
68166824

68176825
**Help:**

openapi/ga/individual/platform.openapi.yaml

Lines changed: 122 additions & 5 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)