Add `bt datasets snapshots` for dataset version control by max-braintrust · Pull Request #176 · braintrustdata/bt

max-braintrust · 2026-05-06T21:38:29Z

TL;DR

Follow up to: #104

Adds bt datasets snapshots for managing saved dataset snapshots from the CLI: list, create, and restore dataset states from saved snapshots.

What changed?

Added dataset snapshot management functionality:

New bt datasets snapshots command with subcommands: list, create, delete and restore
Uses snapshot/dataset snapshot terminology to match the UI
Snapshot listing displays saved snapshots for a dataset
Snapshot creation can infer the current dataset head xact when --xact-id is omitted
Snapshot restore supports restore by snapshot name or by xact ID with --snapshot
Restore includes preview support and requires confirmation unless --force is passed
JSON output includes snapshot list/create/restore modes and found_existing for create responses
Moved snapshot-specific CLI args into the snapshot module to match existing subcommand patterns
Added shared profile utilities for resolving profile info and generating default snapshot name slugs

Added test coverage:

Unit tests for snapshot arg parsing, read/write auth routing, response deserialization, default snapshot names, and restore target selection
Fixture coverage for snapshot creation, duplicate xact handling via x-bt-found-existing, and snapshot listing
Mock server support for dataset snapshot create and list endpoints

How to test?

# Create a dataset
bt datasets create my-dataset --rows '[{"id":"case-1","input":{"text":"hi"},"expected":"hello"}]'

# List snapshots
bt datasets snapshots list my-dataset

# Create a snapshot
bt datasets snapshots create my-dataset baseline

# Re-run create for the same xact to verify found_existing handling
bt datasets snapshots create my-dataset baseline-again --xact-id 1000192656880881099 --json

# Restore by snapshot name
bt datasets snapshots restore my-dataset --name baseline

# Restore by xact ID
bt datasets snapshots restore my-dataset --snapshot 1000192656880881099 --force

github-actions · 2026-05-06T23:21:05Z

Latest downloadable build artifacts for this PR commit ad9934c772ce:

Workflow run: https://github.com/braintrustdata/bt/actions/runs/27237360530
Download all artifacts (GitHub CLI): gh run download 27237360530 --repo braintrustdata/bt
Installers are published from main automatically. To publish one for a PR branch, run release-canary manually via workflow_dispatch.

Available artifact names

``artifacts-build-global
``artifacts-build-local-aarch64-pc-windows-msvc
``artifacts-build-local-x86_64-pc-windows-msvc
``artifacts-build-local-x86_64-apple-darwin
``artifacts-build-local-aarch64-apple-darwin
``artifacts-build-local-x86_64-unknown-linux-musl
``artifacts-build-local-x86_64-unknown-linux-gnu
``artifacts-build-local-aarch64-unknown-linux-gnu
``artifacts-plan-dist-manifest
``cargo-dist-cache

Cedric / ViaDézo1er (viadezo1er)

cargo clippy tells me to cargo clippy --fix --bin "bt" -p bt --, which gives a small formatting diff, nothing blocking but nice to format.

diff --git a/src/functions/pull.rs b/src/functions/pull.rs
index 8f2e6de..0b51ef2 100644
--- a/src/functions/pull.rs
+++ b/src/functions/pull.rs
@@ -1410,7 +1410,7 @@ fn format_py_value_inner(value: &Value, depth: usize) -> String {
             let closing_indent = "    ".repeat(depth);
             let mut out = String::from("{\n");
             let mut entries = object.iter().collect::<Vec<_>>();
-            entries.sort_by(|(left, _), (right, _)| left.cmp(right));
+            entries.sort_by_key(|(left, _)| *left);
             for (index, (key, val)) in entries.into_iter().enumerate() {
                 out.push_str(&indent);
                 out.push_str(&serde_json::to_string(key).unwrap_or_else(|_| "\"\"".to_string()));
diff --git a/src/sql.rs b/src/sql.rs
index 4e4b4ac..f6619e4 100644
--- a/src/sql.rs
+++ b/src/sql.rs
@@ -258,11 +258,10 @@ fn handle_key_event(
         KeyCode::End => app.move_end(),
         KeyCode::Up => app.history_prev(),
         KeyCode::Down => app.history_next(),
-        KeyCode::Char(ch) if !key.modifiers.contains(KeyModifiers::CONTROL) => {
-            if !key.modifiers.contains(KeyModifiers::ALT) {
+        KeyCode::Char(ch) if !key.modifiers.contains(KeyModifiers::CONTROL)
+            && !key.modifiers.contains(KeyModifiers::ALT) => {
                 app.insert_char(ch);
             }
-        }
         _ => {}
     }
 
diff --git a/src/traces.rs b/src/traces.rs
index 94e0b3d..c136548 100644
--- a/src/traces.rs
+++ b/src/traces.rs
@@ -2084,8 +2084,8 @@ fn handle_span_detail_key(
                 }
             }
         }
-        KeyCode::Char('d') if key.modifiers.contains(KeyModifiers::CONTROL) => {
-            if app.detail_pane_focus == DetailPaneFocus::Detail {
+        KeyCode::Char('d') if key.modifiers.contains(KeyModifiers::CONTROL)
+            && app.detail_pane_focus == DetailPaneFocus::Detail => {
                 if detail_message_list_mode {
                     if app.detail_view == DetailView::Thread {
                         for _ in 0..10 {
@@ -2100,7 +2100,6 @@ fn handle_span_detail_key(
                     scroll_detail_down_bounded(app, 10)
                 }
             }
-        }
         _ => {}
     }
     Ok(())
@@ -5229,11 +5228,10 @@ fn parse_trace_url(input: &str) -> Result<ParsedTraceUrl> {
                     parsed.span_id = Some(value.to_string());
                 }
             }
-            "tvt" => {
-                if !value.is_empty() {
+            "tvt"
+                if !value.is_empty() => {
                     parsed.trace_view_type = Some(value.to_string());
                 }
-            }
             _ => {}
         }
     }

Cedric / ViaDézo1er (viadezo1er) · 2026-06-05T22:45:37Z

I have never used bt datasets before so I don't really know if it's good or not.

Cedric / ViaDézo1er (viadezo1er) · 2026-06-05T22:47:23Z

bt datasets snapshots list
error: no datasets found
If this seems like a bug, file an issue at https://github.com/braintrustdata/bt/issues/new and include `bt --version`, `bt status --json`, and the command you ran.

Maybe make an empty list instead?

Cedric / ViaDézo1er (viadezo1er) · 2026-06-05T22:49:55Z

Why is there a -d/-dataset flag when datasets can also be passed as arguments?

Cedric / ViaDézo1er (viadezo1er) · 2026-06-05T23:04:54Z

Maybe a bt datasets snapshots rm/delete?

Cedric / ViaDézo1er (viadezo1er) · 2026-06-09T00:16:03Z

❯ bt -V
bt 0.10.1-canary.aa9ca1c4cfb3
❯ bt status
org: ced-test-1
project: input
profile: profile
user: Cédric Halber (cedric@braintrustdata.com)
source: /Users/cedric@braintrustdata.com/.config/bt/config.json
❯ bt auth profiles
✓ profile — oauth — Cédric Halber (cedric@braintrustdata.com)

Credentials: /Users/cedric@braintrustdata.com/.config/bt/auth.json
❯ # Create a dataset
bt datasets create my-dataset --rows '[{"id":"case-1","input":{"text":"hi"},"expected":"hello"}]'

# List snapshots
bt datasets snapshots list my-dataset

# Create a snapshot
bt datasets snapshots create my-dataset baseline

# Re-run create for the same xact to verify found_existing handling
bt datasets snapshots create my-dataset baseline-again --xact-id 1000192656880881099 --json

# Restore by snapshot name
bt datasets snapshots restore my-dataset --name baseline

# Restore by xact ID
bt datasets snapshots restore my-dataset --snapshot 1000192656880881099 --force
✓ Created 'my-dataset' with 1 records
0 snapshots found for ced-test-1 / input / my-dataset

Name      Description      Xact      Created      
✓ Created snapshot 'baseline' for 'my-dataset' (xact 1000197309515474282).
{"dataset":{"created":"2026-06-09T00:14:47.512Z","created_at":null,"description":null,"id":"ae6420c8-713f-40b5-b3af-25f34100d4e5","metadata":null,"name":"my-dataset","project_id":"e91206c4-9b2f-4222-9891-5412a6528e5d"},"found_existing":false,"mode":"snapshot_create","snapshot":{"created":"2026-06-09T00:14:54.528Z","dataset_id":"ae6420c8-713f-40b5-b3af-25f34100d4e5","description":null,"id":"a3b7b38d-8dc4-41da-bc84-e1eebf80aa1e","name":"baseline-again","xact_id":"1000192656880881099"}}
Restore preview for my-dataset to snapshot 'baseline' (xact 1000197309515474282):

Rows to restore: 0
Rows to delete: 0

Restore dataset 'my-dataset' to snapshot 'baseline' (xact 1000197309515474282)? yes
✗ Failed to restore dataset 'my-dataset' to snapshot 'baseline' (xact 1000197309515474282)
error: failed to parse response
Restore preview for my-dataset to xact 1000192656880881099:

Rows to restore: 0
Rows to delete: 1

✓ Restored dataset 'my-dataset' to xact 1000192656880881099 (xact 1000197309516552558; 0 restored, 1 deleted).

Is it normal that I get ✗ Failed to restore dataset 'my-dataset' to snapshot 'baseline' ?

Cedric / ViaDézo1er (viadezo1er) · 2026-06-09T00:22:38Z

echo 'bt datasets delete my-dataset'|sh
✓ Deleted 'my-dataset'
Run `bt datasets list` to see remaining datasets.

When in non interactive mode, the delete command does not require the --force flag, but it's required for the restore command. I think it would be better if the -f/--force flag was required for delete too.

max-braintrust · 2026-06-09T20:34:25Z

❯ bt -V
bt 0.10.1-canary.aa9ca1c4cfb3
❯ bt status
org: ced-test-1
project: input
profile: profile
user: Cédric Halber (cedric@braintrustdata.com)
source: /Users/cedric@braintrustdata.com/.config/bt/config.json
❯ bt auth profiles
✓ profile — oauth — Cédric Halber (cedric@braintrustdata.com)

Credentials: /Users/cedric@braintrustdata.com/.config/bt/auth.json
❯ # Create a dataset
bt datasets create my-dataset --rows '[{"id":"case-1","input":{"text":"hi"},"expected":"hello"}]'

# List snapshots
bt datasets snapshots list my-dataset

# Create a snapshot
bt datasets snapshots create my-dataset baseline

# Re-run create for the same xact to verify found_existing handling
bt datasets snapshots create my-dataset baseline-again --xact-id 1000192656880881099 --json

# Restore by snapshot name
bt datasets snapshots restore my-dataset --name baseline

# Restore by xact ID
bt datasets snapshots restore my-dataset --snapshot 1000192656880881099 --force
✓ Created 'my-dataset' with 1 records
0 snapshots found for ced-test-1 / input / my-dataset

Name      Description      Xact      Created      
✓ Created snapshot 'baseline' for 'my-dataset' (xact 1000197309515474282).
{"dataset":{"created":"2026-06-09T00:14:47.512Z","created_at":null,"description":null,"id":"ae6420c8-713f-40b5-b3af-25f34100d4e5","metadata":null,"name":"my-dataset","project_id":"e91206c4-9b2f-4222-9891-5412a6528e5d"},"found_existing":false,"mode":"snapshot_create","snapshot":{"created":"2026-06-09T00:14:54.528Z","dataset_id":"ae6420c8-713f-40b5-b3af-25f34100d4e5","description":null,"id":"a3b7b38d-8dc4-41da-bc84-e1eebf80aa1e","name":"baseline-again","xact_id":"1000192656880881099"}}
Restore preview for my-dataset to snapshot 'baseline' (xact 1000197309515474282):

Rows to restore: 0
Rows to delete: 0

Restore dataset 'my-dataset' to snapshot 'baseline' (xact 1000197309515474282)? yes
✗ Failed to restore dataset 'my-dataset' to snapshot 'baseline' (xact 1000197309515474282)
error: failed to parse response
Restore preview for my-dataset to xact 1000192656880881099:

Rows to restore: 0
Rows to delete: 1

✓ Restored dataset 'my-dataset' to xact 1000192656880881099 (xact 1000197309516552558; 0 restored, 1 deleted).

Is it normal that I get ✗ Failed to restore dataset 'my-dataset' to snapshot 'baseline' ?

We were not parsing the no-op case correctly - if nothing gets written (ie you run restore on case where there is no diff) then no xact id is returned.

Fixed this, it should display an appropriate message now.

Cedric / ViaDézo1er (viadezo1er) · 2026-06-09T21:21:23Z

❯ git pull
Already up to date.
❯ cargo build
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.21s
❯ bt datasets create my-dataset --rows '[{"id":"case-1","input":{"text":"hi"},"expected":"hello"}]'

⠋ Creating dataset...                                                                                                                        ✓ Created 'my-dataset' with 1 records
❯ bt datasets list
1 dataset found in ced-test-1 / My Project

Name            Description      Created         
my-dataset      -                2026-06-0…      
❯ echo 'bt datasets delete my-dataset'|sh
✓ Deleted 'my-dataset'
Run `bt datasets list` to see remaining datasets.
❯ bt datasets list
0 datasets found in ced-test-1 / My Project

Name      Description      Created

I think the -f flag is not required in non interactive mode?

max-braintrust · 2026-06-09T21:37:27Z

❯ git pull
Already up to date.
❯ cargo build
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.21s
❯ bt datasets create my-dataset --rows '[{"id":"case-1","input":{"text":"hi"},"expected":"hello"}]'

⠋ Creating dataset...                                                                                                                        ✓ Created 'my-dataset' with 1 records
❯ bt datasets list
1 dataset found in ced-test-1 / My Project

Name            Description      Created         
my-dataset      -                2026-06-0…      
❯ echo 'bt datasets delete my-dataset'|sh
✓ Deleted 'my-dataset'
Run `bt datasets list` to see remaining datasets.
❯ bt datasets list
0 datasets found in ced-test-1 / My Project

Name      Description      Created

I think the -f flag is not required in non interactive mode?

oh sorry misread, I added it to bt datasets snapshots delete, not bt datasets delete - should require confirmation or -f for dataset delete as well now.

Cedric / ViaDézo1er (viadezo1er) · 2026-06-09T22:01:13Z

I got confused yesterday (was a bit ill....) I was thinking of snapshot delete that's on me 😅 (though I also think requiring the -f flag for bt datasets delete is good).

Cedric / ViaDézo1er (viadezo1er) · 2026-06-09T22:01:42Z

Anyway lgtm

init snapshot work

bb15342

max-braintrust changed the title ~~Add bt datasets snapshots for better dataset version control~~ Add bt datasets snapshots for dataset version control May 6, 2026

max-braintrust requested a review from Parker Henderson (parkerhendo) May 6, 2026 21:40

max-braintrust added 2 commits May 6, 2026 16:07

Merge branch 'main' into max/add-dataset-snapshots

e12d447

fix build error

f10e020

max-braintrust added 9 commits May 6, 2026 16:27

fix clippy

2364e75

Merge branch 'main' into max/add-dataset-snapshots

041e3cc

Merge branch 'main' into max/add-dataset-snapshots

59f4234

Merge branch 'main' into max/add-dataset-snapshots

270c488

add back test and fix imports

3f762b3

Merge branch 'main' into max/add-dataset-snapshots

bfb30cf

fix merge

56d5516

fix fixture

b9fab8d

Merge branch 'main' into max/add-dataset-snapshots

aa9ca1c

Cedric / ViaDézo1er (viadezo1er) reviewed Jun 5, 2026

View reviewed changes

Comment thread src/datasets/api.rs Outdated

Cedric / ViaDézo1er (viadezo1er) approved these changes Jun 5, 2026

View reviewed changes

max-braintrust added 3 commits June 8, 2026 12:05

cleanup

0157716

update snapshot fixtures

2740374

format fixture

5216ec3

handle no write change correctly

599bcef

require delete confirmation

244f9c9

add restore test coverage

ad9934c

Cedric / ViaDézo1er (viadezo1er) approved these changes Jun 9, 2026

View reviewed changes

max-braintrust marked this pull request as ready for review June 9, 2026 22:06

max-braintrust merged commit e54bcc3 into main Jun 9, 2026
32 checks passed

Conversation

max-braintrust commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TL;DR

What changed?

Added dataset snapshot management functionality:

Added test coverage:

How to test?

Uh oh!

github-actions Bot commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Cedric / ViaDézo1er (viadezo1er) left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Cedric / ViaDézo1er (viadezo1er) commented Jun 5, 2026

Uh oh!

Cedric / ViaDézo1er (viadezo1er) commented Jun 5, 2026

Uh oh!

Cedric / ViaDézo1er (viadezo1er) commented Jun 5, 2026

Uh oh!

Cedric / ViaDézo1er (viadezo1er) commented Jun 5, 2026

Uh oh!

Cedric / ViaDézo1er (viadezo1er) commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Cedric / ViaDézo1er (viadezo1er) commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

max-braintrust commented Jun 9, 2026

Uh oh!

Cedric / ViaDézo1er (viadezo1er) commented Jun 9, 2026

Uh oh!

max-braintrust commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Cedric / ViaDézo1er (viadezo1er) commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Cedric / ViaDézo1er (viadezo1er) commented Jun 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

max-braintrust commented May 6, 2026 •

edited

Loading

github-actions Bot commented May 6, 2026 •

edited

Loading

Cedric / ViaDézo1er (viadezo1er) left a comment •

edited

Loading

Cedric / ViaDézo1er (viadezo1er) commented Jun 9, 2026 •

edited

Loading

Cedric / ViaDézo1er (viadezo1er) commented Jun 9, 2026 •

edited

Loading

max-braintrust commented Jun 9, 2026 •

edited

Loading

Cedric / ViaDézo1er (viadezo1er) commented Jun 9, 2026 •

edited

Loading