Skip to content

Commit 48b78a0

Browse files
authored
✨ feat(tox-toml-fmt): add inline table key reordering (#264)
Inline tables in `tox.toml` (like `{ replace = "default", default = ".", extend = true }`) had no enforced key ordering, leading to inconsistent formatting across projects. This PR adds schema-driven key reordering for the four tox inline table types — `replace`, `prefix`, `product`, and `value` — so keys always appear in a canonical order that puts the discriminator key first. ✨ The reordering logic lives in the shared `common` crate as a generic `InlineTableSchema` + `reorder_inline_table_keys` API, making it reusable by other formatters. Each schema declares a discriminator key to identify which inline tables it applies to and the desired key order. The implementation uses the parse-and-extract pattern to rebuild reordered inline tables, and collects all target nodes before mutation to avoid iterator invalidation. Beyond inline tables, this PR also brings the formatter up to date with recent tox features: 8 new environment keys (`factors`, `default_base_python`, `virtualenv_spec`, `pylock`, `recreate_commands`, `fail_fast`, `commands_retry`, `extra_setup_commands`), `env_base.*` table support, proper `-r`/`-c` file reference handling in `deps`/`constraints`, alphabetical sorting of remaining envs not in `env_list`, and product expansion inline tables excluded from `env_list` sorting. 📝 Documentation and `CONTRIBUTING.md` have been updated to reflect all changes.
1 parent 610d290 commit 48b78a0

16 files changed

Lines changed: 2443 additions & 89 deletions

CONTRIBUTING.md

Lines changed: 36 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -330,9 +330,14 @@ fn test_load_text(#[case] input: &str, #[case] kind: SyntaxKind, #[case] expecte
330330

331331
### Coverage Goals and Measurement
332332

333-
We require **98% line coverage for Rust code** and **100% coverage for Python code**. To generate an HTML coverage
334-
report for Rust code, run `tox r -e coverage` from the repository root. This generates lcov output and opens an HTML
335-
report in your browser. For a quick summary, use `cargo llvm-cov --workspace --no-default-features --summary-only`.
333+
We require **98% line coverage for Rust code** and **100% coverage for Python code**. Additionally, **diff coverage must
334+
be 100% per test suite** — changes to `common/src/` must be fully covered by the common test suite alone, and changes to
335+
`tox-toml-fmt/rust/src/` must be fully covered by the tox-toml-fmt test suite alone. Use per-package coverage checks to
336+
verify: `cargo llvm-cov -p common --summary-only` or `cargo llvm-cov -p tox-toml-fmt --summary-only`.
337+
338+
To generate an HTML coverage report for Rust code, run `tox r -e coverage` from the repository root. This generates lcov
339+
output and opens an HTML report in your browser. For a quick summary, use
340+
`cargo llvm-cov --workspace --no-default-features --summary-only`.
336341

337342
#### Testing PyO3 Code from Rust
338343

@@ -414,27 +419,30 @@ fn test_format(#[case] input: &str, #[case] expected: &str) {
414419
}
415420
```
416421

417-
Snapshot testing approach (preferred):
422+
Snapshot testing approach (preferred, using inline snapshots):
418423

419424
```rust
420425
#[rstest]
421426
#[case::simple("input")]
422427
fn test_format(#[case] input: &str) {
423428
let result = format_toml(input);
424-
insta::assert_snapshot!(result);
429+
insta::assert_snapshot!(result, @"");
425430
}
426431
```
427432

433+
The `@""` syntax creates an **inline snapshot** where the expected value is stored directly in the test file. This is
434+
preferred over file-based snapshots because it keeps the expected output next to the test input, making tests easier to
435+
read and review.
436+
428437
Snapshot testing workflow:
429438

430-
- Run tests with `cargo insta test` to generate snapshots
431-
- Review changes with `cargo insta review` (interactive) or view diffs manually
439+
- Run tests with `cargo insta test --accept` to populate inline snapshots
440+
- Review changes with `cargo insta review` (interactive) or view diffs in the test file directly
432441
- Accept all changes with `cargo insta test --accept`
433442
- Reject changes with `cargo insta reject`
434443

435444
When formatter behavior changes (like switching parsers), you can update all test expectations with a single
436-
`cargo insta test --accept` instead of manually updating hundreds of inline strings. Snapshots are stored in
437-
`src/tests/snapshots/` and committed to git.
445+
`cargo insta test --accept` instead of manually updating hundreds of inline strings.
438446

439447
## Common Patterns
440448

@@ -468,6 +476,25 @@ update_content(value_node, |text| {
468476
});
469477
```
470478

479+
### Reordering Inline Table Keys
480+
481+
When a formatter needs to enforce a consistent key order within inline tables, use the `InlineTableSchema` and
482+
`reorder_inline_table_keys` from `common::table`. Each schema specifies a discriminator key (used to identify which
483+
schema applies) and the desired key order. Keys not listed in the schema are appended at the end.
484+
485+
```rust
486+
use common::table::{reorder_inline_table_keys, InlineTableSchema};
487+
488+
const SCHEMAS: &[InlineTableSchema] = &[
489+
InlineTableSchema {
490+
discriminator: "replace",
491+
key_order: &["replace", "default", "extend"],
492+
},
493+
];
494+
495+
reorder_inline_table_keys(&root_ast, SCHEMAS);
496+
```
497+
471498
### Creating New Nodes
472499

473500
When you need to create new syntax nodes, use the functions in `common::create`. These use the parse-and-extract pattern

common/src/string.rs

Lines changed: 44 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -298,29 +298,57 @@ where
298298
}
299299
}
300300

301+
fn is_valid_bare_key(s: &str) -> bool {
302+
!s.is_empty() && s.chars().all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-')
303+
}
304+
305+
fn normalize_key_segment(kind: SyntaxKind, text: &str) -> String {
306+
match kind {
307+
BARE_KEY => text.to_string(),
308+
LITERAL_STRING => {
309+
let inner = &text[1..text.len() - 1];
310+
if is_valid_bare_key(inner) {
311+
return inner.to_string();
312+
}
313+
let escaped = inner.replace('\\', "\\\\").replace('"', "\\\"");
314+
format!("\"{escaped}\"")
315+
}
316+
BASIC_STRING => {
317+
let inner = &text[1..text.len() - 1];
318+
if is_valid_bare_key(inner) {
319+
return inner.to_string();
320+
}
321+
text.to_string()
322+
}
323+
_ => text.to_string(),
324+
}
325+
}
326+
301327
pub fn normalize_key_quotes(root: &SyntaxNode) {
302328
use crate::create::make_key;
303329

304-
for descendant in root.descendants() {
305-
if descendant.kind() != KEYS {
306-
continue;
307-
}
308-
let has_literal = descendant.children_with_tokens().any(|c| c.kind() == LITERAL_STRING);
309-
if !has_literal {
330+
let keys_nodes: Vec<SyntaxNode> = root.descendants().filter(|n| n.kind() == KEYS).collect();
331+
for descendant in keys_nodes {
332+
let needs_normalization = descendant.children_with_tokens().any(|c| {
333+
let kind = c.kind();
334+
if kind == LITERAL_STRING {
335+
return true;
336+
}
337+
if kind == BASIC_STRING {
338+
let text = c.to_string();
339+
let inner = &text[1..text.len() - 1];
340+
return is_valid_bare_key(inner);
341+
}
342+
false
343+
});
344+
if !needs_normalization {
310345
continue;
311346
}
312347
let mut key_parts = Vec::new();
313348
for child in descendant.children_with_tokens() {
314-
match child.kind() {
315-
BARE_KEY => key_parts.push(child.to_string()),
316-
LITERAL_STRING => {
317-
let text = child.to_string();
318-
let inner = &text[1..text.len() - 1];
319-
let escaped = inner.replace('\\', "\\\\").replace('"', "\\\"");
320-
key_parts.push(format!("\"{escaped}\""));
321-
}
322-
BASIC_STRING => key_parts.push(child.to_string()),
323-
_ => {}
349+
let kind = child.kind();
350+
if matches!(kind, BARE_KEY | LITERAL_STRING | BASIC_STRING) {
351+
key_parts.push(normalize_key_segment(kind, &child.to_string()));
324352
}
325353
}
326354
let new_key = make_key(&key_parts.join("."));

common/src/table.rs

Lines changed: 77 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,8 @@ use std::ops::Index;
55

66
use tombi_syntax::SyntaxKind::{
77
ARRAY_OF_TABLE, BARE_KEY, BASIC_STRING, BRACKET_END, BRACKET_START, COMMENT, DANGLING_COMMENT_GROUP,
8-
DOUBLE_BRACKET_START, EQUAL, KEY_VALUE, KEY_VALUE_GROUP, KEY_VALUE_WITH_COMMA_GROUP, KEYS, LINE_BREAK,
9-
LITERAL_STRING, TABLE, WHITESPACE,
8+
DOUBLE_BRACKET_START, EQUAL, INLINE_TABLE, KEY_VALUE, KEY_VALUE_GROUP, KEY_VALUE_WITH_COMMA_GROUP, KEYS,
9+
LINE_BREAK, LITERAL_STRING, TABLE, WHITESPACE,
1010
};
1111
use tombi_syntax::{SyntaxElement, SyntaxKind, SyntaxNode};
1212

@@ -1063,3 +1063,78 @@ fn add_intermediate_parents(table_name: &str, prefix_dots: usize, result: &mut V
10631063
current = parent;
10641064
}
10651065
}
1066+
1067+
pub struct InlineTableSchema {
1068+
pub discriminator: &'static str,
1069+
pub key_order: &'static [&'static str],
1070+
}
1071+
1072+
fn inline_table_key_name(kv_node: &SyntaxNode) -> String {
1073+
let raw = kv_node
1074+
.children_with_tokens()
1075+
.find(|c| c.kind() == KEYS)
1076+
.and_then(|c| c.as_node().map(|n| n.text().to_string().trim().to_string()))
1077+
.unwrap_or_default();
1078+
raw.strip_prefix('"')
1079+
.and_then(|s| s.strip_suffix('"'))
1080+
.or_else(|| raw.strip_prefix('\'').and_then(|s| s.strip_suffix('\'')))
1081+
.unwrap_or(&raw)
1082+
.to_string()
1083+
}
1084+
1085+
fn detect_schema<'a>(keys: &[String], schemas: &'a [InlineTableSchema]) -> Option<&'a [&'static str]> {
1086+
schemas
1087+
.iter()
1088+
.find(|s| keys.iter().any(|k| k == s.discriminator))
1089+
.map(|s| s.key_order)
1090+
}
1091+
1092+
fn reorder_single_inline_table(node: &SyntaxNode, schemas: &[InlineTableSchema]) {
1093+
let kv_pairs: Vec<(String, String)> = node
1094+
.children()
1095+
.filter(|n| n.kind() == KEY_VALUE_WITH_COMMA_GROUP)
1096+
.flat_map(|group| {
1097+
group
1098+
.children()
1099+
.filter(|n| n.kind() == KEY_VALUE)
1100+
.map(|kv| (inline_table_key_name(&kv), kv.text().to_string().trim().to_string()))
1101+
.collect::<Vec<_>>()
1102+
})
1103+
.collect();
1104+
1105+
if kv_pairs.len() < 2 {
1106+
return;
1107+
}
1108+
1109+
let keys: Vec<String> = kv_pairs.iter().map(|(k, _)| k.clone()).collect();
1110+
let Some(schema) = detect_schema(&keys, schemas) else {
1111+
return;
1112+
};
1113+
1114+
let key_position = |k: &str| -> usize { schema.iter().position(|s| *s == k).unwrap_or(usize::MAX) };
1115+
let mut sorted = kv_pairs.clone();
1116+
sorted.sort_by_key(|(k, _)| key_position(k));
1117+
1118+
if sorted.iter().map(|(k, _)| k).eq(kv_pairs.iter().map(|(k, _)| k)) {
1119+
return;
1120+
}
1121+
1122+
let entries: Vec<&str> = sorted.iter().map(|(_, v)| v.as_str()).collect();
1123+
let rebuilt = format!("_x = {{ {} }}\n", entries.join(", "));
1124+
let parsed = parse(&rebuilt);
1125+
let new_children: Option<Vec<SyntaxElement>> = parsed
1126+
.descendants()
1127+
.find(|n| n.kind() == INLINE_TABLE)
1128+
.map(|n| n.children_with_tokens().collect());
1129+
1130+
if let Some(children) = new_children {
1131+
node.splice_children(0..node.children_with_tokens().count(), children);
1132+
}
1133+
}
1134+
1135+
pub fn reorder_inline_table_keys(root_ast: &SyntaxNode, schemas: &[InlineTableSchema]) {
1136+
let inline_tables: Vec<SyntaxNode> = root_ast.descendants().filter(|n| n.kind() == INLINE_TABLE).collect();
1137+
for node in inline_tables {
1138+
reorder_single_inline_table(&node, schemas);
1139+
}
1140+
}

common/src/tests/string_tests.rs

Lines changed: 24 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -602,8 +602,8 @@ fn test_strip_quotes_triple_quotes() {
602602
fn test_normalize_key_quotes_simple_literal() {
603603
let root = parse("'key' = 1\n");
604604
normalize_key_quotes(&root);
605-
insta::assert_snapshot!(root.to_string(), @r#""key" = 1
606-
"#);
605+
insta::assert_snapshot!(root.to_string(), @"key = 1
606+
");
607607
}
608608

609609
#[test]
@@ -615,11 +615,11 @@ fn test_normalize_key_quotes_bare_key_unchanged() {
615615
}
616616

617617
#[test]
618-
fn test_normalize_key_quotes_basic_string_unchanged() {
618+
fn test_normalize_key_quotes_basic_string_stripped() {
619619
let root = parse("\"key\" = 1\n");
620620
normalize_key_quotes(&root);
621-
insta::assert_snapshot!(root.to_string(), @r#""key" = 1
622-
"#);
621+
insta::assert_snapshot!(root.to_string(), @"key = 1
622+
");
623623
}
624624

625625
#[test]
@@ -674,15 +674,31 @@ fn test_normalize_key_quotes_literal_with_backslash_and_quote() {
674674
fn test_normalize_key_quotes_multiple_literal_segments() {
675675
let root = parse("'first'.'second' = 1\n");
676676
normalize_key_quotes(&root);
677-
insta::assert_snapshot!(root.to_string(), @r#""first"."second" = 1
678-
"#);
677+
insta::assert_snapshot!(root.to_string(), @"first.second = 1
678+
");
679679
}
680680

681681
#[test]
682682
fn test_normalize_key_quotes_preserves_basic_in_dotted() {
683683
let root = parse("bare.\"basic\".'literal' = 1\n");
684684
normalize_key_quotes(&root);
685-
insta::assert_snapshot!(root.to_string(), @r#"bare."basic"."literal" = 1
685+
insta::assert_snapshot!(root.to_string(), @"bare.basic.literal = 1
686+
");
687+
}
688+
689+
#[test]
690+
fn test_normalize_key_quotes_preserves_required_quotes() {
691+
let root = parse("\"key with spaces\" = 1\n");
692+
normalize_key_quotes(&root);
693+
insta::assert_snapshot!(root.to_string(), @r#""key with spaces" = 1
694+
"#);
695+
}
696+
697+
#[test]
698+
fn test_normalize_key_quotes_inline_table_keys() {
699+
let root = parse("val = { \"else\" = \"no\", \"then\" = \"yes\" }\n");
700+
normalize_key_quotes(&root);
701+
insta::assert_snapshot!(root.to_string(), @r#"val = { else = "no", then = "yes" }
686702
"#);
687703
}
688704

0 commit comments

Comments
 (0)