Skip to content

Commit 4cf6f54

Browse files
committed
fix(config): let only one bag claim a placeholder's address, +3 UI defects
The first finding corrects this branch's own reasoning. When the addressed placeholder was introduced I considered requiring the address to match the slot it appears in and rejected it, because a legitimate removal shifts positions. Right rejection, wrong rule: the address never needed to match the POSITION, it needed to be claimed by only one bag. Duplicating a section by hand — which the new banners tell users to do for anything the wizard cannot edit — copies its placeholders too, and both copies resolved to the same stored credential. Many placeholders inside one bag share its address and always will; two different bags naming the same one is now a 400. A transient GET /sources failure repointed a report: the Builder cleared source.ref when the list came back empty, and a failed call was indistinguishable from an empty one, so a blip converted a registry-backed report into an inline one on save. Format and destination ids were matched ordinally while the engine resolves them case-insensitively — a stored "CSV" with the csv checkbox ticked counted as two formats and saved two CSV outputs; the same on destination types dropped the stored properties. A 404 on PUT was reported as an unreachable engine. A report deleted from another tab is a rejected request carrying a usable message, and mapping it to Unavailable threw that message away and blamed the network. Each fix verified by reverting it and confirming the new test fails, then the full suite — not only the projects I thought I had touched.
1 parent e3365f2 commit 4cf6f54

10 files changed

Lines changed: 209 additions & 17 deletions

File tree

CHANGELOG.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,18 @@ The `NeoReports.Abstractions` contract follows SemVer strictly.
5252
- **An edited source property keeps its JSON kind.** Editing `90` turned it into `"90"`; only
5353
untouched rows were protected. The kind is carried from the stored value, never guessed from the
5454
text, so an all-digits account id stays a string.
55+
- **Two sections cannot both claim the same stored credential.** Duplicating an output or destination
56+
block by hand copies its placeholder too, and both copies used to resolve to the same secret. An
57+
address may now be claimed by one property bag only; many placeholders *inside* one bag still share
58+
it, as they always must.
59+
- **A transient `GET /api/sources` failure no longer repoints a report.** The Builder cleared
60+
`source.ref` when the list came back empty, and a failed call was indistinguishable from an empty
61+
one — so a blip converted a registry-backed report into an inline one on save.
62+
- **Format and destination ids are matched case-insensitively**, the way the engine resolves them; a
63+
stored `"CSV"` with `csv` ticked used to save two CSV outputs, and the same on destination types
64+
dropped the stored properties.
65+
- **A `404` when saving an edit shows the engine's message**, not "the engine is not reachable" — a
66+
report deleted from another tab is a rejected request, not a network failure.
5567

5668
### Added
5769
- **`GET /api/reports/{name}/config`** returns a config-origin report's stored document with

DECISIONS.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1961,6 +1961,28 @@ authorization:
19611961
had when the new text still fits it, and stays a string otherwise — the kind is carried, never
19621962
guessed from the text, so an all-digits account id does not silently become a number.
19631963

1964+
### The fourth review pass
1965+
1966+
A `/code-review` after the non-string fix found four more, and the first one is worth reading as a
1967+
correction to this ADR's own reasoning above. When the addressed placeholder was introduced I
1968+
considered requiring an address to match the slot it appears in, and rejected it because a legitimate
1969+
removal shifts positions. That was the right rejection of the wrong rule: the address never needed to
1970+
match the *position*, it needed to be claimed by only **one** bag. Duplicating a section by hand —
1971+
which the new banners tell users to do for anything the wizard cannot edit — copies its placeholders
1972+
too, and both copies resolved to the same stored credential. Many placeholders inside one bag share
1973+
its address and always will; two different bags naming the same one is now a 400.
1974+
1975+
- **A transient `GET /sources` failure repointed a report.** The Builder cleared `source.ref` when the
1976+
registered-source list came back empty, and a failed call was indistinguishable from an empty one —
1977+
so a blip converted a registry-backed report into an inline one on save, silently changing what it
1978+
reads from.
1979+
- **Format and destination ids were matched ordinally** while the engine resolves them
1980+
case-insensitively: a stored `"format": "CSV"` with the `csv` checkbox ticked counted as two formats
1981+
and saved two CSV outputs; the same on destination types dropped the stored properties.
1982+
- **A `404` on `PUT` was reported as an unreachable engine.** A report deleted from another tab is a
1983+
rejected request carrying a usable message, and mapping it to *Unavailable* threw that message away
1984+
and blamed the network.
1985+
19641986
### Verified end to end
19651987

19661988
Driven in a browser against `samples/09-web-ui-live`: a report carrying a literal password, a

src/NeoReports.Core/Configuration/ReportConfigSecrets.cs

Lines changed: 38 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -157,8 +157,14 @@ public static string Restore(string document, string storedDocument)
157157
JsonObject root = ParseObject(document);
158158
JsonObject stored = ParseObject(storedDocument);
159159

160+
// An address may be claimed by at most one incoming bag. Many placeholders inside one bag
161+
// share its address (accessKey and secretKey of the same destination), which is fine; two
162+
// *different* bags naming the same one is not — duplicating a section by hand copies its
163+
// placeholders too, and resolving both would hand a second destination a credential the
164+
// editor cannot see, which is the outcome the address exists to prevent.
165+
var claims = new Dictionary<string, JsonObject>(StringComparer.Ordinal);
160166
foreach ((JsonObject bag, string? _) in PropertyBagSlots(root))
161-
RestoreMembers(bag, stored, []);
167+
RestoreMembers(bag, new RestoreContext(stored, bag, claims), []);
162168

163169
return root.ToJsonString();
164170
}
@@ -229,20 +235,25 @@ private static void RedactMembers(JsonObject owner, string sentinel)
229235
}
230236
}
231237

232-
private static void RestoreMembers(JsonObject bag, JsonObject stored, IReadOnlyList<object> path)
238+
/// <param name="Stored">The whole stored document — a placeholder names the slot it came from.</param>
239+
/// <param name="Bag">The incoming property bag currently being restored.</param>
240+
/// <param name="Claims">Which bag has already claimed each address, so no two can share one.</param>
241+
private sealed record RestoreContext(JsonObject Stored, JsonObject Bag, Dictionary<string, JsonObject> Claims);
242+
243+
private static void RestoreMembers(JsonObject owner, RestoreContext context, IReadOnlyList<object> path)
233244
{
234-
foreach (string key in bag.Select(pair => pair.Key).ToArray())
235-
Replace(bag, key, RestoreValue(bag[key], stored, [.. path, key]));
245+
foreach (string key in owner.Select(pair => pair.Key).ToArray())
246+
Replace(owner, key, RestoreValue(owner[key], context, [.. path, key]));
236247
}
237248

238249
// Inside a property bag an array is ordered data, so an element's address is its index. Elements
239250
// go through the same RestoreValue as members, which is what makes a redacted *scalar* element
240251
// work — `"mirrors": ["…", "…"]` with a credential in one of them.
241-
private static void RestoreElements(JsonArray array, JsonObject stored, IReadOnlyList<object> path)
252+
private static void RestoreElements(JsonArray array, RestoreContext context, IReadOnlyList<object> path)
242253
{
243254
for (var i = 0; i < array.Count; i++)
244255
{
245-
JsonNode? restored = RestoreValue(array[i], stored, [.. path, i]);
256+
JsonNode? restored = RestoreValue(array[i], context, [.. path, i]);
246257
if (!ReferenceEquals(array[i], restored))
247258
array[i] = restored;
248259
}
@@ -254,16 +265,18 @@ private static void RestoreElements(JsonArray array, JsonObject stored, IReadOnl
254265
/// children restored in place.
255266
/// </summary>
256267
/// <param name="value">The incoming value.</param>
257-
/// <param name="stored">The whole stored document — a placeholder names the slot it came from.</param>
268+
/// <param name="context">The stored document, the bag being restored, and the addresses claimed so far.</param>
258269
/// <param name="path">Where inside its bag this value sits, as member names and array indices.</param>
259-
private static JsonNode? RestoreValue(JsonNode? value, JsonObject stored, IReadOnlyList<object> path)
270+
private static JsonNode? RestoreValue(JsonNode? value, RestoreContext context, IReadOnlyList<object> path)
260271
{
261272
if (value is JsonValue sentinelValue
262273
&& sentinelValue.TryGetValue(out string? text)
263274
&& IsRedactedPlaceholder(text))
264275
{
265276
string? address = AddressOf(text);
266-
if (!TryResolveStored(stored, address, path, out JsonNode? original))
277+
ClaimAddress(context, address);
278+
279+
if (!TryResolveStored(context.Stored, address, path, out JsonNode? original))
267280
{
268281
throw new ConfigurationException(
269282
$"The property at '{Describe(address, path)}' was sent as a redacted placeholder, " +
@@ -276,11 +289,11 @@ private static void RestoreElements(JsonArray array, JsonObject stored, IReadOnl
276289
switch (value)
277290
{
278291
case JsonObject nested:
279-
RestoreMembers(nested, stored, path);
292+
RestoreMembers(nested, context, path);
280293
break;
281294

282295
case JsonArray array:
283-
RestoreElements(array, stored, path);
296+
RestoreElements(array, context, path);
284297
break;
285298

286299
default:
@@ -290,6 +303,20 @@ private static void RestoreElements(JsonArray array, JsonObject stored, IReadOnl
290303
return value;
291304
}
292305

306+
private static void ClaimAddress(RestoreContext context, string? address)
307+
{
308+
string claimed = address ?? SourceMember;
309+
if (context.Claims.TryGetValue(claimed, out JsonObject? owner) && !ReferenceEquals(owner, context.Bag))
310+
{
311+
throw new ConfigurationException(
312+
$"Two different sections both sent a redacted placeholder for '{claimed}'. A section copied " +
313+
"from another one needs its own value — the placeholder only stands for the value of the " +
314+
"section it came from.");
315+
}
316+
317+
context.Claims[claimed] = context.Bag;
318+
}
319+
293320
/// <summary>
294321
/// Follows a placeholder's address to its stored property bag and then <paramref name="path"/>
295322
/// inside it. Returns <c>false</c> when any step is missing; a stored JSON <c>null</c> resolves

src/UI/NeoReports.UI/Pages/Builder.razor

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,9 @@
9393
private IReadOnlyList<string> _engineSources = Array.Empty<string>();
9494
private IReadOnlyList<ApiSourceView> _registeredSources = Array.Empty<ApiSourceView>();
9595

96+
/// <summary>Whether the registered-source list actually loaded, as opposed to failing to.</summary>
97+
private bool _sourcesLoaded;
98+
9699
/// <summary>
97100
/// Set by ReportDetail's "Edit" button (<c>?edit=name</c>) — hydrates the wizard from that
98101
/// report instead of starting blank.
@@ -132,7 +135,13 @@
132135
}
133136

134137
_engineSources = capabilities.Sources;
135-
_registeredSources = await Api.TryListSourcesAsync() ?? Array.Empty<ApiSourceView>();
138+
139+
// null means the call failed, which is NOT the same as "this host has no registered sources".
140+
// Conflating them cleared an edited report's source.ref on a transient blip and saved it back
141+
// as an inline source — a silent repointing of what the report reads from.
142+
IReadOnlyList<ApiSourceView>? registered = await Api.TryListSourcesAsync();
143+
_sourcesLoaded = registered is not null;
144+
_registeredSources = registered ?? Array.Empty<ApiSourceView>();
136145

137146
// Loaded after the registered sources on purpose: a ref-based document names a source but
138147
// not its type (the type belongs to the registry, D42), and the Configure step's editor
@@ -150,7 +159,7 @@
150159
if (!Wizard.IsEditing && !_engineSources.Contains(Wizard.SourceType))
151160
Wizard.SourceType = _engineSources.Contains("sql") ? "sql" : _engineSources[0];
152161

153-
if (!string.IsNullOrEmpty(Wizard.SourceRef) && _registeredSources.All(s => s.Name != Wizard.SourceRef))
162+
if (_sourcesLoaded && !string.IsNullOrEmpty(Wizard.SourceRef) && _registeredSources.All(s => s.Name != Wizard.SourceRef))
154163
Wizard.SourceRef = "";
155164
}
156165

src/UI/NeoReports.UI/Services/BuilderConfigMapper.cs

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,10 @@ public static bool Hydrate(BuilderState state, string document, Func<string, str
113113
.Select(format => format!)
114114
.ToArray();
115115

116-
state.Formats = formats.ToHashSet(StringComparer.Ordinal);
116+
// The engine resolves format and destination ids case-insensitively, so the wizard has to
117+
// as well: a stored "CSV" with the "csv" checkbox ticked otherwise counts as two formats
118+
// and saves two CSV outputs.
119+
state.Formats = formats.ToHashSet(StringComparer.OrdinalIgnoreCase);
117120

118121
// The Format step is a set of checkboxes, so it cannot represent "two csv outputs" —
119122
// nothing stops a config document declaring them, with different writer properties each.
@@ -360,7 +363,7 @@ private static JsonArray BuildOutputs(BuilderState state, JsonArray? original)
360363
// entirely by properties the wizard cannot show, and must not be flattened by an edit.
361364
JsonObject[] stored = original?
362365
.OfType<JsonObject>()
363-
.Where(output => string.Equals((Get(output, "format") as JsonValue)?.ToString(), format, StringComparison.Ordinal))
366+
.Where(output => string.Equals((Get(output, "format") as JsonValue)?.ToString(), format, StringComparison.OrdinalIgnoreCase))
364367
.ToArray() ?? [];
365368

366369
if (stored.Length == 0)
@@ -387,7 +390,7 @@ private static JsonArray BuildOutputs(BuilderState state, JsonArray? original)
387390
{
388391
JsonObject? first = original?.OfType<JsonObject>().FirstOrDefault();
389392
bool sameDestination = first is not null
390-
&& string.Equals((Get(first, "type") as JsonValue)?.ToString(), state.DestinationType, StringComparison.Ordinal);
393+
&& string.Equals((Get(first, "type") as JsonValue)?.ToString(), state.DestinationType, StringComparison.OrdinalIgnoreCase);
391394

392395
// On a type switch the stored properties are dropped: an S3 bucket and region are not
393396
// configuration a local-filesystem destination should inherit.

src/UI/NeoReports.UI/Services/NeoReportsApiClient.cs

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -662,9 +662,13 @@ public async Task<ApiCreateResult> TryReplaceReportAsync(
662662
string? error = await TryReadErrorAsync(response, cancellationToken).ConfigureAwait(false);
663663
// A 409 here is not "name taken" (the name is this report's own) but "this report is
664664
// code-registered", which is a different message entirely — hence Invalid, not NameTaken.
665+
// A 404 means the report was deleted from somewhere else while this wizard was open; it
666+
// is a rejected request carrying a usable message, not an unreachable engine, and mapping
667+
// it to Unavailable threw that message away and blamed the network instead.
665668
ApiCreateOutcome outcome = response.StatusCode switch
666669
{
667-
HttpStatusCode.BadRequest or HttpStatusCode.Conflict => ApiCreateOutcome.Invalid,
670+
HttpStatusCode.BadRequest or HttpStatusCode.Conflict or HttpStatusCode.NotFound =>
671+
ApiCreateOutcome.Invalid,
668672
_ => ApiCreateOutcome.Unavailable,
669673
};
670674
return new ApiCreateResult(outcome, null, error);

tests/NeoReports.Core.UnitTests/ReportConfigSecretsTests.cs

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -500,6 +500,39 @@ public void A_section_without_a_property_bag_does_not_shift_the_addresses_after_
500500
.GetProperty("properties").GetProperty("accessKey").GetString().ShouldBe("KEY-BETA");
501501
}
502502

503+
[Fact]
504+
public void Two_sections_cannot_both_claim_the_same_stored_credential()
505+
{
506+
const string stored = """{"destinations":[{"type":"s3","properties":{"accessKey":"KEY-ALPHA"}}]}""";
507+
// Duplicating a section by hand copies its placeholder too. Resolving both would hand the
508+
// second destination a credential the editor cannot see — the outcome the address exists to
509+
// prevent, arrived at by duplication instead of by inference.
510+
const string duplicated = """
511+
{"destinations":[{"type":"s3","properties":{"accessKey":"${neoreports:redacted:destinations[0]}"}},
512+
{"type":"s3","properties":{"accessKey":"${neoreports:redacted:destinations[0]}"}}]}
513+
""";
514+
515+
Should.Throw<ConfigurationException>(() => ReportConfigSecrets.Restore(duplicated, stored))
516+
.Message.ShouldContain("destinations[0]");
517+
}
518+
519+
[Fact]
520+
public void Several_placeholders_inside_one_section_share_its_address_freely()
521+
{
522+
// The claim is per bag, not per placeholder: one destination's accessKey and secretKey both
523+
// name the same slot, and always will.
524+
const string stored = """
525+
{"destinations":[{"type":"s3","properties":{"accessKey":"KA","secretKey":"KS","bucket":"b"}}]}
526+
""";
527+
528+
string restored = ReportConfigSecrets.Restore(ReportConfigSecrets.Redact(stored), stored);
529+
530+
JsonElement properties = JsonDocument.Parse(restored).RootElement
531+
.GetProperty("destinations").EnumerateArray().Single().GetProperty("properties");
532+
properties.GetProperty("accessKey").GetString().ShouldBe("KA");
533+
properties.GetProperty("secretKey").GetString().ShouldBe("KS");
534+
}
535+
503536
[Fact]
504537
public void A_placeholder_addressing_a_section_that_no_longer_exists_is_rejected()
505538
{

tests/NeoReports.UI.UnitTests/BuilderConfigMapperTests.cs

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -675,6 +675,41 @@ public void An_addressed_placeholder_on_the_connection_is_sent_back_verbatim()
675675
.GetProperty("connectionString").GetString().ShouldBe("${neoreports:redacted}");
676676
}
677677

678+
[Fact]
679+
public void A_format_id_stored_in_a_different_case_is_not_treated_as_a_second_format()
680+
{
681+
var state = new BuilderState();
682+
BuilderConfigMapper.Hydrate(state, """
683+
{"name":"feed","source":{"type":"http"},"outputs":[{"format":"CSV","properties":{"delimiter":";"}}]}
684+
""").ShouldBeTrue();
685+
state.Formats.Add("csv");
686+
687+
using JsonDocument doc = JsonDocument.Parse(BuilderConfigMapper.ToConfigJson(state));
688+
689+
// The engine resolves format ids case-insensitively; matching them ordinally here made a
690+
// stored "CSV" plus a ticked "csv" checkbox save two CSV outputs.
691+
JsonElement[] outputs = doc.RootElement.GetProperty("outputs").EnumerateArray().ToArray();
692+
outputs.Length.ShouldBe(1);
693+
outputs[0].GetProperty(PropertiesMember).GetProperty("delimiter").GetString().ShouldBe(";");
694+
}
695+
696+
[Fact]
697+
public void A_destination_type_stored_in_a_different_case_keeps_its_properties()
698+
{
699+
var state = new BuilderState();
700+
BuilderConfigMapper.Hydrate(state, """
701+
{"name":"feed","source":{"type":"http"},
702+
"destinations":[{"type":"S3","properties":{"bucket":"reports","path":"a.csv"}}]}
703+
""").ShouldBeTrue();
704+
state.DestinationType = "s3";
705+
706+
using JsonDocument doc = JsonDocument.Parse(BuilderConfigMapper.ToConfigJson(state));
707+
708+
// Ordinal matching treated this as a type change and dropped the stored bucket.
709+
doc.RootElement.GetProperty("destinations").EnumerateArray().Single()
710+
.GetProperty(PropertiesMember).GetProperty("bucket").GetString().ShouldBe("reports");
711+
}
712+
678713
[Fact]
679714
public void Switching_the_source_drops_the_stored_properties_and_the_kept_connection()
680715
{

tests/NeoReports.UI.UnitTests/Pages/BuilderReviewTests.cs

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,23 @@ public void Editing_with_an_invalid_config_reports_the_engine_error_and_leaves_t
115115
cut.Markup.ShouldContain("Query references an unknown column.");
116116
}
117117

118+
[Fact]
119+
public void Editing_a_report_deleted_elsewhere_shows_the_engines_message_not_a_network_error()
120+
{
121+
Wizard.IsEditing = true;
122+
Wizard.EditingOriginalName = "clientsVip";
123+
Api.ReplaceReport = (_, _, _) => Task.FromResult(
124+
new ApiCreateResult(ApiCreateOutcome.Invalid, null, "No report named 'clientsVip' is registered."));
125+
126+
var cut = RenderReview();
127+
cut.FindAll("button").First(b => b.TextContent.Contains("Save report")).Click();
128+
129+
// A 404 is a rejected request carrying a usable message — mapping it to Unavailable threw the
130+
// message away and blamed the network for a report someone deleted in another tab.
131+
cut.Markup.ShouldContain("No report named 'clientsVip' is registered.");
132+
cut.Markup.ShouldNotContain("The engine is not reachable right now.");
133+
}
134+
118135
[Fact]
119136
public void Editing_saves_through_a_single_replace_call_and_never_deletes()
120137
{

0 commit comments

Comments
 (0)