Skip to content

Commit 82da871

Browse files
thiagolugathiagoluga
committed
test(release): verify the published packages from a consumer's position (ADR D85)
The ask was a sample that installs the Pro packages instead of referencing the projects. Pointing samples/15 at nuget.org was investigated and rejected: the samples are the repo's compile-time canary. Break a public API in Core and samples 14 and 15 stop building immediately; on a pinned PackageReference they would keep building against the last release and the break would ship. It could not have been done by halves either. samples/15 reaches the NeoReports projects through AllSourcesShared, which sample 14 also uses, so converting only the Pro references would put a package Core and a project Core in one build. tools/consumer-smoke instead: outside NeoReports.sln, with a Directory.Build.props that does not import the repo's and an empty Directory.Packages.props beside it to stop the walk-up to Central Package Management. Everything resolves from nuget.org at the versions a customer would type. It proves strictly more than a converted sample would, because it exercises the artifact on the feed rather than a local build of it. Three levels: identity (versions, the 2.0.0+<commit> stamp, the embedded key is the production one); enforcement (all three Pro packages refuse to work unlicensed through both the static API and DI); and, with NEOREPORTS_LICENSE_KEY set, a real sectioned-workbook run whose .xlsx is opened and checked for two worksheets. Writing it surfaced three consumer-facing facts about 2.0.0 that no in-repo build could have: * Core requires Microsoft.Extensions.* 10.0.10 — a consumer on 9.x hits NU1605. * Core resolves ILoggerFactory from DI but only depends on Logging.Abstractions, so the consumer must bring a logging implementation. * PowerShell's `>` writes UTF-16, so a key captured that way arrives with NULs and is reported as "malformed" — wrong encoding wearing a bad-signature mask. Two things the harness got wrong first, both recorded rather than quietly fixed. An assertion that the assemblies "came from the package cache" can never pass: the SDK copies package assemblies into bin/, so Assembly.Location is identical either way. That guarantee is structural, and a single added ProjectReference would defeat the harness with every check still green. And the enforcement checks failed whenever a license WAS available, because ProLicenseGate falls back to the environment variable by design — a broken test, not a broken product. The variable is now cleared around those checks and restored after, so they stay meaningful in the mode where a regression would cost the most. Verified locally both ways: 10/10 without a license, 15/15 with one.
1 parent 22256ab commit 82da871

8 files changed

Lines changed: 491 additions & 0 deletions

File tree

‎DECISIONS.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1700,3 +1700,59 @@ A `--public-key` **flag** was considered for this and rejected: it would let the
17001700
run against the key it was just handed, which checks nothing, and the footgun would sit on the command
17011701
whose entire purpose is catching that class of mistake. An internal seam has the same testing benefit
17021702
with none of that surface.
1703+
1704+
## D85 — Consumer smoke test, and why the samples were not converted (2026-08-08)
1705+
1706+
With the Pro packages published (D83), the maintainer asked for a sample that *installs* them rather
1707+
than referencing the projects. The obvious move — point `samples/15-aspire-pro-demo` at the published
1708+
packages — was investigated and rejected.
1709+
1710+
**The samples are the repo's compile-time canary.** Break a public API in `Core` and samples 14 and 15
1711+
stop building immediately, before anything is tagged. On a pinned `PackageReference` they would keep
1712+
building happily against the last release and the break would ship. That canary is worth more than a
1713+
demonstration of how a customer installs, because the second goal has a cheaper answer that proves
1714+
strictly more: exercise the artifact **on nuget.org**, not a local build of it.
1715+
1716+
It also could not have been done by halves. `samples/15` reaches the NeoReports projects through
1717+
`AllSourcesShared`, which sample 14 uses too, so converting only the Pro references would have put a
1718+
*package* `NeoReports.Core` and a *project* `NeoReports.Core` in one build (NU1605). Converting the
1719+
shared project would have dragged sample 14 along — a sample nobody asked to change.
1720+
1721+
### What was built instead
1722+
1723+
`tools/consumer-smoke`: outside `NeoReports.sln`, with a `Directory.Build.props` that does not import
1724+
the repo's and an empty `Directory.Packages.props` beside it to stop the walk-up to Central Package
1725+
Management. Everything resolves from nuget.org at the versions a customer would type.
1726+
1727+
Three levels: identity (versions, `2.0.0+<commit>` informational version, the embedded key is the
1728+
production one), **enforcement** (all three Pro packages refuse to work unlicensed, via both the static
1729+
API and DI), and — only when `NEOREPORTS_LICENSE_KEY` is set — a real sectioned-workbook report whose
1730+
output `.xlsx` is opened and inspected for two worksheets.
1731+
1732+
### What it caught immediately
1733+
1734+
Writing it surfaced three things about consuming 2.0.0 that no in-repo build could have:
1735+
1736+
1. `NeoReports.Core 2.0.0` requires `Microsoft.Extensions.*` **10.0.10**; a consumer still on 9.x gets
1737+
a hard `NU1605`.
1738+
2. `Core` resolves `ILoggerFactory` from DI but only depends on `Logging.Abstractions`, so the consumer
1739+
must bring a logging implementation or the provider throws at `GetRequiredService`.
1740+
3. PowerShell's `>` redirect writes UTF-16, so a license key captured that way arrives with embedded
1741+
NULs and is reported as *malformed* — a wrong-encoding file masquerading as a bad signature.
1742+
1743+
### An assertion that could not exist
1744+
1745+
The harness first tried to prove "these came from packages, not projects" by inspecting
1746+
`Assembly.Location`. That check can never pass: the SDK copies package assemblies into `bin/`, so the
1747+
path is identical either way. **The package-not-project guarantee is structural, not testable** — a
1748+
single added `ProjectReference` would defeat the harness with every check still green. Recorded in the
1749+
`.csproj` and README as the actual guard, rather than papered over with an assertion that looks like
1750+
protection and is not.
1751+
1752+
### A broken test, not a broken product
1753+
1754+
The enforcement checks initially failed whenever a license *was* available, because `ProLicenseGate`
1755+
falls back to `NEOREPORTS_LICENSE_KEY` by design: with a key exported, the Pro calls correctly
1756+
succeed. "No license registered" is not the state "no license available". The variable is now cleared
1757+
around those checks and restored after, which keeps them meaningful in both modes instead of skipping
1758+
them in the mode where a regression would be most expensive.

‎tools/consumer-smoke/Checks.cs‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
namespace NeoReports.ConsumerSmoke;
2+
3+
/// <summary>
4+
/// A deliberately tiny assertion recorder. This project cannot use the repo's test stack (xUnit,
5+
/// Shouldly) without a <c>ProjectReference</c> or the repo's Central Package Management, and pulling
6+
/// either in would reconnect it to the working tree it exists to stay away from.
7+
/// </summary>
8+
/// <remarks>
9+
/// Collects failures instead of throwing on the first one: when a release is broken it is far more
10+
/// useful to see every check that failed in a single run than to fix and re-run six times.
11+
/// </remarks>
12+
internal sealed class Checks
13+
{
14+
private readonly List<string> _failures = [];
15+
private int _passed;
16+
17+
/// <summary>Records a condition. <paramref name="detail"/> is appended to the failure line.</summary>
18+
public void That(bool condition, string description, string? detail = null)
19+
{
20+
if (condition)
21+
{
22+
_passed++;
23+
Console.WriteLine($" ok {description}");
24+
return;
25+
}
26+
27+
string line = detail is null ? description : $"{description} — {detail}";
28+
_failures.Add(line);
29+
Console.WriteLine($" FAIL {line}");
30+
}
31+
32+
/// <summary>Records that <paramref name="action"/> throws <typeparamref name="TException"/>.</summary>
33+
/// <remarks>
34+
/// A wrong exception type is reported distinctly from no exception at all: the first usually means
35+
/// the guard moved, the second means it is gone. They call for different fixes.
36+
/// </remarks>
37+
public void Throws<TException>(Action action, string description)
38+
where TException : Exception
39+
{
40+
try
41+
{
42+
action();
43+
That(false, description, $"nothing was thrown — expected {typeof(TException).Name}");
44+
}
45+
catch (TException)
46+
{
47+
That(true, description);
48+
}
49+
catch (Exception ex)
50+
{
51+
That(false, description, $"threw {ex.GetType().Name} instead of {typeof(TException).Name}");
52+
}
53+
}
54+
55+
/// <summary>Prints the summary and returns the process exit code (0 = every check passed).</summary>
56+
public int Report()
57+
{
58+
Console.WriteLine();
59+
if (_failures.Count == 0)
60+
{
61+
Console.WriteLine($"All {_passed} checks passed against the published packages.");
62+
return 0;
63+
}
64+
65+
Console.WriteLine($"{_failures.Count} of {_passed + _failures.Count} checks FAILED:");
66+
foreach (string failure in _failures)
67+
Console.WriteLine($" - {failure}");
68+
69+
return 1;
70+
}
71+
}

‎tools/consumer-smoke/Directory.Build.props‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
<Project>
2+
3+
<!-- Deliberately does NOT import the repo-root props, and there is no Directory.Packages.props
4+
beside it either. MSBuild stops walking up the moment it finds these files, so this project is
5+
cut off from Central Package Management and from every repo-wide setting.
6+
That isolation is the point: this project must resolve NeoReports from nuget.org exactly as a
7+
customer would, with no ProjectReference, no local artifact directory and no version pinned by
8+
the repo. Inheriting the repo's props would quietly reintroduce the coupling it exists to avoid. -->
9+
10+
<PropertyGroup>
11+
<ManagePackageVersionsCentrally>false</ManagePackageVersionsCentrally>
12+
</PropertyGroup>
13+
14+
</Project>

‎tools/consumer-smoke/Directory.Packages.props‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
<Project>
2+
<!-- Empty on purpose: stops the walk-up to build/Directory.Packages.props, so the versions below
3+
are the ones a customer would type, not the ones this repo happens to build with. -->
4+
</Project>

‎tools/consumer-smoke/InMemorySales.cs‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
using NeoReports.Abstractions;
2+
3+
namespace NeoReports.ConsumerSmoke;
4+
5+
/// <summary>A row of the smoke report.</summary>
6+
internal sealed record Sale(long Id, string Customer, decimal Amount);
7+
8+
/// <summary>
9+
/// A source built here rather than taken from a source package, on purpose: implementing
10+
/// <see cref="IBatchSource{T}"/> against the published <c>NeoReports.Abstractions</c> is itself part
11+
/// of what this harness verifies. If the ABI shipped in a state a consumer cannot implement against,
12+
/// this file stops compiling — which is the earliest and loudest possible signal.
13+
/// </summary>
14+
/// <remarks>
15+
/// Every other row is given a non-positive amount so the two worksheets the report sections into are
16+
/// both non-empty. A section that silently produced zero rows would still yield a valid workbook, so
17+
/// the check downstream would pass for the wrong reason.
18+
/// </remarks>
19+
internal sealed class InMemorySales(int total) : IBatchSource<Sale>
20+
{
21+
private const int PageSize = 7; // not a divisor of any sensible total, so the last page is partial
22+
23+
/// <summary>
24+
/// The report's own <c>.Column(...)</c> declarations drive the output, so this source declares an
25+
/// empty schema rather than duplicating them — a source is only obliged to describe what it can.
26+
/// </summary>
27+
public ReportSchema Schema { get; } = new(Array.Empty<ReportColumn>());
28+
29+
public Task<BatchResult<Sale>> ReadBatchAsync(BatchContext context, CancellationToken cancellationToken)
30+
{
31+
ArgumentNullException.ThrowIfNull(context);
32+
33+
int offset = context.Cursor is null ? 0 : int.Parse(context.Cursor, System.Globalization.CultureInfo.InvariantCulture);
34+
var page = new List<Sale>(PageSize);
35+
36+
for (int i = offset; i < offset + PageSize && i < total; i++)
37+
{
38+
decimal amount = i % 2 == 0 ? (i + 1) * 10.5m : -(i + 1);
39+
page.Add(new Sale(i + 1, $"Customer {i + 1}", amount));
40+
}
41+
42+
int next = offset + page.Count;
43+
return Task.FromResult(new BatchResult<Sale>(
44+
page,
45+
nextCursor: next < total ? next.ToString(System.Globalization.CultureInfo.InvariantCulture) : null,
46+
hasMore: next < total));
47+
}
48+
}

‎tools/consumer-smoke/NeoReports.ConsumerSmoke.csproj‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
<Project Sdk="Microsoft.NET.Sdk">
2+
3+
<PropertyGroup>
4+
<OutputType>Exe</OutputType>
5+
<TargetFramework>net8.0</TargetFramework>
6+
<Nullable>enable</Nullable>
7+
<ImplicitUsings>enable</ImplicitUsings>
8+
<LangVersion>latest</LangVersion>
9+
<RootNamespace>NeoReports.ConsumerSmoke</RootNamespace>
10+
<!-- Not part of NeoReports.sln and never packed: this is a verification harness, not a product. -->
11+
<IsPackable>false</IsPackable>
12+
</PropertyGroup>
13+
14+
<!--
15+
PackageReference, never ProjectReference. The whole value of this project is that it resolves the
16+
published artifacts from nuget.org — the same bytes a customer downloads — rather than whatever
17+
happens to be in the working tree. A single ProjectReference here would silently turn it back into
18+
a build of local source and it would keep passing while the published package was broken.
19+
20+
The version is pinned rather than floating so a run states exactly which release it verified.
21+
Bump it when a new version ships; a mismatch between this and the latest tag is the signal that a
22+
release has not been smoke-tested yet.
23+
-->
24+
<ItemGroup>
25+
<PackageReference Include="NeoReports.Core" Version="2.0.0" />
26+
<PackageReference Include="NeoReports.Destinations.Local" Version="2.0.0" />
27+
<PackageReference Include="NeoReports.Licensing" Version="2.0.0" />
28+
<!-- The commercial packages under test (PolyForm Small Business). -->
29+
<PackageReference Include="NeoReports.Xlsx.Pro" Version="2.0.0" />
30+
<PackageReference Include="NeoReports.Sources.Join.Pro" Version="2.0.0" />
31+
<PackageReference Include="NeoReports.QueryBuilder.Pro" Version="2.0.0" />
32+
</ItemGroup>
33+
34+
<!--
35+
Only the DI container implementation, which the published packages depend on the *Abstractions* of
36+
but deliberately do not carry themselves — the host chooses it. Pinned to 10.0.10 to match what
37+
NeoReports.Core 2.0.0 requires: an earlier 9.x here is a hard NU1605 downgrade error, which is
38+
worth knowing, because it is the first thing a consumer still on Microsoft.Extensions 9.x will hit.
39+
-->
40+
<ItemGroup>
41+
<PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="10.0.10" />
42+
<!-- NeoReports.Core resolves ILoggerFactory from DI but only depends on Logging.Abstractions,
43+
so a consumer has to bring a logging implementation. Finding that out here is the point. -->
44+
<PackageReference Include="Microsoft.Extensions.Logging" Version="10.0.10" />
45+
</ItemGroup>
46+
47+
</Project>

0 commit comments

Comments
 (0)