[Span-First] Audit with OSS App

[buenaflor]

Instrument OSS Flutter App for Span-First Validation

Background

We originally planned to dog food the Sentry Dart Plugin internally, but there's a version mismatch between the Sentry library and the Dart plugin that makes them incompatible without a major bump to the plugin's minimum supported Dart version.

Instead we'll instrument an open-source Flutter app: openfoodfacts/smooth-app. This follows the same process we used for the replay audit.

---

Goals

• Validate SDK behavior against a realistic, production-like Flutter codebase
• Exercise auto-instrumentation and manual span APIs end-to-end
• Surface gaps or ergonomic issues

---

Tasks

Instrumentation

Use the Sentry Flutter branch: #3534
This is not merged yet but contains the final App Start instrumentation port which is important (otherwise there are no span-first app start spans)

Instrument the app as you realistically would in a production Flutter project. Use the internal Sentry SDKs project or a personal Sentry project and lean on auto-instrumentation wherever supported. For this app, that includes:

• TTID / TTFD
• App Starts (automatic)
• Hive
• HTTP

Testing

Verify the instrumentation works correctly on both iOS and Android.

Automation

Where possible, use Maestro to automate click-through flows so we can run multiple iterations efficiently.

• Automate all major flows that trigger as many spans as possible
• Android should be tested on a physical device in release mode
• iOS should be tested on simulator in debug mode (because Maestro only works in simulator for iOS and profile/release builds aren't supported on simulators)

---

Assertions to Validate / Tasks

Work through the following assertions and document findings:

• ignoreSpans and beforeSendSpan behave correctly
• No spans are unexpectedly dropped by ingest
• Span hierarchies are correct and reflect the expected parent-child relationships
• configureScope in startSpan callback delegates attributes to only children in that callback context
• Use startInactiveSpan and document where it is most useful
• The API is ergonomic — see open question below

---

APIs Under Test

API	Notes
Sentry.startSpan	Primary span API. Note: span ends automatically when the callback completes — there is no manual end(). See open question below.
Sentry.configureScope	You can call Sentry.configureScope inside a span's callback and it will delegate its attributes to all child spans inside that callback.
Sentry.currentHub.startInactiveSpan	Still internal but should be promoted to be a public API (Sentry.startInactiveSpan). Useful for recording spans across async or callback boundaries that are hard to model with startSpan.
options.ignoreSpans	Filter spans by name/pattern.
options.beforeSendSpan	Mutate before they're sent.

---

Open Questions

1. startSpan — manual ending

startSpan currently ends the span automatically when its callback completes. This works well for straightforward cases, but there are likely gaps where you want an active span that needs to outlive the callback (e.g. spans that start depends on some external completer to end).

Action: Note any cases where automatic span ending is insufficient. JavaScript addresses this with a separate Sentry.startSpanManually that behaves identically to startSpan but does not auto-close when the callback finishes. We should evaluate whether we need an equivalent.

2. startSpan API split

We're considering splitting the current startSpan into two functions to be more Dart-idiomatic:

• startSpan — callback-based, for Future-returning operations
• startSpanSync — synchronous variant, consistent with other Dart SDK patterns

Action: Note whether the current single API feels natural during instrumentation, or if the split would improve clarity.

E.g dart:io makes use of this

File.readAsString() / File.readAsStringSync()
File.writeAsBytes() / File.writeAsBytesSync()
Directory.list() / Directory.listSync()
Process.run() / Process.runSync()

Report

Please write the complete report in this issue

[linear]

DART-318

[denrase]

Validation Report

Instrumented openfoodfacts/smooth-app on branch denrase/smooth-app@sentry/smooth-app-span-first using SDK branch feat/span/native-app-start-v2.

Sentry Project: denrase / smooth-app-span-first

Platform-specific evidence (trace links, setup details):

• iOS Report — iPhone 16 Pro Simulator, iOS 18.5, debug mode
• Android Report — Pixel 4a, Android 13, debug mode

---

Instrumentation

Auto-instrumented: TTID/TTFD, App Starts, Hive (SentryHive), HTTP (SentryHttpClient).
Manual spans: product.scan, product.cache_lookup, product.fetch, product.load, product.search, product.search.decode, background_task.lifecycle, background_task.execute.

Key files:

• analytics_helper.dart — SDK init, beforeSendSpan, ignoreSpans
• continuous_scan_model.dart — product.scan, product.cache_lookup, product.fetch
• product_loader_page.dart — product.load
• search_products_manager.dart — product.search, product.search.decode
• background_task_manager.dart — background_task.lifecycle, background_task.execute
• local_database.dart — SentryHive.init()
• network_config.dart — SentryHttpClient

Automation

5 Maestro flows + a dedicated validation entrypoint for scan/background task paths. All 5 flows passed on both platforms. Search results confirmed via screenshots (141 products for "nutella").

Flow	What it exercises
01_app_launch	App Start, Hive openBox/openLazyBox, HTTP
02_search_product	product.search, product.search.decode, HTTP
03_product_details	product.load (deep link), HTTP, TTID/TTFD
04_preferences	Navigation, UI interaction
05_product_edit	product.load (deep link), TTID/TTFD

---

Assertions

These map to the assertions from the issue.

#	Assertion	Result	Evidence
1	ignoreSpans and beforeSendSpan behave correctly	✅	beforeSendSpan fired for all spans — 127 on iOS, 111 on Android. Configured in analytics_helper.dart.
2	No spans unexpectedly dropped by ingest	✅	All spans confirmed in Sentry — dashboard match on iOS, API query (dataset=spans) on Android.
3	Span hierarchies correct	✅	Parent-child nesting verified in trace view on both platforms. One expected exception: product.fetch orphaned due to fire-and-forget pattern — see API findings.
4	configureScope delegates attributes to children	✅	span.setAttribute() works correctly inside startSpan callbacks. Note: scope.setTag() is not part of the span-first API — only setAttribute is supported.
5	startInactiveSpan documented	✅	Two real-world use cases found — see API findings.
6	The API is ergonomic	⚠️	Mostly yes. Two issues: startInactiveSpan is internal (had to use Sentry.currentHub.startInactiveSpan), and SentryAttribute.string() is verbose. Details below.

---

Span Coverage

All auto-instrumented and manual span types verified on both platforms. Numbers differ due to Maestro behaviour (iOS: warm restarts reuse state; Android: cold starts from scratch).

Span type	iOS	Android	Notes
App Start	✅ 1 Cold + 4 Warm	✅ 5 Cold	Sub-spans differ per platform (see iOS, Android)
Hive DB (openBox/openLazyBox)	✅ 9 per launch	✅ 9 per launch
product.search	✅	✅
product.search.decode	✅	✅
product.load	✅ (×2)	✅ (×2)	Via deep link (flows 03, 05)
HTTP	✅	✅	SVG downloads + API calls
TTID/TTFD	✅	✅	root /, _product_loader/:productId
beforeSendSpan	✅ all 127 spans	✅ all 111 spans
product.scan / .cache_lookup / .fetch	N/A	N/A	Camera-only, not testable via Maestro
background_task.lifecycle / .execute	N/A	N/A	Requires OpenFoodFacts login

---

APIs Under Test

API	Status	Notes
Sentry.startSpan	✅ works well	Callback pattern with auto-ending and implicit nesting via zones. Used for product.search, product.load, product.scan. One limitation: doesn't work with fire-and-forget — see below.
Sentry.configureScope	✅ works	span.setAttribute() correctly delegates to children. scope.setTag() is not part of span-first API.
Sentry.currentHub.startInactiveSpan	✅ works, needs promotion	Still internal. Two real-world use cases found. Should be promoted to Sentry.startInactiveSpan.
options.ignoreSpans	✅ works	Configured in analytics_helper.dart.
options.beforeSendSpan	✅ works	Fired for every span on both platforms.

Additional APIs exercised:

• SentryHive — drop-in replacement, worked perfectly
• SentryHttpClient — auto-instrumented all HTTP calls
• SentrySpanStatusV2 — error, deadlineExceeded, cancelled all worked

---

API Findings

startSpan callback doesn't work with fire-and-forget

The app intentionally fire-and-forgets async work to keep the UI responsive (continuous_scan_model.dart):

Sentry.startSpan('product.scan', (span) async {
  _cacheOrLoadBarcode(barcode); // NOT awaited — intentional
  return true;
  // ← callback returns, span auto-ends before child work completes
});
// product.cache_lookup → sometimes nested (timing), product.fetch → orphaned

This is a common pattern in UI apps. startSpan's auto-ending callback can't handle it — child spans get orphaned. This is a second real-world case for startInactiveSpan (alongside the BackgroundTaskManager pattern where a span outlives its creation context).

SentryAttribute.string() is verbose

Used 10× across 4 files. A common operation shouldn't require this much ceremony.

---

Open Questions

1. startSpan — manual ending

Note any cases where automatic span ending is insufficient.

Two cases needed manual span lifetime control:

• Fire-and-forget (product.scan): callback returns before child async work completes → children orphaned.
• Spans outliving creation context (background_task.lifecycle): span starts in one method, ends in a later callback.

Both were solved with startInactiveSpan.

However, per the Span API spec, the base startSpan MUST return a span and MUST NOT auto-end — that's what the Dart SDK currently calls startInactiveSpan. The callback variant that auto-ends is an optional convenience API. The Dart SDK has these inverted: the convenience API got the startSpan name, and the spec-mandated base API is internal as startInactiveSpan.

The naming also conflates two concepts: the spec's active option controls scope parenting (whether new spans become children), not auto-ending. startInactiveSpan sounds like the span isn't started yet.

Recommendation: promote the base API to Sentry.startSpan (matching the spec) and give the callback variant a distinct name (e.g. Sentry.startSpanWithCallback), or align with the spec's single startSpan + active option.

2. startSpan API split (startSpan / startSpanSync)

Note whether the current single API feels natural, or if the split would improve clarity.

Not needed. The current startSpan uses FutureOr<T> so it already handles both sync and async callbacks in one API — sync callbacks return directly, async callbacks return a Future.

[denrase]

Android — Trace Links & Observations

Platform-specific evidence for the Validation Report.

Tested on: Google Pixel 4a, Android 13 (API 33), debug mode.
Entrypoint: main_fdroid.dart (ZXing scanner, scanner_ml_kit disabled due to Xcode 26 arm64 issue)
Flows: 5/5 passed · 111 spans · 7 traces

---

Maestro Setup

Running Maestro on a physical Android device required a workaround for driver APK installation (automated in run_validation_android.sh):

1. Driver APK installation: Maestro 2.2.0 uses dadb (Java ADB library) to install its driver APKs, which fails on USB-connected physical devices. The driver APKs (maestro-server.apk, maestro-app.apk) were extracted from the Maestro JAR and installed manually via adb install. Alternatively, switching the device to adb tcpip 5555 and connecting via Wi-Fi resolves this, but requires the Mac and device to be on the same reachable network.
2. App link approval: openLink deep links open in Chrome by default on Android (debug builds can't verify domain ownership). Fixed by running adb shell pm set-app-links --package org.openfoodfacts.scanner 2 all after install to force the app as the default handler.
3. Maestro sleep syntax: Maestro 2.2.0 removed the sleep() function from evalScript. Replaced with runScript using a JS file with a Date.now() busy-wait loop.

---

Trace Links

Trace	Spans	Description
be27b023…	16	Flow 01 — Cold Start, Hive openBox/openLazyBox × 9
82862726…	19	Flow 02 — product.search + product.search.decode, Cold Start, Hive, POST search.pl
49d36d87…	12	Flow 03 — product.load, HTTP × 6 (API, folksonomy, robotoff, prices), TTID/TTFD
2f9975e8…	16	Flow 05 — product.load, HTTP × 10 (API, SVG, folksonomy, robotoff, prices), TTID/TTFD
641eea67…	16	Flow 04 — Cold Start, Hive

---

Android-Specific Observations

• No behavioral differences from iOS. All span types behave identically — same hierarchy, same attributes, same status codes.
• App start sub-spans are different. Android shows Process Initialization and App start to plugin registration; iOS shows Pre Runtime Init, UIKit init, and Runtime init to Pre Main initializers.
• Maestro launchApp behaviour: On Android, each launchApp is a Cold Start (app fully terminated and restarted). All 5 launches were Cold Starts, compared to iOS (1 Cold + 4 Warm).
• Search results confirmed via screenshot: 141 products found for "nutella" — identical results to iOS, with product cards, images, and score badges visible in 02_04_search_results.png.
• Product details confirmed via screenshot: Nutella 400g product page loaded via deep link, matching iOS exactly.

[denrase]

iOS — Trace Links & Observations

Platform-specific evidence for the Validation Report.

Tested on: iOS Simulator, iPhone 16 Pro, iOS 18.5, debug mode.
Entrypoint: main_ios.dart
Flows: 5/5 passed · 127 spans · 7 traces

---

Trace Links

Trace	Spans	Description
d2379c20…	20	Flow 01 — Cold Start, Hive openBox/openLazyBox × 9, HTTP (GitHub assets)
69e282ed…	21	Flow 02 — product.search + product.search.decode, Warm Start, Hive, POST search.pl
4f2255a3…	16	Flow 03 — product.load, HTTP × 10 (API, SVG, robotoff, folksonomy, prices), TTID/TTFD
ba74d21d…	12	Flow 05 — product.load, HTTP × 8, TTID/TTFD
9d2325c5…	20	Flow 04 — Warm Start, Hive, SVG downloads

---

iOS-Specific Observations

• App start sub-spans include iOS-specific phases: Pre Runtime Init, Runtime init to Pre Main initializers, UIKit init, App start to plugin registration, Before Sentry Init Setup, First frame render.
• Maestro launchApp behaviour: On iOS, launchApp terminates and re-creates the app (Warm Start). 1 Cold Start (flow 01 with clearState) + 4 Warm Starts.
• Search results confirmed via screenshot: 141 products found for "nutella" — product cards with images, Nutri-Score badges, and green scores visible in 02_04_search_results.png.
• Product details confirmed via screenshot: Nutella 400g product page loaded via deep link with Nutri-Score E, compatibility score, and edit button visible.

[denrase]

@buenaflor Created a fork of the app and also a fort of the dependencies repo to inject our sentry http client. See above for results (summary written with AP from the automated runs, hope I have captured everything we need).