feat(api): opt-in profiling for get_transaction recent-first by igorls · Pull Request #181 · eosrio/hyperion-history-api

igorls · 2026-06-05T09:58:14Z

Summary

Adds an opt-in diagnostic, api.hot_first_transaction_profiling (default false), that instruments the get_transaction recent-first path (PR #180) so operators can size hot_first_window from data instead of guessing.

A get_transaction issued without a block_hint has no block range to prune on, so it fans out across every action partition. The recent-first option (hot_first_transaction) probes the newest hot_first_window partitions first and widens to the full set only on a miss. Whether a given window captures most lookups depends on how far back the requested transactions actually are — which this profiling measures.

Behaviour

When enabled, logs one line per get_transaction served without a block_hint:

[gtx-profile] trx=<id12> served=hot|full hot_hits=N found_block=B found_part=P head_part=H parts_back=K phase1_ms=.. widen_ms=.. total_ms=..

served — hot (answered from the hot window) or full (widened, or recent-first disabled).
parts_back — partitions older than head the transaction was found in. A hot_first_window of parts_back + 1 would have served it from the hot path. -1 = not found.
phase1_ms / widen_ms / total_ms — per-phase Elasticsearch timing.

Aggregating parts_back over a sample shows the block-age distribution of lookups, indicating whether increasing hot_first_window would help or whether the requests are spread across history.

Notes

No behaviour change when disabled. When enabled it adds only a few Date.now() calls and one log line per request, but it logs one line per request — intended to be enabled briefly to sample, then disabled.
tsc --noEmit clean; config-validation unit test passes.

Usage

"hot_first_transaction": true,
"hot_first_transaction_profiling": true

After sampling:

grep gtx-profile <log> | grep -oE 'parts_back=[0-9-]+' | sort | uniq -c | sort -rn

Adds api.hot_first_transaction_profiling (default off). When enabled, logs one line per get_transaction served WITHOUT a block_hint: [gtx-profile] trx=<id12> served=hot|full hot_hits=N found_block=B found_part=P head_part=H parts_back=K phase1_ms=.. widen_ms=.. total_ms=.. `parts_back` = how many partitions older than head the trx actually was; a hot_first_window of (parts_back + 1) would have served it from the hot path. Aggregating parts_back across a sample reveals the block-age distribution of real lookups — i.e. whether a larger window would capture them, or whether the traffic is spread across history (in which case no window helps and the fix is block_hint at the client / bounded fan-out). Noisy under load; enable briefly. No behaviour change when disabled.

gemini-code-assist

Code Review

This pull request introduces an opt-in diagnostic profiling feature (hot_first_transaction_profiling) for the get_transaction endpoint when queried without a block hint. It tracks execution times for both the initial hot-window search and the widened search, logging details such as partition age and latency. The review feedback suggests two robust improvements: safely handling the head_block_num type which can vary between a plain number and a Wharfkit UInt32 object, and guarding the partition distance calculation with Math.max(0, ...) to prevent replication lag from producing negative values that could be misinterpreted as a "not found" status.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-06-05T09:59:43Z

+                const bn = h?._source?.block_num ?? 0;
+                if (bn > foundBlock) foundBlock = bn;
+            }
+            const headBlock = Number(pResults[0]?.head_block_num ?? 0);


When $getInfo is retrieved from the Redis cache, it is parsed via JSON.parse, meaning head_block_num will be a plain number. However, when it is fetched fresh from the Antelope chain API, it is a Wharfkit UInt32 instance.

Using Number() directly on a Wharfkit UInt32 object relies on implicit object-to-primitive coercion (which calls toString()). It is safer and more robust to explicitly check for the .value property (which holds the primitive number on Wharfkit numeric types) before falling back to the raw value.

Suggested change

const headBlock = Number(pResults[0]?.head_block_num ?? 0);

const headBlock = pResults[0] ? Number(pResults[0].head_block_num?.value ?? pResults[0].head_block_num) : 0;

gemini-code-assist · 2026-06-05T09:59:43Z

+            const headBlock = Number(pResults[0]?.head_block_num ?? 0);
+            const foundPart = foundBlock ? Math.ceil(foundBlock / partSize) : 0;
+            const headPart = headBlock ? Math.ceil(headBlock / partSize) : 0;
+            const partsBack = (foundPart && headPart) ? (headPart - foundPart) : -1;


If there is micro-lag between Elasticsearch and the get_info node, it is possible for foundPart to be greater than headPart (i.e., a transaction is found in a block newer than the reported head block).

In this case, headPart - foundPart will yield a negative number. If it yields -1, it will be incorrectly interpreted as "not found" in your log parsing, skewing the profiling metrics. Guarding this with Math.max(0, ...) ensures that any such lag is safely treated as 0 partitions back (meaning it is in the hot window).

Suggested change

const partsBack = (foundPart && headPart) ? (headPart - foundPart) : -1;

const partsBack = (foundPart && headPart) ? Math.max(0, headPart - foundPart) : -1;

Copilot

Pull request overview

Adds an opt-in diagnostic to profile get_transaction requests that arrive without a block_hint, so operators can measure how far back in history lookups typically are and whether increasing the hot-window would materially reduce cold-tier fan-out.

Changes:

Adds api.hot_first_transaction_profiling config flag (default false) and schema validation.
When enabled, logs one [gtx-profile] line per uncached get_transaction request without block_hint, including hot-vs-widened path, per-phase timings, and parts_back.
Updates the reference config JSON with the new flag.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
src/interfaces/hyperionConfig.ts	Adds the new API config option and validates it in the Zod schema.
src/api/routes/v2-history/get_transaction/get_transaction.ts	Implements the opt-in profiling log line and computes diagnostic fields/timings.
references/config.ref.json	Documents the new config key with a default value.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

        // $getInfo resolves null on failure — don't turn a recoverable lib-lookup miss into a 500.
        response.lib = pResults[0]?.last_irreversible_block_num;

+        const phase1Ms = profile ? Date.now() - tStart : 0;


+            const tWiden = profile ? Date.now() : 0;
+            const widenedRes = await runSearch(fullPattern);
+            hits = widenedRes.hits.hits;
+            widenMs = profile ? Date.now() - tWiden : 0;
+            widened = true;


+        if (profile) {
+            // parts_back = how many partitions older than head the trx is; a hot_first_window of
+            // (parts_back + 1) would have served this lookup from the hot path. -1 = not found.
+            const partSize = conf.settings.index_partition_size;
+            let foundBlock = 0;
+            for (const h of hits) {
+                const bn = h?._source?.block_num ?? 0;
+                if (bn > foundBlock) foundBlock = bn;
+            }
+            const headBlock = Number(pResults[0]?.head_block_num ?? 0);
+            const foundPart = foundBlock ? Math.ceil(foundBlock / partSize) : 0;
+            const headPart = headBlock ? Math.ceil(headBlock / partSize) : 0;
+            const partsBack = (foundPart && headPart) ? (headPart - foundPart) : -1;


+    // Diagnostic: when true, log one line per get_transaction served WITHOUT a block_hint —
+    // which phase served it (hot vs widened), per-phase timings, and how many partitions back the
+    // trx actually was (`parts_back`). Aggregating parts_back reveals the block-age distribution of
+    // lookups, i.e. the hot_first_window that would capture a given fraction of them. Default off;
+    // noisy under load (one line per request) — enable briefly to sample, then disable.
+    hot_first_transaction_profiling?: boolean;


- Unwrap head_block_num .value before coercion: it is a Wharfkit UInt32 when fresh from get_info but a plain number when read from the redis cache. - Clamp parts_back with Math.max(0, ...) so get_info/ES micro-lag (a hit one block past the reported head) can't produce a negative that collides with the -1 "not found" sentinel. - Keep source comments to the mechanism only (drop deployment-specific notes).

igorls · 2026-06-05T10:04:05Z

Thanks @gemini-code-assist — both addressed in 0fa365b:

head_block_num type — now unwraps .value before coercion (Number(headRaw?.value ?? headRaw ?? 0)), so it works whether it arrives as a Wharfkit UInt32 (fresh from get_info) or a plain number (from the redis cache).
Negative parts_back — clamped with Math.max(0, ...) so get_info/ES micro-lag can't produce a negative that collides with the -1 "not found" sentinel.

tsc clean; config-validation test passes.

gemini-code-assist · 2026-06-05T10:04:15Z

Thanks for the update, @igorls. The adjustments to head_block_num coercion and the Math.max clamping for parts_back look solid and should correctly handle the edge cases described. The implementation looks good to go.

Copilot AI review requested due to automatic review settings June 5, 2026 09:58

Copilot started reviewing on behalf of igorls June 5, 2026 09:58 View session

gemini-code-assist Bot reviewed Jun 5, 2026

View reviewed changes

Copilot AI reviewed Jun 5, 2026

View reviewed changes

igorls merged commit 57ce004 into dev Jun 5, 2026
2 checks passed

igorls deleted the feat/get-transaction-profiling branch June 5, 2026 10:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(api): opt-in profiling for get_transaction recent-first - #181

feat(api): opt-in profiling for get_transaction recent-first#181
igorls merged 2 commits into
devfrom
feat/get-transaction-profiling

igorls commented Jun 5, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Jun 5, 2026

Uh oh!

gemini-code-assist Bot Jun 5, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

igorls commented Jun 5, 2026

Uh oh!

gemini-code-assist Bot commented Jun 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	const headBlock = Number(pResults[0]?.head_block_num ?? 0);
	const headBlock = pResults[0] ? Number(pResults[0].head_block_num?.value ?? pResults[0].head_block_num) : 0;

	const partsBack = (foundPart && headPart) ? (headPart - foundPart) : -1;
	const partsBack = (foundPart && headPart) ? Math.max(0, headPart - foundPart) : -1;

Uh oh!

Conversation

igorls commented Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Behaviour

Notes

Usage

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Jun 5, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

igorls commented Jun 5, 2026

Uh oh!

gemini-code-assist Bot commented Jun 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

igorls commented Jun 5, 2026 •

edited

Loading