Skip to content

[receiver/oracledb] Add transaction, lock, and recovery metrics - #49067

Merged
dmitryax merged 14 commits into
open-telemetry:mainfrom
rluidash:feat/oracledb-txn-lock-recovery
Jul 15, 2026
Merged

[receiver/oracledb] Add transaction, lock, and recovery metrics#49067
dmitryax merged 14 commits into
open-telemetry:mainfrom
rluidash:feat/oracledb-txn-lock-recovery

Conversation

@rluidash

@rluidash rluidash commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Description

The Oracle DB receiver scrapes v$sysstat with SELECT * FROM v$sysstat, which returns all system-statistics rows. However, the scraper only processes a subset of those rows in its switch block and silently discards the rest — including the transaction-rollback, lock-contention, and recovery statistics. This change surfaces 7 discarded v$sysstat statistics as 5 new opt-in metrics, giving operators visibility into commit/rollback churn, enqueue/lock pressure, instance- and transaction-recovery activity, and RAC cache-fusion latency — core Oracle durability and availability diagnostics.

This change adds 5 new opt-in metrics (disabled by default, stability: development):

  • oracledb.transaction.rollbacks — Number of transactions rolled back. No attributes. Covers v$sysstat: transaction rollbacks. Distinct from the existing oracledb.user_rollbacks (which counts only user-issued ROLLBACK statements); this counts all transaction rollbacks, including internal/recursive ones.
  • oracledb.lock.time — Cumulative time, in seconds, spent on transaction lock activity. Attribute: oracledb.session.type (background|foreground). Covers v$sysstat: transaction lock background get time, transaction lock foreground wait time (the raw centisecond value is divided by 100 in the scraper).
  • oracledb.recovery.blocks — Number of blocks read during instance or media recovery. No attributes. Covers v$sysstat: recovery blocks read.
  • oracledb.smon.posts — Number of times SMON was posted to perform recovery. Attribute: oracledb.smon.type (instance|transaction). Covers v$sysstat: SMON posted for instance recovery (instance), SMON posted for txn recovery for other instances (transaction, RAC cross-instance).
  • oracledb.gc.current_block.time — Cumulative time, in seconds, spent transferring current blocks between instances over Oracle RAC cache fusion. Attribute: network.io.direction (receive). Covers v$sysstat: gc current block receive time (cs ÷ 100). Here "gc" denotes Oracle's global cache (Cache Fusion), not JVM garbage collection; the value is 0 on a single-instance (non-RAC) database.

oracledb.lock.time and oracledb.gc.current_block.time are emitted as a Sum with aggregation_temporality: cumulative, monotonic: true, value type double, unit s. The remaining three are emitted as Sum with aggregation_temporality: cumulative, monotonic: true, value type int. Per-metric units: s (oracledb.lock.time, oracledb.gc.current_block.time); {rollback} (oracledb.transaction.rollbacks); {block} (oracledb.recovery.blocks); {post} (oracledb.smon.posts).

The two time metrics report seconds (s) rather than the raw cs (centisecond) units that Oracle exposes via v$sysstat; the scraper divides the raw value by 100 before recording, matching the conversion already in place for the existing oracledb.cpu_time metric.

Note (change since the initial submission): an earlier revision included oracledb.lock.wait.time mapped to a v$sysstat statistic named Total Lock Time. That statistic does not exist in v$sysstat on any supported Oracle version (verified against v$statname on Oracle 19c — it returns no such row), so the metric would never emit. It has been removed. Background/foreground lock timing is already captured by oracledb.lock.time.

These metrics can be enabled in the collector configuration:

receivers:
  oracledb:
    datasource: "oracle://user:password@host:1521/service"
    metrics:
      oracledb.transaction.rollbacks:
        enabled: true
      oracledb.lock.time:
        enabled: true
      oracledb.recovery.blocks:
        enabled: true
      oracledb.smon.posts:
        enabled: true
      oracledb.gc.current_block.time:
        enabled: true

Link to tracking issue

Fixes #49064

Testing

Unit tests added in scraper_test.go:

  • TestScraper_ScrapeTransactionLockRecoveryMetrics exercises all 5 new metrics end-to-end through the scraper using the existing fake DB client, asserting one expected value per data point (2 oracledb.lock.time data points — one per oracledb.session.type — plus 2 oracledb.smon.posts data points — one per oracledb.smon.type — plus 3 standalone data points = 7 data points per scrape across the 5 metrics).
  • The shared queryResponses[statsSQL] fixture is extended with 7 new fake v$sysstat rows (one per covered NAME), so TestScraper_Scrape, TestScraper_ScrapeOperationalMetrics, and TestScraper_ScrapeIOPerformanceMetrics continue to pass unchanged.
  • The test explicitly verifies the centiseconds → seconds conversion: raw 350/1200 cs from transaction lock background get time / transaction lock foreground wait time produce 3.5/12.0 s on the background/foreground data points; 640 cs from gc current block receive time produces 6.4 s.
  • Auto-generated tests in internal/metadata/generated_metrics_test.go and generated_config_test.go are regenerated by make generate and cover the new metric configs / metric builders.

Documentation

Auto-generated documentation.md updated with descriptions and metadata for the 5 new metrics, the new oracledb.smon.type attribute, the reused network.io.direction attribute (on oracledb.gc.current_block.time), and the extended oracledb.session.type attribute (now background/foreground). internal/metadata/generated_*.go and internal/metadata/testdata/config.yaml regenerated via make generate. internal/metadata/config.schema.yaml was updated for the 5 new metric stanzas, as that file is not produced by make generate.

Authorship

  • I, a human, wrote this pull request description myself.

@rluidash
rluidash force-pushed the feat/oracledb-txn-lock-recovery branch from 9922b8f to f17e930 Compare June 15, 2026 09:40
Surfaces transaction-rollback, lock-timing, recovery, SMON, and RAC
cache-fusion statistics already returned by the existing system-statistics
query as seven opt-in metrics. No new SQL queries are introduced.
@rluidash
rluidash force-pushed the feat/oracledb-txn-lock-recovery branch from f17e930 to 275a237 Compare June 15, 2026 17:38
rluidash added 6 commits June 17, 2026 23:31
…tion in descriptions

- Clarify oracledb.gc.current_block.receive.time is RAC global cache (Cache Fusion), not JVM GC
- Note transaction.rollbacks vs user_rollbacks and lock.wait.time vs lock.time distinctions
…sstat statistic)

The 'Total Lock Time' statistic does not exist in v$sysstat on supported Oracle
versions (confirmed via v$statname on 19c), so the metric never emitted. Background
and foreground lock timing remain available via oracledb.lock.time.
# Conflicts:
#	receiver/oracledbreceiver/internal/metadata/generated_metrics_test.go
…ock-recovery

# Conflicts:
#	receiver/oracledbreceiver/documentation.md
#	receiver/oracledbreceiver/internal/metadata/config.schema.yaml
#	receiver/oracledbreceiver/internal/metadata/generated_config.go
#	receiver/oracledbreceiver/internal/metadata/generated_config_test.go
#	receiver/oracledbreceiver/internal/metadata/generated_metrics.go
#	receiver/oracledbreceiver/internal/metadata/generated_metrics_test.go
#	receiver/oracledbreceiver/internal/metadata/testdata/config.yaml
#	receiver/oracledbreceiver/metadata.yaml
#	receiver/oracledbreceiver/scraper.go
#	receiver/oracledbreceiver/scraper_test.go
Replace oracledb.smon.instance_recovery.posts and
oracledb.smon.txn_recovery.posts with a single oracledb.smon.posts metric
carrying an oracledb.smon.type attribute (instance|transaction), matching
the general-noun + .type attribute convention used elsewhere in the
receiver (e.g. oracledb.sort.operations). Also trim the changelog subtext
to drop the metric and attribute enumeration.
@rluidash
rluidash marked this pull request as ready for review July 2, 2026 00:04
@rluidash
rluidash requested review from a team, atoulme, crobert-1 and dmitryax as code owners July 2, 2026 00:04
Comment thread receiver/oracledbreceiver/metadata.yaml Outdated
oracledb.executions:
description: The number of times a specific SQL query has been executed (reporting delta).
type: int
oracledb.lock.type:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oracledb.blocking.lock.type represents actual Oracle lock types, whereas the proposed oracledb.lock.type uses background and foreground. IMHO, lock.type should have a consistent meaning with or without the blocking context. The values background and foreground seem closer to an execution context, such as oracledb.session.type. Could having two different meanings for “lock type” be confusing to users?

@rluidash rluidash Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — you're right that background/foreground are session classes, not lock modes, so overloading lock.type alongside blocking.lock.type was confusing. The two source statistics ("transaction lock background get time" / "foreground wait time") attribute the time by session class. I've dropped oracledb.lock.type and oracledb.lock.time now reuses the existing oracledb.session.type attribute (extended to [background, foreground]), which removes the collision and reuses an attribute the receiver already defines (it's also used by oracledb.db_time).

component: receiver/oracledb

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Add transaction, lock, and recovery metrics sourced from the existing v$sysstat query.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
note: Add transaction, lock, and recovery metrics sourced from the existing v$sysstat query.
note: Add transaction, lock, and recovery metrics

Remove implementation details, the subtext provides enough context for the average end user.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread receiver/oracledbreceiver/metadata.yaml Outdated
value_type: int
input_type: string
unit: "{executions}"
oracledb.gc.current_block.receive.time:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should receive be some kind of attribute rather than namespace?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point — Oracle also exposes gc current block send time, so receive is really a direction. Renamed the metric to oracledb.gc.current_block.time and added an oracledb.gc.direction attribute (receive now, with send available as a future value), keeping current_block in the name since it's a distinct block state from cr block.

Comment thread receiver/oracledbreceiver/metadata.yaml Outdated
value_type: int
input_type: string
unit: "{queries}"
oracledb.recovery.blocks_read:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"blocks_read" doesn't seem right. Should it be oracledb.recovery.blocks.count and then an attribute for read?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, renamed to oracledb.recovery.blocks ({block}) , matching the existing oracledb.redo.blocks. v$sysstat only exposes "recovery blocks read" (no "written" counterpart), so a single-valued read attribute would be permanently decorative — happy to switch to .count + a read attribute if you'd prefer that style.

Comment thread receiver/oracledbreceiver/metadata.yaml
Comment thread receiver/oracledbreceiver/metadata.yaml

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple comments:

  1. Sort groups alphabetically
  2. Remove comments about unit conversions

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

rluidash added 2 commits July 7, 2026 23:31
…tidy descriptions/scraper

Reuse the existing oracledb.session.type attribute on oracledb.lock.time
(dropping oracledb.lock.type); rename oracledb.gc.current_block.receive.time
to oracledb.gc.current_block.time with an oracledb.gc.direction attribute;
trim the changelog note, remove unit references from descriptions, and
alphabetize the added constant/gate/case groups in scraper.go
Drop the underscore in the leaf name; use the plural-noun count form
Comment thread receiver/oracledbreceiver/metadata.yaml Outdated
oracledb.executions:
description: The number of times a specific SQL query has been executed (reporting delta).
type: int
oracledb.gc.direction:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is gc the namespace for this? I see it's used for a gc metric, but since the attribute is not specific to gc maybe it should be named to match its description more closely.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point — the direction isn't gc-specific. Reused the network.io.direction attribute on oracledb.gc.current_block.time instead of a gc-specific one, and generalized that attribute's description to cover both uses.

Comment thread receiver/oracledbreceiver/metadata.yaml Outdated
value_type: int
unit: By
oracledb.transaction.rollbacks:
description: Number of transactions rolled back.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Maybe add "Total" at the beginning of the description to make it slightly more obvious?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done - added 'Total' to the description.

Comment thread receiver/oracledbreceiver/scraper.go Outdated
tableScansRowidRangesStat = "table scans (rowid ranges)"
userCallsStat = "user calls"
// Transactions, Locks & Recovery v$sysstat names
gcCurrentBlockReceiveTime = "gc current block receive time"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please alphabetize this list, as well as the scrape method and added variables in scrape_test.go too.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

…acks with Total, alphabetize lists

Reuse the existing network.io.direction attribute on
oracledb.gc.current_block.time instead of a gc-specific attribute (and
generalize that attribute's description). Add 'Total' to the
oracledb.transaction.rollbacks description. Merge the added v$sysstat name
constants and enable-gate entries into the existing alphabetical lists and
alphabetize the enable-config block and switch cases in scraper_test.go.
…ock-recovery

# Conflicts:
#	receiver/oracledbreceiver/internal/metadata/generated_config_test.go
#	receiver/oracledbreceiver/internal/metadata/generated_metrics.go
#	receiver/oracledbreceiver/internal/metadata/generated_metrics_test.go
#	receiver/oracledbreceiver/metadata.yaml
@rluidash
rluidash requested a review from crobert-1 July 10, 2026 06:58
@@ -393,6 +401,71 @@ func TestScraper_ScrapeOperationalMetrics(t *testing.T) {
}

// scrapeWithConfig starts a scraper over the shared fake responses and returns one scrape.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment needs to be moved to be next to its relevant method

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@opentelemetry-pr-dashboard

Copy link
Copy Markdown

This PR has review comments. Review suggestions, whether from maintainers or automated reviewers, aren't always correct or required. Please evaluate each comment on its merits, then make sure each thread has a clear outcome.

For example, link to the commit if you applied a suggestion, explain why it wasn't applied, or ask a follow-up question.

Automation flags a PR for human review once every review thread has a reply or is marked as resolved.

Status across open PRs is visible on the pull request dashboard.

rluidash added 2 commits July 14, 2026 14:02
…ock-recovery

# Conflicts:
#	receiver/oracledbreceiver/documentation.md
#	receiver/oracledbreceiver/internal/metadata/generated_config_test.go
#	receiver/oracledbreceiver/scraper_test.go
@rluidash
rluidash requested a review from crobert-1 July 15, 2026 05:29
@crobert-1 crobert-1 added the ready to merge Code review completed; ready to merge by maintainers label Jul 15, 2026

@songy23 songy23 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please fix merge conflicts

@songy23 songy23 removed the ready to merge Code review completed; ready to merge by maintainers label Jul 15, 2026
…ock-recovery

# Conflicts:
#	receiver/oracledbreceiver/documentation.md
#	receiver/oracledbreceiver/internal/metadata/config.schema.yaml
#	receiver/oracledbreceiver/internal/metadata/generated_config.go
#	receiver/oracledbreceiver/internal/metadata/generated_config_test.go
#	receiver/oracledbreceiver/internal/metadata/generated_metrics.go
#	receiver/oracledbreceiver/internal/metadata/generated_metrics_test.go
#	receiver/oracledbreceiver/internal/metadata/testdata/config.yaml
#	receiver/oracledbreceiver/metadata.yaml
@rluidash

Copy link
Copy Markdown
Contributor Author

Done — conflicts resolved.

@rluidash
rluidash requested a review from songy23 July 15, 2026 21:56
@crobert-1 crobert-1 added the ready to merge Code review completed; ready to merge by maintainers label Jul 15, 2026
@dmitryax
dmitryax merged commit f7fef01 into open-telemetry:main Jul 15, 2026
174 checks passed
@otelbot

otelbot Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution @rluidash! 🎉 We would like to hear from you about your experience contributing to OpenTelemetry by taking a few minutes to fill out this survey.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready to merge Code review completed; ready to merge by maintainers receiver/oracledb

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add transaction, lock, and recovery metrics: rollbacks, lock timing, SMON recovery, and RAC cache fusion

7 participants