ScrubbingOptions extra_patterns: scrub-message leaks the matched credential substring

[msrdic]

Bug Report

Summary

When using logfire.ScrubbingOptions(extra_patterns=[...]), Logfire replaces matched attribute values with [Scrubbed due to '<reason>']. The <reason> field literally contains the matched substring, exposing the credential the scrubbing was meant to hide.

Reproduction

import re
import logfire

logfire.configure(
    scrubbing=logfire.ScrubbingOptions(
        extra_patterns=[re.compile(r"://[^:@/]+:[^@/]+@")]  # match embedded URL credentials
    )
)

with logfire.span("connect", config_url="postgresql://admin:s3cr3t_pass@db.internal:5432/mydb"):
    pass

Inspect the exported span. The config_url attribute will contain:

[Scrubbed due to '://admin:s3cr3t_pass@']

The credential admin:s3cr3t_pass is visible in the scrub marker itself.

Expected Behaviour

The scrub marker must not echo the matched text. Acceptable alternatives:

Format	Notes
[Scrubbed]	Simplest
[Scrubbed: matched sensitive pattern]	Adds context without leaking value
[Scrubbed: pattern matched at offset N, length M]	Structural info only

Why This Matters

The scrub message is often written to the same telemetry backend the operator reads to investigate incidents. An operator with read access to traces sees [Scrubbed due to '://user:password@db/…'] — the credential is right there, defeating the purpose of scrubbing. This is worse than no scrubbing: it draws the eye to a well-formatted credential.

This was discovered during a production logging audit. Attributes like db.connection_string (auto-instrumented by SQLAlchemy via OpenTelemetry) and config URL fields passed explicitly to spans were leaking URL-embedded credentials through this mechanism.

Suggested Fix

In the scrub-message construction, remove the matched substring from the replacement string. Something like:

# Before (leaks the matched text)
replacement = f"[Scrubbed due to '{matched_text}']"

# After (safe)
replacement = "[Scrubbed: matched sensitive pattern]"

If the reason string is meant to identify which pattern matched, use the pattern index or a user-supplied label — never the matched value itself.

Workaround

Avoid extra_patterns for URL-credential fields captured in span arguments. Instead, use request_attributes_mapper (FastAPI/ASGI) or a scrub callback that returns None/[REDACTED] rather than the default marker. For OTel auto-instrumented fields (db.connection_string, http.url) the scrub message leak still applies until this is fixed upstream, so treat [Scrubbed due to …] markers as unreliable evidence of redaction.

Environment

• Affects all versions of logfire that support extra_patterns in ScrubbingOptions
• Python 3.11+

Related

• support scrub http.url #1504 — scrubbing http.url attributes (related scrubbing coverage gap)

[alexmojaki]

Right, scrubbing wasn't designed with this kind of structural pattern in mind, it's meant to spot keys like 'password'.

There are two related problems:

• The matched substring also appears in the logfire.scrubbed attribute. This is there to generate code config suggestions in the UI and we don't want to lose that.
• When the whole value matches a pattern, it's not scrubbed. This is meant to avoid scrubbing things like mode: 'password'.

I agree this is a real problem but I think it might need a different API to treat this kind of pattern differently. Ideas welcome.

[msrdic]

I understand it wasn't designed with that kind of pattern in mind and that its primary use is simple scrubbing. As indicated above, there are workarounds for that.

I don't see this as the contract problem. The surprising part is: scrubbing was supposed to make sure sensitive data (1) is not logged and (2) doesn't leave the source system. By default, (1) will be true for simple scenarios. But (2) can still be classified as a security leak, since metadata leaves the source system.

Am I reading this wrong?

[alexmojaki]

I don't understand what you're asking.

[msrdic]

I am sorry, let me try again and be more clear.

If I, as a user of logfire, read scrubbing guide and follow it, I assume that my sensitive data is going to be scrubbed. That is correct, in a limited way, for simple patterns – and that's fine, I think it should have simple mechanism and ways of extending scrubbing.

However, my secrets are still, by default, being sent as metadata (as scrubbing reason) to logfire platform. They're leaving the source system. That is a security leak for me.

My questions are: is my understanding wrong? Why is that behaviour the default?

It is likely I mis-interpreted the whole mechanism of scrubbing and its intent, but it was very surprising implementing scrubbing and still seeing them in logfire platform.

[alexmojaki]

Because we made a mistake not anticipating this kind of usage.

[alexmojaki]

To clarify, I wasn't aware of this problem (apart from the whole value match thing) until you pointed it out.