Skip to content

[do not merge] bisect - #20357

Draft
KowalskiThomas wants to merge 2 commits into
4.12from
kowalski/bisect
Draft

[do not merge] bisect#20357
KowalskiThomas wants to merge 2 commits into
4.12from
kowalski/bisect

Conversation

@KowalskiThomas

@KowalskiThomas KowalskiThomas commented Sep 16, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

@KowalskiThomas
KowalskiThomas changed the base branch from main to 4.11 September 16, 2026 06:04
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Sep 16, 2026 •
edited
Loading

Copy link
Copy Markdown

Codeowners resolved as

.gitlab/package.yml                                                     @DataDog/python-guild @DataDog/apm-core-python
.gitlab/scripts/patch-wheel-versions.py                                 @DataDog/python-guild @DataDog/apm-core-python
.gitlab/testrunner.yml                                                  @DataDog/python-guild @DataDog/apm-core-python
.gitlab/validate-ddtrace-package.py                                     @DataDog/python-guild @DataDog/apm-core-python

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Sep 16, 2026 •
edited by datadog-prod-us1-3 Bot
Loading

Copy link
Copy Markdown
Contributor

Pipelines

✨ Unblock PR with BitsAI

❌ Errors

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 19 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-py | build linux: [amd64, cp315-cp315, v113741238-d2b8243-manylinux2014_x86_64] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-py | build linux: [amd64, cp315-cp315, v113741491-d2b8243-musllinux_1_2_x86_64] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-py | build linux: [arm64, cp315-cp315, v113741357-d2b8243-manylinux2014_aarch64] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

View all 19 failed jobs.

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 5c3cb12 | Docs | View more details | Give us feedback!

@pr-commenter

pr-commenter Bot commented Sep 16, 2026 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-09-16 09:35:43

Comparing candidate commit 5c3cb12 in PR branch kowalski/bisect with baseline commit 1a6d423 in branch 4.12.

📊 Benchmarking dashboard

Found 0 performance improvements and 6 performance regressions! Performance is the same for 617 metrics, 9 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:httppropagationinject-ids_only

  • 🟥 execution_time [+1.636µs; +1.845µs] or [+8.222%; +9.274%]

scenario:iastaspects-strip_aspect

  • 🟥 execution_time [+54.243µs; +57.583µs] or [+17.213%; +18.273%]

scenario:iastaspectsospath-ospathbasename_aspect

  • 🟥 execution_time [+110.235µs; +117.578µs] or [+25.262%; +26.945%]

scenario:iastaspectssplit-rsplit_aspect

  • 🟥 execution_time [+11.271µs; +14.805µs] or [+7.383%; +9.698%]

scenario:span-start

  • 🟥 execution_time [+1.554ms; +1.691ms] or [+10.851%; +11.810%]

scenario:tracer-small

  • 🟥 execution_time [+26.879µs; +29.069µs] or [+8.075%; +8.733%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:coreapiscenario-context_with_data_listeners

  • unstable execution_time [-828.172ns; +633.119ns] or [-7.168%; +5.480%]

scenario:coreapiscenario-core_dispatch_50_listeners

  • unstable execution_time [-1625.941ns; +1673.886ns] or [-9.601%; +9.884%]

scenario:coreapiscenario-core_dispatch_exception_listeners

  • unstable execution_time [-1218.776ns; +1240.145ns] or [-9.428%; +9.593%]

scenario:coreapiscenario-core_dispatch_listeners

  • unstable execution_time [-409.087ns; +245.336ns] or [-10.996%; +6.594%]

scenario:coreapiscenario-core_dispatch_no_args_listeners

  • unstable execution_time [-309.597ns; +197.256ns] or [-10.481%; +6.678%]

scenario:coreapiscenario-core_dispatch_with_results_1_listener

  • unstable execution_time [-83.150ns; +63.144ns] or [-6.840%; +5.195%]

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

  • unstable execution_time [-4238.357ns; +3694.591ns] or [-10.381%; +9.049%]

scenario:coreapiscenario-core_dispatch_with_results_listeners

  • unstable execution_time [-824.088ns; +736.499ns] or [-10.107%; +9.032%]

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

  • unstable execution_time [-34.445µs; +47.618µs] or [-5.585%; +7.721%]

KowalskiThomas and others added 2 commits September 16, 2026 11:02
@KowalskiThomas
update pyproject
dc3b447
@KowalskiThomas @brettlangdon
ci: add job to push wheels to adms (#19727)
5c3cb12 
## Description

More context on this change is available [in this document](https://docs.google.com/document/d/1NPQvC81luziY4SwiQUxXzgml5K78IurBp1jFXG3QZgE/edit?tab=t.0#heading=h.ty5eia46b19).

This PR updates our CI pipeline to have two new jobs (that do the same thing for `manylinux` builds and `macOS` builds, respectively). This new job edits the built wheels in order to modify their version, and change it from the one that is in `pyproject.toml` to an equivalent `major.minor.patch[rcX].dev<pipeline_id>+<local_segment_from_pyproject>.<commit_sha>.
The job only runs when the `pyproject` version has a local segment. 


Once all wheels have been patched, they are pushed to our internal Python wheel repository ADMS (operated by Dependency Management) and it is thus possible to "legally" use wheels not released to PyPI in internal services.

## Testing

This has been tested in CI and through `adms` tools. I also have PRs for `dogweb` and `dd-source` which make the necessary changes for them to use the special "unreleased `ddtrace` wheels" internal repository.  (I made some changes to the `pyproject` that are now reverted.)

For someone wanting to use an unreleased version of `ddtrace` internally, that setup will be transparent -- they will only need to set the right version in downstream `requirements` files.

## Alternatives considered

Currently the job runs automatically based on the `pyproject` version; it could have been a manual job instead. [Discussion here.](#19727 (comment))

Co-authored-by: brett.langdon <brett.langdon@datadoghq.com>
@KowalskiThomas
KowalskiThomas changed the base branch from 4.11 to 4.12 September 16, 2026 09:04
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Sep 16, 2026 •
edited
Loading

Copy link
Copy Markdown

Circular import analysis

🚨 New circular imports detected 🚨

4974 new circular import(s) have been introduced by this PR:

Show new cycles (showing 5 of 4974 shortest) 
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.botocore -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.kafka -> ddtrace.internal.datastreams
ddtrace.internal.core -> ddtrace._trace.span -> ddtrace.internal.core
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.aiokafka -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.google_cloud_pubsub -> ddtrace.internal.datastreams

To see all cycles, download the cycles-base.json and cycles-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/cycles.py compare cycles-base.json cycles-pr.json

Please consider refactoring your changes in accordance to the Separation of Concerns principle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@KowalskiThomas