On file complete hook - #57
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new per-file completion hook (on_file_complete) to the acoustic inference pipeline so callers can receive a single-file result as soon as each file finishes processing, enabling streaming persistence / resumable multi-file workflows.
Changes:
- Plumbs
on_file_completethrough all acoustic model APIs (2.4, 3.0, Perch V2) into the shared inference session/pipeline. - Implements a completion-marker → consumer slice-copy → dispatcher-thread callback pipeline to invoke user callbacks off the inference hot path.
- Adds v2.4 encode/predict test coverage, updates changelog/docs, and includes a throughput benchmark script.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/birdnet/acoustic/models/v3_0/model.py | Exposes on_file_complete in v3.0 model encode/predict APIs and passes it into sessions. |
| src/birdnet/acoustic/models/v2_4/model.py | Exposes on_file_complete in v2.4 APIs with detailed docstrings and wiring into sessions. |
| src/birdnet/acoustic/models/perch_v2/model.py | Exposes on_file_complete in Perch V2 APIs and passes it into sessions. |
| src/birdnet/acoustic/inference/strategy.py | Adds optional build_single_file_result hook for strategies to build per-file results from tensor slices. |
| src/birdnet/acoustic/inference/session.py | Activates per-file completion signaling and waits for completion dispatch before returning aggregate results. |
| src/birdnet/acoustic/inference/resources.py | Adds FileCompletionResources to manage queues/signals for the completion path. |
| src/birdnet/acoustic/inference/process_manager.py | Starts/join a file-completion dispatcher thread and wires completion queues into producer/consumer. |
| src/birdnet/acoustic/inference/prediction_strategy.py | Implements single-file prediction result construction (with cached species-name array). |
| src/birdnet/acoustic/inference/encoding_strategy.py | Implements single-file encoding result construction from copied embedding slices. |
| src/birdnet/acoustic/inference/core/producer.py | Emits per-file completion markers (segments emitted, invalid flag, duration) into a queue. |
| src/birdnet/acoustic/inference/core/prediction/prediction_tensor.py | Adds copy_file_slice and PrebuiltPredictionTensor for safe cross-thread per-file result construction. |
| src/birdnet/acoustic/inference/core/prediction/prediction_result.py | Allows passing a prebuilt species_list_array to avoid per-result rebuilds. |
| src/birdnet/acoustic/inference/core/file_completion.py | New dispatcher component that builds per-file results and invokes the user callback on a background thread. |
| src/birdnet/acoustic/inference/core/encoding/encoding_tensor.py | Adds copy_file_slice and PrebuiltEncodingTensor for per-file encoding results. |
| src/birdnet/acoustic/inference/core/consumer.py | Tracks per-file segment completion and emits dispatch items when a file’s segments are fully written. |
| src/birdnet/acoustic/inference/configs.py | Adds file_completion_callback to OutputConfig to carry the callback through the pipeline. |
| src/birdnet_tests/acoustic_models/v2_4/model_py/test_predict/test_on_file_complete_v2_4.py | New tests for prediction per-file callback behavior (ordering, invalid files, cancellation, etc.). |
| src/birdnet_tests/acoustic_models/v2_4/model_py/test_encode/test_on_file_complete_encode_v2_4.py | New tests for encoding per-file callback behavior and edge cases. |
| README.md | Markdown list formatting tweaks for install/log/file-format sections. |
| CHANGELOG.md | Documents the new on_file_complete callback feature and behavior. |
| benchmarks/on_file_complete_benchmark.py | Adds a benchmark script comparing callback overhead vs baseline/noop/persist callbacks. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (3)
src/birdnet/acoustic/models/perch_v2/model.py:152
encode_session()adds the newon_file_completeparameter, but the Args section of the docstring does not describe it. This makes the public API harder to discover and is inconsistent with the detailed parameter docs already present in this docstring.
on_file_complete: Callable[[AcousticFileEncodingResult], None] | None = None,
) -> AcousticEncodingSession:
"""Create an encoding session with explicit resource configuration.
Args:
src/birdnet/acoustic/models/perch_v2/model.py:232
predict_session()adds the newon_file_completeparameter, but the Args section of the docstring does not describe it. Since the docstring already documents the rest of the session configuration in detail, this omission will confuse API users.
on_file_complete: Callable[[AcousticFilePredictionResult], None] | None = None,
) -> AcousticPredictionSession:
"""Create a prediction session allowing manual control over the inference lifecycle.
Args:
src/birdnet/acoustic/models/perch_v2/model.py:455
predict()adds the newon_file_completeparameter, but the Args section of the docstring does not describe it. Documenting this is important because it changes control flow (exceptions cancel the run) and threading behavior (callback is invoked on a background thread).
on_file_complete: Callable[[AcousticFilePredictionResult], None] | None = None,
) -> AcousticPredictionResultBase:
"""Run prediction with the Perch V2 model on files or paths with configurable
inference options.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 21 out of 22 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (3)
src/birdnet/acoustic/models/perch_v2/model.py:152
- The newly added
on_file_completeparameter is not documented in this session factory’s docstringArgs:list, so users won’t learn about the per-file callback behavior from the API docs.
on_file_complete: Callable[[AcousticFileEncodingResult], None] | None = None,
) -> AcousticEncodingSession:
"""Create an encoding session with explicit resource configuration.
Args:
src/birdnet/acoustic/models/perch_v2/model.py:233
predict_session(..)adds anon_file_completeparameter, but the docstring does not mention it, which makes the public API harder to discover.
on_file_complete: Callable[[AcousticFilePredictionResult], None] | None = None,
) -> AcousticPredictionSession:
"""Create a prediction session allowing manual control over the inference lifecycle.
Args:
src/birdnet/acoustic/models/perch_v2/model.py:466
predict(..)adds anon_file_completeparameter, but the docstring header doesn’t mention the per-file callback behavior, making it easy to miss when reading generated docs.
on_file_complete: Callable[[AcousticFilePredictionResult], None] | None = None,
) -> AcousticPredictionResultBase:
"""Run prediction with the Perch V2 model on files or paths with configurable
inference options.
No description provided.