Complete point-in-time universe remediation and v1 research closeout - #12
Conversation
dvp2004
changed the title
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4e33dcaa26
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| raw_date = row.get("date") or row.get("effective date") or "" | ||
| try: | ||
| effective = date.fromisoformat(raw_date) | ||
| except ValueError: | ||
| continue |
There was a problem hiding this comment.
When the pinned Wikimedia change table supplies its rendered date format (for example, September 22, 2025) rather than an ISO string, date.fromisoformat raises and this code silently discards the row. Consequently, the post-2025-08-23 additions and removals are omitted and the endpoint universe remains the stale seed universe; parse the source's actual date representation and fail closed on unexpected nonempty dates instead of skipping them.
Useful? React with 👍 / 👎.
| reconstructed, extension_conflicts = apply_membership_extensions( | ||
| active_tickers=seed_symbols, | ||
| extensions=extensions, | ||
| ) |
There was a problem hiding this comment.
When membership changed between the seed's 2025-08-23 snapshot and the frozen endpoint, the earlier seed_symbols ^ wiki_symbols loop records every legitimate intervening change as an unresolved conflict before these extensions are applied. Even if replay produces exactly wiki_symbols, those conflicts remain and later force membership_sources_do_not_fully_reconcile; compare reconstructed with the endpoint symbols or replace the provisional conflicts after replay.
Useful? React with 👍 / 👎.
| response = self.get_response(url, headers, self.timeout_seconds) | ||
| self._last_request_at = time.monotonic() |
There was a problem hiding this comment.
When requests.get raises a timeout or connection exception, this call exits _fetch immediately because only returned HTTP statuses participate in the retry loop. The pipeline then marks the entire SEC identity source unavailable after one transient failure despite maximum_attempts being configured; catch retryable transport exceptions here and apply the same bounded backoff policy.
Useful? React with 👍 / 👎.
| snapshot_manifest["sources"][price_source.source_id] = { | ||
| "bounded_ticker_limit": int(limits["maximum_price_tickers"]), | ||
| "audit_scope": "all_effective_dated_provider_tickers", | ||
| "requested_tickers": sample_tickers, | ||
| "results": price_results, |
There was a problem hiding this comment.
For an old or delisted ticker where Yahoo returns only a truncated portion of the requested history, this manifest claims a full effective-dated audit even though price_coverage later labels the security available whenever any row exists. Missing prices at the start or end of its membership interval therefore do not contribute to the global price-coverage failure count; drive the audit from effective ticker intervals and classify first/last coverage against those bounds before recording this scope.
Useful? React with 👍 / 👎.
1 participant
Summary
Final canonical universe result
blocked_identity_reconciliation_failure2026-05-01SEC acquisition succeeded with HTTP 200 and 10,398 mappings. Alpha Vantage was not used and remained optional.
Research conclusion
Validation
Boundaries