M6 — Filtering, validation, reporting, preview & publish gate - #116
Merged
Conversation
- FilterEvaluator resolves a filter's field/value via the reference-resolution rule + the shared operator vocabulary; exclude drops on match, include drops on non-match. Wired into the generator: pre filters run before mapping, post filters on the mapped output; excluded items are counted and skipped. - SourceResolverBinder: bind the item's on-demand source resolver once (before pre-filters, reused by mapping) preserving lookup: delegation; idempotent so FieldMappingEvaluator is unchanged. - FeedFilterType admin form + a filters collection on FeedSourceType. - exclude-out-of-stock proven end to end. +9 tests (512 total); PHPStan max (incl. PHP 8.1) + ECS + Rector + container green.
- FeedType::createItem() lets a feed type provide a typed FeedItem (product_variant
-> GoogleShoppingItem). Symfony constraints on GoogleShoppingItem (required g:
fields + title<=150), applied format-scoped (FormatInterface::getItemValidationGroups;
google_rss runs them, CSV/others fall back to the required-field list) so a CSV
product_variant feed isn't held to the RSS spec.
- FeedItemValidator: typed constraint violations + required-field fallback, deduped.
- FeedContextResult resource (feed, contextKey, itemCount/excludedCount/bytes,
errors json, createdAt) + repository (findLatestPublished) — the publish-gate
baseline. Every exclusion is recorded with a reason (filter:{stage}:{field} /
validation:{msg} / skipped), capped, via ExclusionCollector; recorded at finalize
by FeedContextResultRecorder.
+11 tests (523 total); PHPStan max (incl. 8.1) + ECS + Rector + container green.
- Feed.publishConfig (json guardrails); GuardrailRegistry + guardrails (min_items, max_drop_pct, non_empty, min_bytes, max_exclusion_pct, max_growth_pct) comparing candidate vs the last-good FeedContextResult baseline. - PublishGate returns a decision (blocked + reasons); block-severity trips block, warn only records; empty guardrails = no gate. - FeedContextResult gains publishState (pending/published/blocked) + publishCheck; findLatestPublished filters published so a candidate is never its own baseline. - Finalize is now per-context + gated: GenerateFeedContextHandler runs the gate after recording, stamps publishState, dispatches FeedPublishBlockedEvent when blocked/warned; MoveGeneratedFeedSubscriber no longer wipes canonical -- it promotes only published contexts and retains blocked candidates in staging. - Publish-anyway admin action promotes a retained blocked candidate. Acceptance: a 5k-vs-50k regenerate trips max_drop_pct -> blocked, live file retained, event fired. +44 tests (567 total); all gates (incl. PHP 8.1) green.
- Extract MappingResolver (FeedField-first, preset fallback) shared by the
generator + preview so the pipeline is not forked.
- PreviewService runs the exact per-item pipeline (bind -> pre-filter -> map ->
post-filter -> validate) over <=N sampled items, writing nothing, and returns a
funnel (source -> after pre -> after mapping/validation -> after post ->
included), included sample bags, excluded samples with the dropping reason, and
a single-item tester (previewItem by id).
- FeedAuditService computes per-field fill rates, soft warnings (title>150,
HTML-in-description, price=0) and value distributions over the included sample
-- advisory, excludes nothing.
- Admin Preview screen (/feeds/{id}/preview + grid action) and console
--preview[=N] / --audit options.
+14 tests (581 total); all gates (incl. PHP 8.1 + twig lint) green.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## 1.x #116 +/- ##
============================================
- Coverage 95.32% 95.25% -0.07%
- Complexity 998 1231 +233
============================================
Files 134 161 +27
Lines 2417 3057 +640
============================================
+ Hits 2304 2912 +608
- Misses 113 145 +32 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
M6 — Filtering, validation, reporting, preview, publish gate (§11, §6.6)
Four chunks; 581 tests; the Preview admin screen is browser-verified.
Filtering
FilterEvaluatorresolves aFeedFilter's field/value via the reference rule + the sharedoperator vocabulary;
excludedrops on match,includedrops on non-match. Wired into thegenerator: pre filters run before mapping, post filters on the mapped output. Excluded
items are counted and reported.
SourceResolverBinderbinds the item's source resolver once(before pre-filters). exclude-out-of-stock works.
Validation + reporting
FeedType::createItem()lets a type provide a typedFeedItem(product_variant→GoogleShoppingItem). Symfony constraints onGoogleShoppingItem(requiredg:fields, title≤150), applied format-scoped (RSS runs them; CSV/others fall back to the required-field list).
FeedContextResultresource records per-contextitemCount/excludedCount/bytes+ every exclusionwith its reason (
filter:{stage}:{field}/validation:{msg}/skipped), recorded at finalize.Publish gate + guardrails
Feed.publishConfigguardrails (min_items,max_drop_pct,non_empty,min_bytes,max_exclusion_pct,max_growth_pct) compare the candidate vs the last-goodFeedContextResultbaseline;
PublishGateblocks on a trippedblock-severity guardrail, warns otherwise.GenerateFeedContextHandlerruns the gate afterrecording (stamps
publishState, dispatchesFeedPublishBlockedEvent);MoveGeneratedFeedSubscriberno longer wipes canonical — it promotes only published contexts and retains blocked
candidates in staging. A publish-anyway admin action promotes a retained candidate.
max_drop_pct→ blocked, the live file is retained,the event fires.
Preview + audit
MappingResolver(FeedField-first, preset fallback) reused by the generator and preview.PreviewServiceruns the exact per-item pipeline over ≤N sampled items, writing nothing, returninga funnel (source → after-pre → after-mapping/validation → after-post → included), included
sample bags, excluded samples with reasons, and a single-item tester.
FeedAuditServicereports per-field fill rates, soft warnings (title>150, HTML-in-description,price=0) and value distributions — advisory only.
/feeds/{id}/preview+ grid action) and console--preview[=N]/--audit.CI note
Verified locally against the updated PHPStan/Rector/ECS and PHP-8.1-pinned PHPStan + league/csv
9.6, so the gate (Unit / Integration / Static Code Analysis / Coding Standards) should be green. BC,
Dependency Analysis and Mutation remain the standing rewrite reds.