Skip to content

PIPELINE_AUDIT.md

Latest commit

 

History

History
235 lines (169 loc) · 6.05 KB

PIPELINE_AUDIT.md

File metadata and controls

235 lines (169 loc) · 6.05 KB
Raw

🔍 Pipeline Audit Report

Date: December 7, 2025
Status: ⚠️ ISSUES FOUND

✅ What's Working Well

  1. Hyper-niche variations - Added before scoring ✅
  2. Volume lookup timing - Happens after final limit (efficient) ✅
  3. Error handling - Most steps have try/catch ✅
  4. Deduplication - Multiple passes catch duplicates ✅
  5. Parallel processing - AI generation batches run in parallel ✅

⚠️ Issues Found

1. Gap Analysis Scoring Mismatch - MEDIUM PRIORITY

Issue:

  • Gap keywords use aeo_score from SE Ranking as their score
  • Scoring function skips gap_analysis keywords (line 1056)
  • aeo_score ≠ company-fit score

Code:

# Line 373: Gap keywords get aeo_score as score
"score": gap.get("aeo_score", 50),

# Line 1056: Scoring function skips gap keywords
ai_keywords = [kw for kw in keywords if kw.get("source") != "gap_analysis"]

Impact:

  • Gap keywords might not reflect company-fit accurately
  • aeo_score is based on volume/difficulty, not company relevance

Fix:

  • Option A: Score gap keywords too (recommended)
  • Option B: Rename aeo_score → gap_score and keep separate from company-fit score

2. Bonus Keywords Not Properly Scored - HIGH PRIORITY

Issue:

  • Bonus keywords from PAA added AFTER scoring step
  • Hardcoded score of 60 (line 273)
  • Not scored for company-fit

Code:

# Line 270-276: Bonus keywords added after scoring
bonus_kw_dicts = [
    {"keyword": kw, "intent": "question" if "?" in kw else "informational", 
     "score": 60, "source": "serp_paa", "is_question": "?" in kw}
    for kw in bonus_keywords[:config.target_count // 4]
]
all_keywords.extend(bonus_kw_dicts)

Impact:

  • Bonus keywords might not be relevant to company
  • Score of 60 might be too low/high depending on relevance
  • Could filter out good keywords or include bad ones

Fix:

  • Score bonus keywords properly before adding them
  • Or add them before scoring step

3. SERP Analysis Timing - LOW PRIORITY (Actually OK)

Status: ✅ NOT AN ISSUE (False alarm)

Reality:

  • SERP analysis happens AFTER clustering
  • Final limit happens AFTER SERP analysis (line 281)
  • So we analyze keywords that will be in final result

Why it's OK:

  • We want SERP data for final keywords
  • Analyzing before limit ensures we have data for top keywords
  • Only analyzes top 15 anyway (configurable)

Verdict: ✅ Working as intended

4. Google Trends/Autocomplete Not Integrated - FEATURE GAP

Status: ⚠️ MISSING FEATURE (not a bug)

Reality:

  • Code exists (GoogleTrendsAnalyzer, AutocompleteAnalyzer)
  • But not called in main pipeline

Impact:

  • Missing trending keyword detection
  • Missing autocomplete suggestions
  • Missing seasonality data

Fix:

  • Add as optional steps in pipeline
  • Integrate after research, before gap analysis

5. Multiple Deduplication Steps - DESIGN CHOICE

Status: ✅ INTENTIONAL (not an issue)

Reality:

  • Fast dedup after research/gap/AI (Step 3)
  • Fast dedup after hyper-niche (Step 3.5)
  • Semantic dedup after scoring (Step 5)
  • Fast dedup after bonus keywords (Step 8)

Why it's OK:

  • Each step adds new keywords
  • Need to dedupe after each addition
  • Fast dedup is O(n) - very fast
  • Semantic dedup catches near-duplicates

Verdict: ✅ Good design

🔧 Recommended Fixes

Priority 1: Fix Bonus Keywords Scoring

Current:

# Bonus keywords added after scoring with hardcoded score
bonus_kw_dicts = [{"score": 60, ...}]
all_keywords.extend(bonus_kw_dicts)

Fix:

# Option A: Score bonus keywords before adding
if bonus_keywords:
    bonus_kw_dicts = [
        {"keyword": kw, "intent": "question" if "?" in kw else "informational", 
         "score": 0, "source": "serp_paa", "is_question": "?" in kw}
        for kw in bonus_keywords[:config.target_count // 4]
    ]
    all_keywords.extend(bonus_kw_dicts)
    # Score them properly
    all_keywords = await self._score_keywords(all_keywords, company_info)

OR:

# Option B: Add bonus keywords BEFORE scoring step
# Move bonus keyword extraction to before Step 4 (scoring)

Priority 2: Fix Gap Analysis Scoring

Current:

# Gap keywords use aeo_score as score
"score": gap.get("aeo_score", 50),

# Scoring function skips them
ai_keywords = [kw for kw in keywords if kw.get("source") != "gap_analysis"]

Fix:

# Option A: Score gap keywords too
# Remove the filter, score all keywords

# Option B: Keep separate scores
# Rename aeo_score → gap_score, keep score for company-fit

Recommendation: Option A - Score gap keywords for company-fit too

Priority 3: Integrate Google Trends/Autocomplete

Add as optional steps:

# Step 1.5: Google Trends (if enabled)
if config.enable_trends:
    trend_data = await trends_analyzer.analyze_keywords(keywords)
    # Enrich keywords with trend data

# Step 0.5: Autocomplete (if enabled)
if config.enable_autocomplete:
    autocomplete_keywords = await autocomplete_analyzer.get_suggestions(seed)
    all_keywords.extend(autocomplete_keywords)

📊 Summary

Issue Priority Status Impact
Bonus keywords scoring HIGH ⚠️ Bug Keywords might not be relevant
Gap analysis scoring MEDIUM ⚠️ Design Scores might not reflect company-fit
SERP timing LOW ✅ OK Working as intended
Google Trends missing LOW ⚠️ Feature gap Missing trending keywords
Multiple dedup LOW ✅ OK Good design

✅ Overall Assessment

Pipeline is mostly solid with 2 issues to fix:

  1. Bonus keywords need proper scoring (HIGH priority)
  2. Gap keywords should get company-fit scores (MEDIUM priority)

Everything else is working as intended!

🎯 Action Items

  1. ✅ Fix bonus keywords scoring (add before scoring or score after adding)
  2. ✅ Fix gap analysis scoring (score gap keywords too)
  3. ⚠️ Consider integrating Google Trends/Autocomplete (optional)