Skip to content

Refactor: Modularize & Test the Scraper Pipeline #245

Description

@kunal-10-cloud

Describe the bug

The generateLeaderboard.ts script has over 1000 lines, mixing configuration, API calls, and business logic into a single monolithic file. This makes the codebase difficult to maintain, hard to debug, and risky to modify without causing regressions.

To Reproduce

Steps to reproduce the behavior:

  1. Open scripts/generateLeaderboard.ts.
  2. Scroll through the 1000+ lines of code.
  3. Observe mixed responsibilities: API calls, scoring logic, file I/O, and data transformation all in one place.
  4. See that there are no dedicated unit tests for this critical logic.

Expected behavior

The code should be modular and testable.

  • API interactions should be in a GitHubService.
  • Scoring logic should be in a ScoringEngine.
  • Types should be consistent and centralized.
  • Passing unit tests should verify the logic.

Screenshots/GIFs

NA

Device Information [optional]:

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Additional context

Add any other context about the problem here.

Are you working on this issue? (Yes/No)

Yes

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions