Skip to content

FFS-4592: Fix NSC test scenarios in launcher - #2012

Open
bencalegari wants to merge 4 commits into
mainfrom
bc/ffs-4592-fix-nsc-launcher-scenarios
Open

bencalegari wants to merge 4 commits into
mainfrom
bc/ffs-4592-fix-nsc-launcher-scenarios

Conversation

@bencalegari

@bencalegari bencalegari commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

FFS-4592

Rick and other NSC results were failing because nameOnSchoolRecord had gone missing from the api response. We were getting NoMethodErrors, but the retry logic was so broad we didn't see the failure show up locally for many many minutes later. I updated the fixtures and guarded against it, defaulting to the identity's first and last name. Personas also changed status in the sandbox: Dominique now returns CC (currently enrolled) instead of CN, so she took over the full-time scenario and Scott Tobin was added to cover the not-currently-enrolled case.

Changes

  • Re-recorded all NSC fixtures against the live NSC sandbox; replaced the ad-hoc nsc_save_fixture_for_user helper with a FIXTURE_PERSONAS table plus nsc_save_all_fixtures so fixtures can be regenerated in one command.
  • Added the Scott Tobin launcher scenario (not currently enrolled / enrollment older than 18 months) and moved Dominique Ricardo to the "enrolled full time" scenario, matching the re-recorded fixture data.
  • Changed the simple launcher's default student status from Lynette to Dominique and added a "Enrolled, intensity not reported" option backed by Lynette.
  • Launcher::NscForwardDatingService anchors forward-dating on the latest term ending on or before the persona's launcher as-of date, rather than the latest term overall, and adds a stable as-of date for Dominique.
  • ApplicationJob no longer retries programming errors (NameError, TypeError, ArgumentError) or ActiveRecord::RecordInvalid, they fail on the first attempt.
  • Deleted the unused lakhshmi_saxena fixture.
  • Snuck in a change Paul requested to show credit hours/hours if present on education card.

Context for reviewers

  • The NSC sandbox stopped returning nameOnSchoolRecord and studentAddress and now returns schoolType, so the re-recorded fixtures dropped those fields. schoolType is a string of a number referring to whether its a 4 year, 2 year, or 1 (less than 2 year) institution which isn't relevant for our purposes.
  • Lynette's fixture now has two terms, and the newer one (2026-05-01 – 2027-05-01) begins after her as-of date. Anchoring on the max term end shifted dates backwards; anchoring on the latest term at or before the as-of date is what keeps forward-dating correct.
  • There was a bug where self attested credit hours weren't being shown on the doc upload page for Lynette and I snuck in the fix: the issue was enrollment_less_than_half_time? only returns true when the status is literally less_than_half_time where we actually want it to be true when its less_than_half_time, enrolled, or unknown, which is the logic we used everywhere except there.

Acceptance testing

Tag product and design in Slack for acceptance: @emmy-acceptance-testers

  • Acceptance testing in PR Environment
    • This change can be verified in a PR environment. Run this Github Action with the existing PR and most recent git sha.

AI Usage

  • USED_AI: I attest that I have read, understood, and take ownership of all AI-generated code in this PR.
  • NO_AI: I did not use AI.

CMS Cloud review environment

Copilot AI lite review requested due to automatic review settings August 28, 2026 21:28
@github-actions
github-actions Bot temporarily deployed to cms-dev-pr-2012 August 28, 2026 21:30 Destroyed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates NSC launcher/test personas and fixtures to reflect current NSC sandbox responses (missing nameOnSchoolRecord, changed statuses), while hardening enrollment parsing and refining forward-dating so scenarios remain stable and debuggable.

Changes:

  • Re-recorded/updated NSC sandbox fixtures and added a FIXTURE_PERSONAS table + bulk fixture regeneration helper.
  • Updated launcher scenarios (Dominique now full-time; added Scott Tobin for “not currently enrolled”) and adjusted specs/views accordingly.
  • Guarded NscDataFetcherService against missing nameOnSchoolRecord, refined forward-dating anchor term selection, and narrowed job retry behavior for programming/validation errors.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
app/spec/support/nsc_api_helper.rb Adds FIXTURE_PERSONAS and bulk fixture regen helper for NSC sandbox fixtures.
app/spec/support/fixtures/nsc/scott_tobin/insight.json Updates fixture response structure/fields for Scott (CN scenario).
app/spec/support/fixtures/nsc/rick_banas/insight.json Updates Rick fixture, reflecting sandbox schema changes (e.g., schoolType).
app/spec/support/fixtures/nsc/lynette/insight.json Updates Lynette fixture and term ordering/data for forward-dating behavior.
app/spec/support/fixtures/nsc/linda/insight.json Updates Linda fixture formatting and transaction metadata.
app/spec/support/fixtures/nsc/lakhshmi_saxena/insight.json Removes unused fixture.
app/spec/support/fixtures/nsc/dominique_ricardo/insight.json Updates Dominique fixture to match new CC status and schema.
app/spec/services/nsc_data_fetcher_service_spec.rb Adds coverage for missing/present nameOnSchoolRecord and swaps CN scenario to Scott.
app/spec/services/launcher/nsc_forward_dating_service_spec.rb Adds coverage for new anchoring behavior and new Scott scenario behavior.
app/spec/jobs/nsc_synchronization_job_spec.rb Ensures programming errors fail fast and don’t enqueue retries.
app/spec/jobs/application_job_spec.rb Adds explicit non-retryable error behavior tests.
app/spec/factories/identity.rb Adds :nsc_scott identity trait for the new persona.
app/spec/controllers/launcher_controller_spec.rb Updates simple launcher default scenario and verifies preselection behavior.
app/spec/controllers/launcher_controller_advanced_spec.rb Updates advanced launcher persona list and applicant creation checks (adds Scott).
app/app/views/launcher/launcher.html.erb Updates simple launcher default selection and adds “intensity not reported” option.
app/app/views/launcher/advanced.html.erb Updates advanced launcher persona tiles (Dominique full-time; adds Scott).
app/app/services/nsc_data_fetcher_service.rb Falls back to identity name when nameOnSchoolRecord is missing.
app/app/services/launcher/nsc_forward_dating_service.rb Anchors forward-dating on latest term end <= as-of date; adds stable as-of for Dominique.
app/app/jobs/application_job.rb Introduces NON_RETRYABLE_ERRORS and limits retries for programming/validation errors.
app/app/controllers/launcher_controller.rb Adds Scott to launcher persona map.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread app/spec/support/nsc_api_helper.rb
Comment thread app/app/views/launcher/advanced.html.erb
Comment thread app/app/jobs/application_job.rb
@bencalegari
bencalegari marked this pull request as ready for review September 1, 2026 17:00
@bencalegari
bencalegari requested a review from a team as a code owner September 1, 2026 17:00
@github-actions
github-actions Bot requested a review from tdooner September 1, 2026 17:03

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test-classifier: AI triage of failing tests

AI Test Classifier — no action required

Observed — these verdicts are grounded in the actual test run output.

Ran the repo's own CI test path (rspec.yml + setup-project action): started Postgres 16, installed gems with bundler, loaded the test schema, ran npm install and rake assets:precompile, then executed rspec. The precomputed AI_REVIEW_DIFF_RANGE (origin/main HEAD) was verified to genuinely reflect PR #2012's change, so no base re-resolution was needed. Targeted run of the 10 changed spec files: 368 examples, 0 failures. Full CI-scope run (spec/ excluding e2e, as rspec.yml does): 2564 examples, 0 failures. An earlier run showed 101 view-rendering controller failures, but all were 'The asset @uswds/uswds/dist/img/sprite.svg is not present in the asset pipeline' caused by my not yet having run the CI bootstrap's npm install + assets:precompile; after completing that bootstrap every one of them passed, so they are a bootstrap artifact and not classified as failures. Nothing to triage.

React 👍 if this is right (nothing needed triage) / 👎 if a real failure was missed, and on a 👎 please reply with a one-line reason. Advisory, non-blocking.

@pauljgehrig

Copy link
Copy Markdown

LGTM! 🚢-it

around_perform :with_error_reporting

retry_on Exception, wait: :polynomially_longer, attempts: 5
retry_on(*NON_RETRYABLE_ERRORS, attempts: 1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should just retry_on StandardError instead of Exception. All of the NON_RETRYABLE_ERRORS you list are not StandardErrors, and there are other ones that are not StandardErrors too that we probably also don't want to retry on, like SyntaxError, NotImplementedError, and LoadError

last_name: enrollment_detail["nameOnSchoolRecord"]["lastName"],
first_name: name_on_school_record["firstName"] || identity&.first_name,
middle_name: name_on_school_record["middleName"],
last_name: name_on_school_record["lastName"] || identity&.last_name,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. I'm not sure we should fall back to saving the identity data again in the nsc_enrollment_terms - since nsc_enrollment_terms is just capturing the NSC result, supplementing it with other data might confuse us as to whether we got the data from NSC or somewhere else.

Can we just leave these fields blank if NSC returns nothing, and then handle it in whatever is reading the NSC enrollment term?

Also, did you say NSC no longer returns this field, or only sometimes? Or do we think it will be returned in production but not in sandbox?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants