Skip to content

pact:verify passes vacuously - provider names in verification specs don't match generated pacts #411

Description

@bekasovoleg

Title: pact:verify passes vacuously — provider names in verification specs don't match generated pacts

Summary

The provider-verification specs under spec/pact/consumers/ declare provider names that match none of the pacts generated by spec/pact/providers/**. Because the verifier only verifies pacts whose provider.name matches the name passed to *_pact_provider, all three verification specs verify zero interactions and still report success. rake pact:verify is a false green in CI.

How the loop works

  • rake pact:specspec/pact/providers/** (has_*_pact_between) generates pacts into spec/pacts/.
  • rake pact:verifyspec/pact/consumers/** (*_pact_provider) loads the whole spec/pacts/ dir via add_directory_source and verifies only pacts whose provider.name matches the declared provider (lib/pact/provider/base_verifier.rb:175, set_provider_info). No match → 0 interactions → green.

Name mismatches

Verifying spec (spec/pact/consumers/) provider declared provider in generated pact Match
multi_spec.rb pact-test-app pact-ruby-test-app ❌ missing -ruby
kafka_spec.rb pact-test-app-kafka pact-ruby-test-app-kafka ❌ missing -ruby
message_spec.rb Test Message Producer Test Message Provider Producer ≠ Provider

The plugin contracts (myprovider, matttcpprovider) are generated but have no verification spec at all.

Reproduction / evidence

Running spec/pact/consumers/message_spec.rb as-is:

[verifier] verification initialized for provider Test Message Producer
1 example, 0 failures

No Verifying a pact between… line — nothing was verified.

Same spec with the correct name ('Test Message Provider'):

Verifying a pact between Test Message Consumer and Test Message Provider
  a customer created message ...

Real verification actually runs.

Impact

rake pact:verify reports success while verifying nothing, so provider-side contract regressions go undetected. fail_if_no_pacts_found defaults to true but does not trip here — the directory is non-empty, the pacts just don't match the provider name, which the FFI treats as "nothing to do."

Proposed fix

Align *_pact_provider names with the generated provider names

Follow-ups:

  • Add verification specs for the plugin pacts (myprovider, matttcpprovider).
  • Consider making a provider-name-with-no-matching-pact fail loudly so future renames can't silently pass.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions