Add OAuth 2.1 client support for MCP authorization flow

## Motivation and Context

The MCP Authorization spec (2025-11-25) requires clients to support
Protected Resource Metadata discovery (RFC 9728), Authorization
Server Metadata discovery (RFC 8414), Dynamic Client Registration,
and OAuth 2.1 Authorization Code grant with PKCE (S256). The Ruby SDK
previously had no client-side OAuth support: the HTTP transport raised
`RequestHandlerError` on a 401 with no recovery, and 21 `auth/*` scenarios
were listed as known failures in `conformance/expected_failures.yml`.

This aligns the Ruby SDK with the Python and TypeScript SDKs,
both of which expose a pluggable OAuth client provider that drives
the full discovery + DCR + authorization + token exchange flow.
`MCP::Client::OAuth::Provider` mirrors that surface: callers supply
`client_metadata`, a `redirect_uri`, and pluggable `redirect_handler`
/ `callback_handler` hooks plus a token/client information storage.
`MCP::Client::OAuth::InMemoryStorage` is provided as a default.

`MCP::Client::HTTP` accepts a new optional `oauth:` keyword. When present,
the transport attaches `Authorization: Bearer` to every request,
and on a 401 parses `WWW-Authenticate` for the `resource_metadata` URL
and `scope` challenge, drives the discovery + DCR + authorization code flow
through the provider, and retries the failed request once with the acquired token.

Spec compliance details:

- The `WWW-Authenticate` parser handles multiple challenges
  (e.g. `Basic ..., Bearer ...` or `Bearer ..., DPoP ...`), extracting only
  the parameters that belong to the Bearer challenge.
- The PRM `resource` value is required to cover the MCP server URL
  (same scheme/host/port, with the PRM path as a prefix of the server path)
  before the flow continues, preventing redirection of issued tokens to a different audience.
- Both authorization and token requests always include a `resource`
  parameter (RFC 8707), defaulting to a canonicalized form of
  the MCP server URL when PRM does not advertise one.
- `client_secret_basic`, `client_secret_post`, and `none` token endpoint
  authentication methods are supported. When pre-registered client information
  specifies no `token_endpoint_auth_method`, the transport defaults to
  `client_secret_basic` per RFC 6749 §2.3.1 an dthe TS/Python SDK convention.
  Pre-registered client credentials supplied via `Storage#save_client_information`
  with either string or symbol keys are honored, skipping DCR.
- Scope resolution follows the spec order: the `WWW-Authenticate` challenge scope wins first,
  then PRM `scopes_supported`, and `Provider#scope` is the last-resort fallback only
  if neither is supplied (the provider-supplied scope must not pre-empt server-advertised scope).
- Token endpoint error responses (RFC 6749 §5.2) are parsed: a 4x with
  `error: "invalid_grant"` raises `Flow::InvalidGrantError`,
  so `MCP::Client::HTTP` can discard the stored refresh token only when
  the AS truly says it is dead. Generic `AuthorizationError`
  (5xx, network failures, other error codes) leaves the refresh token
  intact, so a transient AS outage does not force interactive reauth
  on the next attempt.
- The OAuth `state` parameter is compared via
  `OpenSSL.fixed_length_secure_compare` (with a `bytesize` guard) to
  prevent timing-based discovery of the expected value.
- The 401 retry guard is a request-scoped local variable, so an MCP server
  that keeps returning 401 is surfaced as an error after one
  retry rather than triggering an infinite re-authorization loop.

## How Has This Been Tested?

Unit tests cover PKCE generation, multi-challenge `WWW-Authenticate` parsing,
PRM and Authorization Server metadata URL builders, the full Flow orchestrator
(success path, state mismatch, missing PKCE support, skipping DCR for both string
and symbol-keyed preregistered client information, `client_secret_basic`
token endpoint authentication, PRM `resource` mismatch rejection, fallback to
the server URL when PRM omits `resource`, scope resolution preferring PRM
`scopes_supported` over `Provider#scope`, default `client_secret_basic` for
confidential clients without an explicit `token_endpoint_auth_method`,
and the new `InvalidGrantError` vs generic `AuthorizationError` split for
token endpoint rejections), and the HTTP transport behavior
(401 recovery + retry, no-provider fallthrough, no infinite loop on repeated 401,
no infinite loop on flow failure, refresh-token retention on transient 5xx from
the token endpoint).

Conformance: `bundle exec rake conformance` newly passes 16 `auth/*`
scenarios beyond the existing baseline, including
`auth/metadata-default`, `metadata-var{1,2,3}`,
`scope-from-www-authenticate`, `scope-from-scopes-supported`,
`scope-omitted-when-undefined`, `scope-retry-limit`,
`token-endpoint-auth-{basic,post,none}`, `pre-registration`,
`resource-mismatch`, and `offline-access-not-supported`. Remaining
`auth/*` scenarios stay in `expected_failures.yml`.
`bundle exec rake test rubocop` is clean.

## Breaking Changes

None. `MCP::Client::HTTP#initialize` accepts a new optional `oauth:`
keyword that defaults to `nil`, preserving existing behavior. When unset,
the transport raises `MCP::Client::RequestHandlerError` on a 401 exactly as before.

Author: koic

README.md

@@ -1885,6 +1885,113 @@ client.tools # will make the call using Bearer auth
 
 You can add any custom headers needed for your authentication scheme, or for any other purpose. The client will include these headers on every request.
 
+#### OAuth 2.1 Authorization
+
+When an MCP server enforces the [MCP Authorization spec](https://modelcontextprotocol.io/specification/2025-11-25/basic/authorization),
+pass an `MCP::Client::OAuth::Provider` to the transport instead of a static `Authorization` header. The transport will:
+
+- Send `Authorization: Bearer <access_token>` on every request when a token is available.
+- On a `401 Unauthorized`, parse the `WWW-Authenticate` header, discover the authorization server (Protected Resource Metadata + RFC 8414 Authorization Server Metadata),
+  perform Dynamic Client Registration if needed, run the OAuth 2.1 Authorization Code flow with PKCE (S256), and retry the failed request with the acquired token.
+- On subsequent 401s with a saved `refresh_token`, exchange it at the token endpoint before falling back to the full interactive flow (RFC 6749 Section 6).
+
+```ruby
+require "mcp"
+
+provider = MCP::Client::OAuth::Provider.new(
+  client_metadata: {
+    client_name: "My MCP App",
+    redirect_uris: ["http://localhost:3030/callback"],
+    grant_types: ["authorization_code", "refresh_token"],
+    response_types: ["code"],
+    token_endpoint_auth_method: "none",
+  },
+  redirect_uri: "http://localhost:3030/callback",
+  redirect_handler: ->(authorization_url) {
+    # Send the user to the authorization URL - typically `Launchy.open(authorization_url)`
+    # or a manual `puts authorization_url` in CLI tools.
+  },
+  callback_handler: -> {
+    # Capture the redirect (for example, by running a small HTTP listener on
+    # `redirect_uri`) and return [code, state] from the query string.
+  },
+)
+
+transport = MCP::Client::HTTP.new(
+  url: "https://api.example.com/mcp",
+  oauth: provider,
+)
+client = MCP::Client.new(transport: transport)
+client.connect # `initialize` is sent here; if the server replies 401 the OAuth flow runs and the handshake is retried with the acquired token
+client.tools
+```
+
+Required keyword arguments to `Provider.new`:
+
+- `client_metadata`: Hash sent to the authorization server's Dynamic Client Registration endpoint. Must include `redirect_uris`, `grant_types`, `response_types`,
+  `token_endpoint_auth_method`. `redirect_uri` (below) must appear in this list, otherwise the constructor raises `Provider::UnregisteredRedirectURIError`.
+- `redirect_uri`: String. Must use HTTPS or be a loopback URL (`localhost`, `127.0.0.0/8`, `::1`); other values raise `Provider::InsecureRedirectURIError`.
+- `redirect_handler`: Callable invoked with the fully-built authorization `URI`. Typically opens the user's browser.
+- `callback_handler`: Callable that returns `[code, state]` after the user is redirected back to `redirect_uri`.
+
+Optional keyword arguments:
+
+- `scope`: Space-separated scopes to request when the server's `WWW-Authenticate` does not specify one.
+- `storage`: Object responding to `tokens`, `save_tokens(t)`, `client_information`, `save_client_information(info)`. Defaults to `MCP::Client::OAuth::InMemoryStorage`,
+  which keeps credentials in process memory only.
+
+To persist credentials across restarts, supply your own storage:
+
+```ruby
+class FileTokenStorage
+  def initialize(path)
+    @path = path
+  end
+
+  def tokens
+    read["tokens"]
+  end
+
+  def save_tokens(value)
+    write("tokens" => value)
+  end
+
+  def client_information
+    read["client"]
+  end
+
+  def save_client_information(value)
+    write("client" => value)
+  end
+
+  private
+
+  def read
+    File.exist?(@path) ? JSON.parse(File.read(@path)) : {}
+  end
+
+  def write(updates)
+    File.write(@path, JSON.dump(read.merge(updates)))
+  end
+end
+
+provider = MCP::Client::OAuth::Provider.new(
+  # ... required keywords ...
+  storage: FileTokenStorage.new(File.expand_path("~/.config/my-app/oauth.json")),
+)
+```
+
+##### Communication Security
+
+When `oauth:` is set, the MCP transport URL and every OAuth-facing URL (PRM, Authorization Server metadata, `authorization_endpoint`, `token_endpoint`, `registration_endpoint`,
+`redirect_uri`) must use HTTPS or a loopback host. Non-loopback `http://` URLs are rejected at the SDK boundary so a bearer token is never sent over plain HTTP to a remote host.
+
+The transport also snapshots the canonicalized origin, path, and query string of the MCP URL at `initialize` time and re-checks them on every outgoing request through
+a Faraday middleware that runs after any user-supplied customizer. That means any URL swap raises `MCP::Client::HTTP::InsecureURLError` before the request reaches the adapter,
+whether the swap was triggered by
+`instance_variable_set(:@url, ...)`, by a Faraday customizer rewriting `url_prefix`, or by a custom middleware rewriting `env.url` (including just `env.url.query`) at request time,
+and whether the new URL is `http://` *or* `https://` to a different host or tenant.
+
 #### Customizing the Faraday Connection
 
 You can pass a block to `MCP::Client::HTTP.new` to customize the underlying Faraday connection.

conformance/client.rb

@@ -8,6 +8,8 @@
 # The scenario name is read from the MCP_CONFORMANCE_SCENARIO environment variable,
 # which is set automatically by the conformance test runner.
 
+require "faraday"
+require "json"
 require_relative "../lib/mcp"
 
 scenario = ENV["MCP_CONFORMANCE_SCENARIO"]
@@ -17,7 +19,82 @@
   abort("Usage: MCP_CONFORMANCE_SCENARIO=<scenario> ruby conformance/client.rb <server-url>")
 end
 
-transport = MCP::Client::HTTP.new(url: server_url)
+# The conformance harness optionally injects scenario-specific data via
+# the `MCP_CONFORMANCE_CONTEXT` environment variable as a JSON document. The shape is
+# defined by the harness, not the MCP spec, and has varied between versions:
+#
+# - Newer (`@modelcontextprotocol/conformance` >= 0.x): scenario fields are
+#   spread at the top level alongside `name`, e.g.
+#   `{"name":"auth/pre-registration","client_id":"...","client_secret":"..."}`.
+# - Older: a nested `context` object: `{"name":"...","context":{...}}`.
+#
+# Both shapes are accepted so the client conforms to whichever harness version
+# the developer has on hand.
+def conformance_context
+  raw = ENV["MCP_CONFORMANCE_CONTEXT"]
+  return {} if raw.nil? || raw.empty?
+
+  parsed = JSON.parse(raw)
+  return {} unless parsed.is_a?(Hash)
+
+  if parsed["context"].is_a?(Hash)
+    parsed["context"]
+  else
+    parsed.reject { |key, _| key == "name" }
+  end
+rescue JSON::ParserError
+  {}
+end
+
+# Builds an OAuth provider that drives the authorization code + PKCE + DCR flow
+# non-interactively against the conformance test's auth server. The conformance
+# `/authorize` endpoint redirects synchronously to `redirect_uri` with
+# `code=test-auth-code`, so we follow it manually instead of opening a browser.
+def build_oauth_provider(context)
+  callback_holder = {}
+  redirect_uri = "http://localhost:0/callback"
+
+  redirect_handler = ->(authorization_url) do
+    response = Faraday.new.get(authorization_url) do |req|
+      req.options.params_encoder = nil
+    end
+    location = response.headers["location"] || response.headers["Location"]
+    abort("Authorization request did not redirect: #{response.status}.") unless location
+
+    callback_holder[:url] = URI.parse(location)
+  end
+
+  callback_handler = -> do
+    query = URI.decode_www_form(callback_holder.fetch(:url).query).to_h
+    [query["code"], query["state"]]
+  end
+
+  storage = MCP::Client::OAuth::InMemoryStorage.new
+  if context["client_id"]
+    storage.save_client_information(
+      "client_id" => context["client_id"],
+      "client_secret" => context["client_secret"],
+      "token_endpoint_auth_method" => context["token_endpoint_auth_method"] || "client_secret_basic",
+    )
+  end
+
+  MCP::Client::OAuth::Provider.new(
+    client_metadata: {
+      client_name: "ruby-sdk-conformance-client",
+      redirect_uris: [redirect_uri],
+      grant_types: ["authorization_code"],
+      response_types: ["code"],
+      token_endpoint_auth_method: "none",
+    },
+    redirect_uri: redirect_uri,
+    redirect_handler: redirect_handler,
+    callback_handler: callback_handler,
+    storage: storage,
+  )
+end
+
+oauth = scenario.start_with?("auth/") ? build_oauth_provider(conformance_context) : nil
+transport = MCP::Client::HTTP.new(url: server_url, oauth: oauth)
 client = MCP::Client.new(transport: transport)
 client.connect(client_info: { name: "ruby-sdk-conformance-client", version: MCP::VERSION })
 
@@ -29,6 +106,11 @@
   add_numbers = tools.find { |t| t.name == "add_numbers" }
   abort("Tool add_numbers not found") unless add_numbers
   client.call_tool(tool: add_numbers, arguments: { a: 1, b: 2 })
+when %r|\Aauth/|
+  # Auth-only scenarios: the protocol-level checks (PRM/AS metadata, DCR, PKCE, token usage)
+  # are observed by the conformance server during `connect` and the subsequent request below.
+  # Listing tools forces a second authenticated MCP request so the bearer token usage check fires.
+  client.tools
 else
   abort("Unknown or unsupported scenario: #{scenario}")
 end

conformance/expected_failures.yml

@@ -4,25 +4,12 @@ client:
   - sse-retry
   # TODO: Elicitation not implemented in Ruby client.
   - elicitation-sep1034-client-defaults
-  # TODO: OAuth/auth not implemented in Ruby client.
-  - auth/metadata-default
-  - auth/metadata-var1
-  - auth/metadata-var2
-  - auth/metadata-var3
+  # TODO: Remaining OAuth/auth scenarios not yet implemented in Ruby client.
   - auth/basic-cimd
-  - auth/scope-from-www-authenticate
-  - auth/scope-from-scopes-supported
-  - auth/scope-omitted-when-undefined
   - auth/scope-step-up
-  - auth/scope-retry-limit
-  - auth/token-endpoint-auth-basic
-  - auth/token-endpoint-auth-post
-  - auth/token-endpoint-auth-none
-  - auth/pre-registration
   - auth/2025-03-26-oauth-metadata-backcompat
   - auth/2025-03-26-oauth-endpoint-fallback
   - auth/client-credentials-jwt
   - auth/client-credentials-basic
   - auth/cross-app-access-complete-flow
   - auth/offline-access-scope
-  - auth/offline-access-not-supported

lib/mcp/client.rb

@@ -1,5 +1,6 @@
 # frozen_string_literal: true
 
+require_relative "client/oauth"
 require_relative "client/stdio"
 require_relative "client/http"
 require_relative "client/paginated_result"