Optimize dev proxy upstream performance (#896)

* Streamline the dev proxy guide to match the ts CLI install

Replace the verbose `cargo run --manifest-path … --target … -- dev proxy`
invocations with the installed `ts` binary (via `cargo install-cli`), so the
guide reads the same way as the CLI guide. Note that `ts dev proxy` is
macOS-only and phrase the install step as "install or update".

Make the quick start a complete first run (`ca install`, then run with
`--rewrite-host` and `--launch chrome`), and make `--rewrite-host` the
recommended default for dev and staging upstreams. The Host-header guidance now
turns on whether the upstream accepts `Host: <FROM>` rather than on the upstream
being Trusted Server Compute.

Correct several stale or wrong examples, verified against the CLI source:

- Firefox `certutil` now uses `-d "sql:…"` and the full CA nickname, matching the
  auto-import the tool performs.
- Add the missing `--connect-timeout` option and note that `[COMMAND]` is `ca`.
- Prefer `--basic-auth-file` over inline `--basic-auth` in the staging example.
- Clarify the CA key is stored outside the repository only by default, and rename
  "First run: CA setup" to "The development CA".

* Address deep-review findings on the dev proxy guide

Keep `--rewrite-host` in the quick start (it is required for upstreams that reject
`Host: <FROM>`), but state its trade-off honestly: against a real Trusted Server
adapter, first-party URLs then render on the upstream host, not the production
domain. Qualify the "first-party URLs always stay on the production domain" claim
accordingly — it holds only when the upstream preserves `X-Forwarded-Host`.

Also:

- Note that Firefox CA auto-import needs `certutil` (`brew install nss`) and
  qualify the troubleshooting row that assumed it is always present.
- State that `--launch` opens only the first mapped hostname while every mapping
  is still proxied.
- Note that connection options (`--rewrite-host`, auth, `--insecure`,
  `--upstream-plaintext`) apply to every mapping, not per-rule.
- Scope the Safari "only while the proxy runs" claim to a clean exit.

* docs: design dev proxy performance optimizations

* docs: tighten dev proxy performance design

* docs: preserve body semantics in retry design

* docs: resolve deep review of proxy performance design

* docs: define early-response pool behavior

* docs: align proxy lease ownership wording

* docs: close final proxy performance design gaps

* docs: define proxy benchmark controls

* docs: plan dev proxy performance optimizations

* Address final dev proxy design review

* Harden dev proxy shutdown design

* Harden dev proxy lifecycle and benchmarks

* Establish dev proxy performance baseline

* Precompute dev proxy upstream identity

* Optimize dev proxy upstream performance

* Harden dev proxy connection lifecycle

* Plan dev proxy review backfill

* Correct dev proxy request outcome metrics

* Use a fresh connection after stale readiness

* Remove rejected HTTP2 origin state

* Cover dev proxy connector configuration

* Backfill dev proxy credential safety tests

* Cover dev proxy streaming failure lifecycles

* Cover forwarding pool isolation and overread

* Backfill adversarial proxy shutdown coverage

* Cover dev proxy pool key isolation

* Complete dev proxy review backfill

* Quiet blind tunnel failures for unmatched hosts

Only unmatched hosts reach the blind tunnel, and real publisher pages
reference many third-party domains that are dead, blackholed, or
DNS-blocked. Logging each failure at warn level drowned out failures on
mapped upstreams, so log them at debug instead. The browser still
receives its 502 and the failure remains visible at debug level.

* Address dev proxy review feedback

Author: aram356

crates/trusted-server-cli/Cargo.toml

@@ -57,6 +57,7 @@ tokio-rustls = { workspace = true }
 webpki-roots = { workspace = true }
 
 [target.'cfg(target_os = "macos")'.dev-dependencies]
+tokio = { workspace = true, features = ["test-util"] }
 x509-parser = { workspace = true }
 
 [target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies]

crates/trusted-server-cli/src/commands/dev/proxy/browser.rs

@@ -738,6 +738,19 @@ fn restore_auto_proxy(
 mod tests {
     use super::*;
     use crate::commands::dev::proxy::rewrite::{Authority, Rule, RuleTable};
+    use crate::commands::dev::proxy::upstream::key::AddressPolicy;
+
+    fn rule(from: &str, to: &str) -> Rule {
+        Rule::new(
+            from.to_string(),
+            Authority::parse(to, false).expect("should parse authority"),
+            false,
+            false,
+            false,
+            AddressPolicy::Dns,
+        )
+        .expect("should build rule")
+    }
 
     #[test]
     fn shell_quote_wraps_and_escapes() {
@@ -826,12 +839,10 @@ mod tests {
 
     #[test]
     fn pac_uses_normalized_connect_address_for_wildcard_bind() {
-        let rules = RuleTable(vec![Rule {
-            from: "www.example-publisher.com".into(),
-            to: Authority::parse("to.edgecompute.app", false).expect("should parse authority"),
-            rewrite_host: false,
-            plaintext: false,
-        }]);
+        let rules = RuleTable(vec![rule(
+            "www.example-publisher.com",
+            "to.edgecompute.app",
+        )]);
         let pac = generate_pac(&rules, "0.0.0.0:18080".parse().expect("addr"));
         assert!(
             pac.contains("PROXY 127.0.0.1:18080"),
@@ -845,12 +856,10 @@ mod tests {
 
     #[test]
     fn pac_proxies_only_https_for_from_hosts() {
-        let rules = RuleTable(vec![Rule {
-            from: "www.example-publisher.com".into(),
-            to: Authority::parse("to.edgecompute.app", false).expect("should parse authority"),
-            rewrite_host: false,
-            plaintext: false,
-        }]);
+        let rules = RuleTable(vec![rule(
+            "www.example-publisher.com",
+            "to.edgecompute.app",
+        )]);
         let pac = generate_pac(
             &rules,
             "127.0.0.1:18080".parse().expect("should parse addr"),

crates/trusted-server-cli/src/commands/dev/proxy/ca.rs

@@ -130,6 +130,8 @@ impl CertAuthority {
     ///
     /// Panics if the leaf-cache mutex is poisoned by a prior panic while held.
     pub fn server_config(&self, host: &str) -> Result<Arc<ServerConfig>, Report<CaError>> {
+        let normalized = host.to_ascii_lowercase();
+        let host = normalized.as_str();
         // Fast path: return a cached config without holding the lock during minting.
         {
             let cache = self
@@ -152,6 +154,19 @@ impl CertAuthority {
         Ok(Arc::clone(entry))
     }
 
+    /// Reports whether a normalized leaf identity is already cached.
+    ///
+    /// # Panics
+    ///
+    /// Panics if the leaf-cache mutex was poisoned by an earlier panic.
+    #[must_use]
+    pub fn is_cached(&self, host: &str) -> bool {
+        self.leaves
+            .lock()
+            .expect("should be able to acquire leaf cache lock")
+            .contains_key(&host.to_ascii_lowercase())
+    }
+
     fn mint(&self, host: &str) -> Result<ServerConfig, Report<CaError>> {
         let leaf_key = KeyPair::generate().change_context(CaError::Generate)?;
 
@@ -305,6 +320,19 @@ mod tests {
         assert!(!Arc::ptr_eq(&a, &c), "different host mints a new config");
     }
 
+    #[test]
+    fn leaf_cache_normalizes_dns_case() {
+        let dir = tempfile::tempdir().expect("should create tempdir");
+        let ca = CertAuthority::load_or_generate(dir.path()).expect("should generate");
+        let lower = ca
+            .server_config("www.example.com")
+            .expect("mint lowercase leaf");
+        let mixed = ca
+            .server_config("WWW.Example.COM")
+            .expect("reuse mixed-case leaf");
+        assert!(Arc::ptr_eq(&lower, &mixed));
+    }
+
     #[test]
     fn mints_leaf_for_ip_literal_host() {
         // An IP-literal host must mint successfully (IP-type SAN, not DNS) — spec §8.3.

crates/trusted-server-cli/src/commands/dev/proxy/config.rs

@@ -6,9 +6,11 @@ use std::path::PathBuf;
 
 use base64::Engine as _;
 use error_stack::{Report, ResultExt as _};
+use hyper::header::HeaderValue;
 
 use super::ProxyArgs;
 use super::rewrite::{Authority, Rule, RuleTable};
+use super::upstream::key::AddressPolicy;
 
 /// Errors from configuration resolution.
 #[derive(Debug, derive_more::Display)]
@@ -52,27 +54,40 @@ pub enum ConfigError {
 impl core::error::Error for ConfigError {}
 
 /// Basic-auth credentials to inject upstream.
-#[derive(Debug, Clone)]
+#[derive(Clone)]
 pub struct BasicAuth {
-    pub user: String,
-    pub pass: String,
+    header: HeaderValue,
 }
 
 impl BasicAuth {
-    /// The `Authorization` header value (`Basic base64(user:pass)`).
+    /// Precomputes a validated `Authorization` header.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`ConfigError::BasicAuth`] if the generated value is not a valid
+    /// HTTP header value.
+    pub fn new(user: &str, pass: &str) -> Result<Self, ConfigError> {
+        let token = base64::engine::general_purpose::STANDARD.encode(format!("{user}:{pass}"));
+        let header =
+            HeaderValue::from_str(&format!("Basic {token}")).map_err(|_| ConfigError::BasicAuth)?;
+        Ok(Self { header })
+    }
+
+    /// The prevalidated `Authorization` header value.
     #[must_use]
-    pub fn header_value(&self) -> String {
-        let token = base64::engine::general_purpose::STANDARD
-            .encode(format!("{}:{}", self.user, self.pass));
-        format!("Basic {token}")
+    pub fn header_value(&self) -> &HeaderValue {
+        &self.header
     }
 
     fn parse(raw: &str) -> Result<Self, ConfigError> {
         let (user, pass) = raw.split_once(':').ok_or(ConfigError::BasicAuth)?;
-        Ok(Self {
-            user: user.to_string(),
-            pass: pass.to_string(),
-        })
+        Self::new(user, pass)
+    }
+}
+
+impl core::fmt::Debug for BasicAuth {
+    fn fmt(&self, formatter: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        formatter.write_str("BasicAuth([REDACTED])")
     }
 }
 
@@ -163,6 +178,7 @@ fn build_rules(args: &ProxyArgs) -> Result<RuleTable, ConfigError> {
             to,
             args.rewrite_host,
             args.upstream_plaintext,
+            args.insecure,
         )?);
     }
     if let (Some(from), Some(to)) = (&args.from, &args.to) {
@@ -171,6 +187,7 @@ fn build_rules(args: &ProxyArgs) -> Result<RuleTable, ConfigError> {
             to,
             args.rewrite_host,
             args.upstream_plaintext,
+            args.insecure,
         )?);
     }
     Ok(RuleTable(rules))
@@ -221,18 +238,22 @@ fn make_rule(
     to: &str,
     rewrite_host: bool,
     plaintext: bool,
+    insecure: bool,
 ) -> Result<Rule, ConfigError> {
     let from = from.to_ascii_lowercase();
     if !is_valid_host(&from) {
         return Err(ConfigError::InvalidFrom { value: from });
     }
     let to = Authority::parse(to, plaintext).map_err(|_| ConfigError::Rule)?;
-    Ok(Rule {
+    Rule::new(
         from,
         to,
         rewrite_host,
         plaintext,
-    })
+        insecure,
+        AddressPolicy::Dns,
+    )
+    .map_err(|_| ConfigError::Rule)
 }
 
 /// Resolves arguments into a [`ResolvedConfig`].
@@ -243,7 +264,7 @@ fn make_rule(
 /// invalid/forbidden listen address, malformed credentials, or an unknown
 /// browser.
 pub fn resolve(args: &ProxyArgs) -> Result<ResolvedConfig, Report<ConfigError>> {
-    let rules = build_rules(args).map_err(Report::from)?;
+    let mut rules = build_rules(args).map_err(Report::from)?;
     if rules.0.is_empty() {
         return Err(Report::new(ConfigError::NoRule));
     }
@@ -281,6 +302,12 @@ pub fn resolve(args: &ProxyArgs) -> Result<ResolvedConfig, Report<ConfigError>>
     let ca_dir = ca_dir(args);
     let resolve = build_resolve(args).map_err(Report::from)?;
 
+    for rule in &mut rules.0 {
+        if let Some(address) = resolve.get(rule.to.host()) {
+            rule.set_address_policy(AddressPolicy::Resolve(*address));
+        }
+    }
+
     // A `--resolve HOST:IP` whose HOST matches no rule's TO host is almost
     // certainly a typo: the pin would silently never apply. Warn rather than
     // error, so a deliberate pin for a host reached indirectly still works.
@@ -322,7 +349,14 @@ fn resolve_basic_auth(args: &ProxyArgs) -> Result<Option<BasicAuth>, ConfigError
 
 #[cfg(test)]
 mod tests {
+    use hyper::header::HeaderValue;
+    use rustls::pki_types::ServerName;
+
     use super::*;
+    use crate::commands::dev::proxy::rewrite::rewrite_for;
+    use crate::commands::dev::proxy::upstream::key::{
+        AddressPolicy, OriginKey, ReferenceIdentity, Transport, VerifyMode,
+    };
 
     fn base_args() -> crate::commands::dev::proxy::ProxyArgs {
         // Construct via clap so defaults match the real surface.
@@ -367,7 +401,11 @@ mod tests {
             .rules
             .first_match("www.example-publisher.com")
             .expect("rule present");
-        assert!(!rule.rewrite_host, "default preserves FROM host");
+        assert_eq!(
+            rewrite_for(rule).host_header,
+            HeaderValue::from_static("www.example-publisher.com"),
+            "default preserves FROM host"
+        );
         assert_eq!(rule.to.host(), "to.edgecompute.app");
     }
 
@@ -377,11 +415,14 @@ mod tests {
         args.map = vec!["www.example-publisher.com=to.edgecompute.app".into()];
         args.rewrite_host = true;
         let cfg = resolve(&args).expect("should resolve");
-        assert!(
-            cfg.rules
-                .first_match("www.example-publisher.com")
-                .expect("rule")
-                .rewrite_host,
+        assert_eq!(
+            rewrite_for(
+                cfg.rules
+                    .first_match("www.example-publisher.com")
+                    .expect("rule")
+            )
+            .host_header,
+            HeaderValue::from_static("to.edgecompute.app"),
             "--rewrite-host sends Host: TO"
         );
     }
@@ -501,15 +542,96 @@ mod tests {
 
     #[test]
     fn basic_auth_header_is_base64() {
-        let auth = BasicAuth {
-            user: "dev".into(),
-            pass: "secret".into(),
-        };
+        let auth = BasicAuth::new("dev", "secret").expect("should build auth");
         assert_eq!(
             auth.header_value(),
-            "Basic ZGV2OnNlY3JldA==",
+            &HeaderValue::from_static("Basic ZGV2OnNlY3JldA=="),
             "Basic base64(user:pass)"
         );
+        let debug = format!("{auth:?}");
+        assert_eq!(debug, "BasicAuth([REDACTED])", "Debug should be redacted");
+        assert!(!debug.contains("dev"), "Debug should not contain the user");
+        assert!(
+            !debug.contains("secret") && !debug.contains("ZGV2OnNlY3JldA=="),
+            "Debug should not contain raw or encoded credentials"
+        );
+    }
+
+    #[test]
+    fn resolve_precomputes_typed_rule_identity_and_headers() {
+        let mut args = base_args();
+        args.map = vec!["www.example.com=TO.Example.com:8443".into()];
+        args.rewrite_host = true;
+        args.insecure = true;
+        args.resolve = vec!["to.example.com:192.0.2.10".into()];
+
+        let cfg = resolve(&args).expect("should resolve");
+        let rule = cfg
+            .rules
+            .first_match("www.example.com")
+            .expect("should find rule");
+        let outcome = rewrite_for(rule);
+
+        assert_eq!(
+            outcome.host_header,
+            HeaderValue::from_static("to.example.com:8443"),
+            "should prevalidate upstream Host"
+        );
+        assert_eq!(
+            outcome.orig_host,
+            HeaderValue::from_static("www.example.com"),
+            "should prevalidate forwarding host"
+        );
+        assert_eq!(
+            outcome.sni,
+            Some(ServerName::try_from("to.example.com").expect("should parse server name")),
+            "should prevalidate normalized SNI"
+        );
+        assert_eq!(
+            rule.origin_key(),
+            &OriginKey::new(
+                Transport::Tls,
+                ReferenceIdentity::dns("to.example.com"),
+                8443,
+                VerifyMode::Insecure,
+                AddressPolicy::Resolve("192.0.2.10".parse().expect("should parse pin")),
+            ),
+            "should precompute the complete transport identity"
+        );
+    }
+
+    #[test]
+    fn resolve_keeps_ip_reference_identities_http1_only() {
+        let mut args = base_args();
+        args.map = vec!["www.example.com=127.0.0.1".into()];
+        args.rewrite_host = true;
+
+        let cfg = resolve(&args).expect("should resolve");
+        let rule = cfg
+            .rules
+            .first_match("www.example.com")
+            .expect("should find rule");
+
+        assert_eq!(
+            rule.origin_key(),
+            &OriginKey::new(
+                Transport::Tls,
+                ReferenceIdentity::ip("127.0.0.1".parse().expect("should parse IP")),
+                443,
+                VerifyMode::Secure,
+                AddressPolicy::Dns,
+            ),
+            "IP identities should validate as IP and remain HTTP/1-only"
+        );
+        assert_eq!(
+            rewrite_for(rule).sni,
+            Some(ServerName::from(
+                "127.0.0.1"
+                    .parse::<IpAddr>()
+                    .expect("should parse IP server name"),
+            )),
+            "TLS should use an IP server name rather than DNS SNI"
+        );
     }
 
     #[test]