Missing Jacobians for physical inputs (CRTM v3.x)

[BenjaminTJohnson]

Description

Following the philosophy that every physical input to the CRTM must have an associated Jacobian, a codebase audit was performed on the current CRTMv3 compile chain. While the model successfully produces Jacobians for primary state variables (Temperature, Absorbers, and basic Surface/Cloud properties), several physical parameters are currently treated as static during the K-matrix calculation.

This issue tracks the identification and required implementation of these missing sensitivities to ensure full differentiability of the model for data assimilation and sensitivity studies.

---

Criteria for Jacobians

1. Included: Any real-valued physical input.
2. Excluded: Classifiers, indices, or boolean flags (e.g., Land_Type, Soil_Type, Algorithm_Id, etc.).
3. Note: there may be derived variables that we're excluding for now, this issue focuses on Jacobians with a physical input to CRTM.

---

1. Atmospheric Jacobians

Current sensitivities are restricted to Temperature and Absorber amounts. The following physical parameters lack Jacobians in the Atmosphere_K structure:

• Level_Pressure & Pressure
  These are used to calculate predictors and path lengths. While typically treated as the coordinate system, any physical variation in the pressure levels themselves (e.g., in sigma-pressure systems) does not currently propagate through the K-matrix.

• Height
  Used in geometry calculations and active sensor (radar) reflectivity. No adjoint is returned for height variations.

• Relative_Humidity (Fix Missing Aerosol Scattering Jacobians and Restore Backscatter Logic #285 resolves this)
  In CRTM_AerosolScatter.f90, RH_AD is calculated locally for interpolation in GOCART/NAAPS models, but it is not consistently propagated back to the Atmosphere_K structure in CRTM_K_Matrix_Module.f90.

---

2. Cloud and Aerosol Jacobians

While Water_Content and Effective_Radius are well-supported, the following are missing:

• Cloud%Effective_Variance (see Fix Missing Aerosol Scattering Jacobians and Restore Backscatter Logic #285, resolves this)
  Defined in CRTM_Cloud_type, but the adjoint is not calculated in CRTM_Compute_CloudScatter_AD. If scattering lookup tables (LUTs) depend on size distribution variance, this Jacobian is required.

• Cloud%Water_Density
  Adjoint calculation is currently inconsistent between MW and IR paths.

---

3. Surface Jacobians (High Priority)

The surface routines contain the most significant gaps, particularly for Land, Snow, and Ice models where Compute_MW_Land_SfcOptics_AD and similar routines are either stubs or ignore structural parameters.

Land Surface

• Soil_Moisture_Content
  Critical for microwave land emissivity; missing Jacobian.

• Canopy_Water_Content (see Canopy_Water_Content input is unused #279 )
  Missing Jacobian

• Vegetation_Fraction
  Missing Jacobian.

• LAI (Leaf Area Index) (see Canopy_Water_Content input is unused #279 )
  Missing Jacobian.

• Soil_Temperature
  Currently often aliased to Land_Temperature; needs independent sensitivity if used as a separate physical layer.

Snow Surface

• Snow_Depth
  Physical input for snow emissivity models; missing Jacobian.

• Snow_Density
  Missing Jacobian.

• Snow_Grain_Size
  Missing Jacobian.

Ice Surface

• Ice_Thickness
  Missing Jacobian.

• Ice_Density
  Missing Jacobian.

• Ice_Roughness
  Missing Jacobian.

---

4. Technical Summary of Missing Jacobian Links

The following files/subroutines require updates to propagate the missing adjoint values:

1. src/CRTM_K_Matrix_Module.f90
   Update the profile_solution loop to ensure all components of Atmosphere_K and Surface_K are being populated from the lower-level AD calls.

2. src/SfcOptics/CRTM_SfcOptics.f90
   Update the accumulation logic in CRTM_Compute_SfcOptics_AD to include the specific snow/ice/land parameters listed above.

3. src/AtmScatter/CRTM_CloudScatter.f90
   Update CRTM_Compute_CloudScatter_AD to handle Effective_Variance.

4. src/SfcOptics/MW_Water/CRTM_MW_Water_SfcOptics.f90
   Reference for implementation. This is one of the few routines correctly propagating non-temperature sensitivities like Salinity and Wind_Speed.

[BenjaminTJohnson]

Just going to treat this issue like an "Epic". Will work through missing Jacobians on a case-by-case basis.

[BenjaminTJohnson]

PR #282 addresses Canopy + LAI.

[BenjaminTJohnson]

Defect fixed in the analytic MW-land Jacobians: +Infinity at Soil_Moisture_Content = 0

Follow-up on the analytic MW-land emissivity Jacobians (9358caa): Soil_Diel's analytic d(esm)/d(vmc) contains a beta·vmc^(beta−1) term, and for soil types with beta < 1 — types 1 (COARSE, the common default) and 9, both frac_sand = 0.92 → beta ≈ 0.9996 — this evaluates 0^(negative) = +Infinity at Soil_Moisture_Content = 0, a legal input.

Because the forward call site in CRTM_MW_Land_SfcOptics always passes the optional derivative arguments, every forward run over land with SMC = 0 raised an IEEE divide-by-zero (aborting -ffpe-trap/DEBUG builds; forward emissivity values themselves unaffected), and the SMC Jacobian came back Inf/NaN.

Fixed on feature/btj_REL-3.2.0 (78aac3d): the one-sided derivative at vmc = 0 is genuinely unbounded, so the boundary is treated as a zero-derivative region — the same convention as the existing emissivity/input clip handling. Verified: all 9 soil types now return finite (zero) dE/dmv at SMC = 0, and the interior analytic derivative is unchanged (matches central FD to 4+ digits at SMC = 0.2). The test_Land_Jacobian unit test uses SMC = 0.2 and did not cover the boundary; worth adding an SMC = 0 case when the Jacobian coverage is next extended.