Cross-posted from microprediction/skaters#179 — more natural home is probably here,
since precise is the online covariance/correlation package and this is the natural
next rung past second moments.
Worth a look: HOFA / HFA — Higher-order multi-cumulant Factor Analysis, from
"Estimation of Factors Using Higher-Order Multi-Cumulants in Weak Factor Models"
(Peter Cotton, Guanglin Huang, Wanbo Lu), accepted at the Journal of Business &
Economic Statistics.
What it is
PCA recovers latent factors from the covariance matrix, which works when factors are
"strong" (loadings large relative to noise). When factors are weak, the covariance
structure alone may not carry enough signal to identify them. If the underlying factors
are non-Gaussian, though, information about them survives in higher-order cumulants that
PCA never looks at. HOFA constructs a higher-order cumulant matrix and does an
eigenanalysis on that matrix times its own transpose — simpler than the authors' earlier
GMM-type approach with Dries Cornilly and Tim Verdonck. The paper derives the
estimator's asymptotic properties, checks it in simulation, and applies it to FRED-MD,
where the recovered factors improve out-of-sample forecasts of the S&P 500 monthly
equity premium.
Why it's relevant to precise
precise already tracks covariance/correlation online (running_cov, ema_cov,
ledoit_wolf_cov in skaters' skaters.cov, itself "inspired by the precise package").
HOFA is the natural next step past the second moment: what a weak, non-Gaussian latent
factor looks like once you stop stopping at the covariance matrix.
Is there an online version at all?
Plausibly yes, and it decomposes cleanly:
- The inputs — 3rd/4th-order central co-moment tensors across the panel — have
well-known incremental update formulas (the Welford/Pébay generalization of online
variance to higher order and to the multivariate case), exactly the same idea
ema_cov/running_cov already use for the covariance matrix, just carried further.
An EWMA-discounted streaming version of those moment tensors looks very natural.
- The eigenanalysis step itself would stay a snapshot computation done on-demand from
the current moment estimates (same pattern as "online PCA" today: covariance updates
continuously, eigendecomposition happens fresh whenever factors are actually wanted),
not something incrementally updated eigenvector-by-eigenvector unless incremental-SVD
machinery is added on top.
- The real bottleneck is scale, not concept: a dense 3rd/4th-order moment tensor over
n series costs O(n^3)/O(n^4) to store and update, which gets expensive fast for
FRED-MD-sized panels (100+ series). Tensor-sketching or random-projection tricks from
the broader higher-order-ICA / streaming blind-source-separation literature are the
likely fix, though this needs someone to actually check the relevant citations rather
than assume.
Caveats before adopting anything
- HOFA as published is a batch/offline estimator (eigenanalysis of a fixed-sample
cumulant matrix) on a cross-sectional panel — see above for what an online
re-derivation would actually require.
- Paper is newly accepted; no public preprint link on hand yet — track by citation for
now.
Action: read the paper once available, and scope whether a streaming approximation
(EWMA moment tensors + on-demand eigenanalysis, per above) is worth prototyping, or
whether this stays out of scope pending a tractable-at-scale (sketched/projected)
version.
Cross-posted from microprediction/skaters#179 — more natural home is probably here,
since
preciseis the online covariance/correlation package and this is the naturalnext rung past second moments.
Worth a look: HOFA / HFA — Higher-order multi-cumulant Factor Analysis, from
"Estimation of Factors Using Higher-Order Multi-Cumulants in Weak Factor Models"
(Peter Cotton, Guanglin Huang, Wanbo Lu), accepted at the Journal of Business &
Economic Statistics.
What it is
PCA recovers latent factors from the covariance matrix, which works when factors are
"strong" (loadings large relative to noise). When factors are weak, the covariance
structure alone may not carry enough signal to identify them. If the underlying factors
are non-Gaussian, though, information about them survives in higher-order cumulants that
PCA never looks at. HOFA constructs a higher-order cumulant matrix and does an
eigenanalysis on that matrix times its own transpose — simpler than the authors' earlier
GMM-type approach with Dries Cornilly and Tim Verdonck. The paper derives the
estimator's asymptotic properties, checks it in simulation, and applies it to FRED-MD,
where the recovered factors improve out-of-sample forecasts of the S&P 500 monthly
equity premium.
Why it's relevant to precise
precisealready tracks covariance/correlation online (running_cov,ema_cov,ledoit_wolf_covin skaters'skaters.cov, itself "inspired by the precise package").HOFA is the natural next step past the second moment: what a weak, non-Gaussian latent
factor looks like once you stop stopping at the covariance matrix.
Is there an online version at all?
Plausibly yes, and it decomposes cleanly:
well-known incremental update formulas (the Welford/Pébay generalization of online
variance to higher order and to the multivariate case), exactly the same idea
ema_cov/running_covalready use for the covariance matrix, just carried further.An EWMA-discounted streaming version of those moment tensors looks very natural.
the current moment estimates (same pattern as "online PCA" today: covariance updates
continuously, eigendecomposition happens fresh whenever factors are actually wanted),
not something incrementally updated eigenvector-by-eigenvector unless incremental-SVD
machinery is added on top.
n series costs O(n^3)/O(n^4) to store and update, which gets expensive fast for
FRED-MD-sized panels (100+ series). Tensor-sketching or random-projection tricks from
the broader higher-order-ICA / streaming blind-source-separation literature are the
likely fix, though this needs someone to actually check the relevant citations rather
than assume.
Caveats before adopting anything
cumulant matrix) on a cross-sectional panel — see above for what an online
re-derivation would actually require.
now.
Action: read the paper once available, and scope whether a streaming approximation
(EWMA moment tensors + on-demand eigenanalysis, per above) is worth prototyping, or
whether this stays out of scope pending a tractable-at-scale (sketched/projected)
version.