add jvn vignette

Author: marcburri

R/jvn.R

@@ -161,7 +161,7 @@ jvn_nowcast <- function(df,
         "Filter to a single ID first."
       ))
     }
-    df <- vintages_wide(df, names_from = "release")
+    df <- suppressWarnings(vintages_wide(df, names_from = "release"))
     # Handle if vintages_wide returns a list
     if (is.list(df) && !is.data.frame(df)) df <- df[[1]]
   }

R/kk.R

@@ -256,7 +256,7 @@ kk_nowcast <- function(
         "Filter to a single ID first."
       ))
     }
-    df <- vintages_wide(df, names_from = "release")
+    df <- suppressWarnings(vintages_wide(df, names_from = "release"))
     # Handle if vintages_wide returns a list
     if (is.list(df) && !is.data.frame(df)) df <- df[[1]]
   }

_pkgdown.yml

@@ -53,7 +53,8 @@ articles:
   - understanding-revisions
   - revision-analysis
   - efficient-release
-  - nowcasting-revisions
+  - nowcasting-revisions-kk
+  - nowcasting-revisions-jvn
 
 - title: "Why revisions matter"
   contents:

vignettes/nowcasting-revisions-jvn.Rmd

@@ -0,0 +1,357 @@
+---
+title: "Nowcasting revisions using the Jacobs-Van Norden model"
+output: rmarkdown::html_vignette
+bibliography: references.bib
+biblio-style: apalike
+link-citations: true
+vignette: >
+  %\VignetteIndexEntry{nowcasting-revisions-jvn}
+  %\VignetteEngine{knitr::rmarkdown}
+  %\VignetteEncoding{UTF-8}
+---
+
+```{r, include = FALSE}
+knitr::opts_chunk$set(
+  collapse = TRUE,
+  comment = "#>"
+)
+```
+
+Having established that revisions are predictable, we can now apply nowcasting techniques to estimate the current state of the economy. This vignette demonstrates how to implement a nowcasting model using the Jacobs-Van Norden (JVN) framework (@jacobsModelingDataRevisions2011).
+
+## The Jacobs-van Norden Model
+
+The JVN model provides a flexible state-space framework for decomposing data revisions into economically meaningful components: **news** and **noise**. Unlike traditional approaches that treat all revisions as either pure information updates or pure measurement error, the JVN model allows for a mixture of both, providing a more realistic representation of the revision process.
+
+### Key Features
+
+- **News component** ($\nu_t$): Represents genuine new information about the true state of the economy that was not available at the time of the initial release
+- **Noise component** ($\zeta_t$): Represents measurement error or temporary distortions in preliminary estimates that are corrected in subsequent revisions
+- **Spillover effects**: Allow revisions to one vintage to affect revisions to other vintages, capturing the complex dynamics of the revision process
+- **Flexible dynamics**: Accommodates autoregressive behavior in the true underlying series
+
+### Model Structure
+
+The JVN model is represented in state-space form (notation following @durbinTimeSeriesAnalysis2012) with two key equations:
+
+**1. The Observation Equation**
+
+The observation equation links the observed data vintages to the latent state variables:
+
+$$
+y_t = Z \alpha_t
+$$
+
+where $y_t$ contains the different vintages of data available at time $t$, and $\alpha_t$ is the state vector containing the true value, news, and noise components.
+
+**2. The State Equation**
+
+The state equation describes the dynamics of the latent states:
+
+$$
+\alpha_{t+1} = T \alpha_t + R \eta_t
+$$
+
+where $\eta_t \sim N(0, Q)$ represents the structural shocks.
+
+### Example: AR(2) Model with Three Releases
+
+Consider an AR(2) process for the true output growth with three data releases. The complete model can be written as:
+
+$$
+\begin{bmatrix}
+y_{t}^{t+1} \\ 
+y_{t}^{t+2} \\ 
+y_{t}^{t+3} 
+\end{bmatrix}
+= 
+\begin{bmatrix}
+1 & 0 & 1 & 0 & 0 & 1 & 0 & 0 \\
+1 & 0 & 0 & 1 & 0 & 0 & 1 & 0 \\
+1 & 0 & 0 & 0 & 1 & 0 & 0 & 1
+\end{bmatrix}
+\begin{bmatrix}
+\tilde{y}_{t} \\ 
+\tilde{y}_{t-1} \\ 
+\nu_{1t} \\ 
+\nu_{2t} \\
+\nu_{3t} \\
+\zeta_{1t} \\
+\zeta_{2t} \\
+\zeta_{3t}
+\end{bmatrix}
+$$
+
+The state vector evolves according to:
+
+$$
+\begin{bmatrix}
+\tilde{y}_{t} \\ 
+\tilde{y}_{t-1} \\ 
+\nu_{1t} \\ 
+\nu_{2t} \\
+\nu_{3t} \\
+\zeta_{1t} \\
+\zeta_{2t} \\
+\zeta_{3t}
+\end{bmatrix}
+= 
+\begin{bmatrix}
+\rho_1 & \rho_2 & 0 & 0 & 0 & 0 & 0 & 0 \\ 
+1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ 
+0 & 0 & \tau_{\nu 1} & 0 & 0 & 0 & 0 & 0 \\ 
+0 & 0 & 0 & \tau_{\nu 2} & 0 & 0 & 0 & 0 \\
+0 & 0 & 0 & 0 & \tau_{\nu 3} & 0 & 0 & 0 \\
+0 & 0 & 0 & 0 & 0 & \tau_{\zeta 1} & 0 & 0 \\
+0 & 0 & 0 & 0 & 0 & 0 & \tau_{\zeta 2} & 0 \\
+0 & 0 & 0 & 0 & 0 & 0 & 0 & \tau_{\zeta 3}
+\end{bmatrix}
+\begin{bmatrix}
+\tilde{y}_{t-1} \\ 
+\tilde{y}_{t-2} \\ 
+\nu_{1,t-1} \\ 
+\nu_{2,t-1} \\
+\nu_{3,t-1} \\
+\zeta_{1,t-1} \\
+\zeta_{2,t-1} \\
+\zeta_{3,t-1}
+\end{bmatrix}
++ \\
+\begin{bmatrix}
+\sigma _{e} & \sigma _{\nu 1} & \sigma _{\nu 2} & \sigma _{\nu 3} & 0 & 0 & 0 \\ 
+0           &               0 &               0 &               0 & 0 & 0 & 0 \\ 
+0           & -\sigma _{\nu 1} & -\sigma _{\nu 2} & -\sigma _{\nu 3} & 0 & 0 & 0 \\ 
+0           & 0                & -\sigma _{\nu 2} & -\sigma _{\nu 3} & 0 & 0 & 0 \\ 
+0 & 0 & 0 & -\sigma _{\nu 3}  & 0 & 0 & 0 \\ 
+0 & 0 & 0 & 0 & 0 & \sigma _{\zeta 1} & 0 & 0 \\ 
+0 & 0 & 0 & 0 & 0 & 0 & \sigma _{\zeta 2} & 0  \\ 
+0 & 0 & 0 & 0 & 0 & 0 & 0 & \sigma _{\zeta 3}  \\ 
+\end{bmatrix}
+\cdot 
+\begin{bmatrix}
+\eta _{et} \\ 
+\eta _{\nu _{1}t} \\ 
+\eta _{\nu _{2}t} \\ 
+\eta _{\nu _{3}t} \\ 
+\eta _{\zeta _{1}t} \\ 
+\eta _{\zeta _{2}t} \\ 
+\eta _{\zeta _{3}t} \\ 
+\end{bmatrix}
+$$
+
+The error loading matrix $R$ and shock vector $\eta_t$ capture how structural innovations affect each component. The true value $\tilde{y}_t$ is affected by all news shocks (cumulative information), while individual news and noise components receive their own independent shocks.
+
+### Parameters to Estimate
+
+For this example, the model has 15 free parameters:
+
+- **AR coefficients**: $\rho_1, \rho_2$ (dynamics of true value)
+- **Standard deviations**: $\sigma_e$ (AR shock), $\sigma_{\nu 1}, \sigma_{\nu 2}, \sigma_{\nu 3}$ (news), $\sigma_{\zeta 1}, \sigma_{\zeta 2}, \sigma_{\zeta 3}$ (noise)
+- **Spillover parameters**: $\tau_{\nu 1}, \tau_{\nu 2}, \tau_{\nu 3}$ (news persistence), $\tau_{\zeta 1}, \tau_{\zeta 2}, \tau_{\zeta 3}$ (noise persistence)
+
+### Nested Models
+
+The JVN framework is highly flexible and nests several special cases:
+
+- **News-only model**: Set $\sigma_{\zeta j} = 0$ for all $j$ (all revisions are information)
+- **Noise-only model**: Set $\sigma_{\nu j} = 0$ for all $j$ (all revisions are measurement error)
+- **No spillovers**: Set $\tau_{\nu j} = 0$ and $\tau_{\zeta j} = 0$ (revisions are i.i.d.)
+
+## Nowcasting with the JVN Model
+
+We demonstrate nowcasting Euro Area GDP using the JVN model. The procedure follows these steps:
+
+- Identify the efficient release (the first release that is not systematically revised)
+- Estimate the JVN model using Maximum Likelihood
+- Examine model fit and parameters
+
+### Identify Efficient Release
+
+```{r warning = FALSE, message=FALSE}
+library(reviser)
+library(dplyr)
+library(lubridate)
+library(ggplot2)
+
+# Prepare GDP data
+gdp <- reviser::gdp %>%
+  tsbox::ts_pc() %>%
+  dplyr::filter(
+    id == "EA",
+    time >= min(pub_date),
+    time <= as.Date("2020-01-01")
+  ) %>%
+  tidyr::drop_na()
+
+# Get first 15 releases
+df <- get_nth_release(gdp, n = 0:14)
+
+# Get final release (4 years after initial)
+final_release <- get_nth_release(gdp, n = 15)
+
+# Test for efficient release
+efficient_release <- get_first_efficient_release(
+  df,
+  final_release
+)
+
+data <- efficient_release$data
+e <- efficient_release$e
+
+summary(efficient_release)
+```
+
+The efficient release test identifies that $e = `r e`$, meaning the `r e+1`th release is an efficient estimate of the final value.
+
+### Estimate the JVN Model
+
+The `jvn_nowcast()` function estimates the model using Maximum Likelihood Estimation (MLE). You can specify:
+
+- **AR order**: The order of the autoregressive process for the true value
+- **Model components**: Whether to include news, noise, and/or spillovers
+- **Optimization method**: L-BFGS-B (default), two-step, nlminb, and more
+- **Standard error calculation**: Hessian-based (default)
+
+```{r warning = FALSE, message=FALSE}
+# Estimate JVN model with news and noise
+nowcast <- jvn_nowcast(
+  df = data,
+  e = e,
+  ar_order = 1,
+  h = 4,  # 4-period ahead forecast
+  include_news = TRUE,
+  include_noise = TRUE,
+  include_spillovers = FALSE,
+  method = "L-BFGS-B"
+)
+
+```
+
+### Examine Model Fit and Parameter
+
+```{r warning = FALSE, message=FALSE}
+# Model diagnostics
+summary(nowcast)
+```
+
+### Extract State Estimates
+
+The model provides both filtered and smoothed estimates:
+
+- **Filtered estimates**: Use only information available up to time $t$
+- **Smoothed estimates**: Use the full sample (better for historical analysis)
+
+```{r warning = FALSE, message=FALSE}
+# Extract filtered states
+filtered_states <- nowcast$states %>%
+  filter(filter == "filtered", state == "true_lag_0")
+
+# View recent estimates
+tail(filtered_states, 8)
+```
+
+### Visualize Results
+
+```{r warning = FALSE, message=FALSE}
+plot(nowcast)
+
+# Compare news and noise components
+nowcast$states %>%
+  filter(filter == "smoothed", grepl("news|noise", state)) %>%
+  ggplot(aes(x = time, y = estimate, color = state)) +
+  geom_line() +
+  labs(
+    title = "News and Noise Components",
+    x = "Time",
+    y = "Contribution"
+  ) +
+  theme_minimal()
+```
+
+## Advanced Features
+
+### Multi-Start Optimization
+
+For complex models, it's recommended to use multi-start optimization to avoid local optima:
+
+```{r warning = FALSE, message=FALSE, eval=FALSE}
+nowcast_robust <- jvn_nowcast(
+  df = data,
+  e = e,
+  ar_order = 2,
+  include_news = TRUE,
+  include_noise = TRUE,
+  include_spillovers = TRUE,
+  solver_options = list(
+    n_starts = 5,      # Try 5 different starting points
+    trace = 1,         # Show progress
+    maxiter = 2000     # Increase max iterations
+  )
+)
+```
+
+### Model Comparison
+
+Compare different specifications to find the best fit:
+
+```{r warning = FALSE, message=FALSE, eval=FALSE}
+# News-only model
+model_news <- jvn_nowcast(data, e, include_news = TRUE, include_noise = FALSE)
+
+# Noise-only model
+model_noise <- jvn_nowcast(data, e, include_news = FALSE, include_noise = TRUE)
+
+# Full model with spillovers
+model_full <- jvn_nowcast(
+  data, e,
+  include_news = TRUE,
+  include_noise = TRUE,
+  include_spillovers = TRUE
+)
+
+# Compare BIC (lower is better)
+data.frame(
+  Model = c("News Only", "Noise Only", "Full with Spillovers"),
+  BIC = c(model_news$bic, model_noise$bic, model_full$bic),
+  AIC = c(model_news$aic, model_noise$aic, model_full$aic)
+)
+```
+
+### Custom Starting Values
+
+For difficult optimization problems, you can provide custom starting values:
+
+```{r warning = FALSE, message=FALSE, eval=FALSE}
+# Get the number of parameters
+n_params <- nowcast$jvn_model_mat$param_info$n_params
+
+# Provide custom starting values
+custom_starts <- rep(0.1, n_params)
+
+nowcast_custom <- jvn_nowcast(
+  df = data,
+  e = e,
+  solver_options = list(
+    startvals = custom_starts,
+    transform_se = TRUE
+  )
+)
+```
+
+## Interpretation
+
+### News vs Noise
+
+The key insight from the JVN model is the decomposition of revisions:
+
+- **Large news variances** ($\sigma_{\nu j}$): Initial releases understate the true value; revisions contain important information
+- **Large noise variances** ($\sigma_{\zeta j}$): Initial releases are noisy; revisions mainly remove measurement error
+- **AR dynamics** ($\rho_1, \rho_2$): Persistence in the true underlying series
+
+### Forecasting Implications
+
+- If revisions are mainly **news**: Initial releases should be given less weight; wait for revisions
+- If revisions are mainly **noise**: Initial releases already contain most information; revisions are less informative
+- **Spillovers** indicate that information in one revision affects expectations about other revisions
+
+## References