30-dif-in-dif.Rmd

# Difference-in-Differences {#sec-difference-in-differences}

[Difference-in-Differences](#sec-difference-in-differences) (DID) is a widely used causal inference method for estimating the effect of policy interventions or exogenous shocks when randomized experiments are not feasible. The key idea behind DID is to compare changes in outcomes over time between treated and control groups, under the assumption that, absent treatment, both groups would have followed parallel trends.

The method is well suited to business analytics because transaction logs, panel dashboards, and quarterly filings already furnish the required data. Its identifying assumptions are easy to communicate, and it controls for economy-wide shocks without heavy modeling.

Practical Advantages of DID

-   Intuitive visuals: A two-line plot can immediately signal whether pre-treatment trends look parallel.
-   Light data demands: Only two time points and a clear treatment onset are needed.
-   Extendable: Regression frameworks accommodate staggered treatments, multiple periods, or continuous exposures.
-   Transparent: Assumptions and identifying variation are easy to communicate to executives, regulators, or reviewers.

<!-- [List of packages](https://github.com/lnsongxf/DiD-1) -->

DID analysis can go beyond simple treatment effects by exploring causal mechanisms using mediation and moderation analyses:

-   [Mediation Under DiD]: Examines how intermediate variables (e.g., consumer sentiment, brand perception) mediate the treatment effect [@habel2021variable].
-   [Moderation] Analysis: Studies how treatment effects vary across different groups (e.g., high vs. low brand loyalty) [@goldfarb2011online].

The remainder of this chapter develops DID estimation, diagnoses its assumptions, and shows how to defend findings against the standard concerns raised by reviewers and other audiences. The exposition leads with intuition and applied cases before introducing the corresponding formalism.

## Where DID has been put to work

The published record of DID studies is enormous, but it is worth pausing on a handful of representative applications because the *kinds* of questions DID has answered tell you a great deal about the kinds of question it can answer well, namely, questions in which a sharp policy or competitive event hits one group while leaving a comparable group untouched, with the outcomes of both observable before and after.

In marketing and business, the canonical questions concern how consumers and firms respond to interventions that arrive as discrete events. Several studies trace the impact of advertising and content on consumer behaviour: @liaukonyte2015television show how TV ads ripple into online shopping, @wang2018border use geographic discontinuities at state borders to disentangle the effect of political ad source and tone on turnout, and @datta2018changing track how adopting a music-streaming service reshapes total consumption. A second cluster studies how firms react to digital and regulatory shocks. @janakiraman2018effect trace how customer spending shifts after a publicly announced data breach; @israeli2018online ask whether digital monitoring tightens enforcement of minimum-advertised-price policies; @ramani2019effects examine how Indian firms responded to the 1991 FDI liberalization; @pattabhiramaiah2019paywalls quantify the readership cost of newspaper paywalls; @akca2020value study how online aggregators reshape airline-ticket sales; @lim2020competitive ask whether nutritional labels nudge competing brands toward healthier formulations; @guo2020let examine how payment-disclosure laws reach into physician prescribing; @he2022market exploit an Amazon policy change to measure the effect of fake reviews on sales and ratings; and @peukert2022regulatory assess how GDPR rewired website usage and online business models. The common ingredient across all of these is a clean event date and a credible comparison group, without both, the design loses its bite.

DID has been at least as central in economics, where it underpins much of modern policy evaluation. @rosenzweig2000natural's review of natural experiments in development economics maps the design's reach in that subfield; @angrist2001instrumental connect DID to instrumental-variable thinking by showing how natural experiments can serve both purposes; and @fuchs2016natural extend the logic to macroeconomic policy analysis, where unit heterogeneity is severe and the parallel-trends assumption demands particular care.

------------------------------------------------------------------------

## Visualization {#sec-visualization-did}

Before diving into estimation, it is always wise to (i) **confirm the treatment pattern** and (ii) **eyeball the outcomes**.

The `panelView` package offers quick heatmaps and outcome traces that make these checks straightforward.

### Data check

```{r}
library(panelView)
library(fixest)
library(tidyverse)
base_stagg <- fixest::base_stagg |>
    # treatment status
    dplyr::mutate(treat_stat = dplyr::if_else(time_to_treatment < 0, 0, 1)) |> 
    select(id, year, treat_stat, y)

head(base_stagg)
```

### Treatment Assignment Heatmap

Figure \@ref(fig:did-treatment-assignment-heatmap) shows the heatmap of treatment status for each unit over 10 years.

```{r did-treatment-assignment-heatmap, fig.cap='Treatment assignment over time by unit.', fig.alt='Heatmap showing treatment status by unit over 10 years. The y-axis lists individual units, and the x-axis marks years. Units transition from light blue (under control) to dark blue (under treatment) at different years, illustrating staggered treatment adoption. A legend at the bottom labels the two status colors.'}
panelView::panelview(
    y ~ treat_stat,
    data = base_stagg,
    index = c("id", "year"),
    xlab = "Year",
    ylab = "Unit",
    display.all = F,
    gridOff = T,
    by.timing = T
)
```

The diagonal "step" confirms that **not all units adopt at once**. This would be perfect for a staggered-DiD design. Horizontal segments without a color change indicate units that never adopt.

Alternatively, specifying the outcome and treatment status will also return the exact figure (Figure \@ref(fig:did-treatment-assignment-heatmap-alt))

```{r did-treatment-assignment-heatmap-alt, fig.cap='Staggered treatment timing across units.', fig.alt='Heatmap of treatment status over 10 years. The x-axis shows years, and the y-axis lists individual units. Each unit transitions from light blue (under control) to dark blue (under treatment) at different points in time, forming a downward diagonal boundary that reflects staggered adoption. A legend identifies the two treatment states.'}
# alternatively specification
panelView::panelview(
    Y = "y",
    D = "treat_stat",
    data = base_stagg,
    index = c("id", "year"),
    xlab = "Year",
    ylab = "Unit",
    display.all = F,
    gridOff = T,
    by.timing = T
)
```

### Raw Outcome Trajectories

Figure \@ref(fig:did-treatment-status-overtime) shows the trajectories of different cohorts over time.

```{r did-treatment-status-overtime, fig.cap='Raw panel data by treatment status over time.', fig.alt='Line plot showing individual outcome trajectories over time. Gray lines represent control units, orange lines show treated units before treatment, and red lines represent treated units after treatment. The y-axis measures outcome y, and the x-axis spans 10 years. Red lines tend to diverge upward or downward after year 5, indicating possible treatment effects.'}
# Average outcomes for each cohort
panelView::panelview(
    data = base_stagg, 
    Y = "y",
    D = "treat_stat",
    index = c("id", "year"),
    by.timing = T,
    display.all = F,
    type = "outcome", 
    by.cohort = T
)
```

If the red segments diverge immediately after treatment while the orange segments blend with gray beforehand, the visual evidence is supportive of a treatment effect and parallel pre-trends.

### Event-time Averages

A more focused diagnostic is to plot the **average outcome in event time** (years relative to first treatment) (Figure \@ref(fig:did-event-time-averages)).

```{r did-event-time-averages, fig.cap="Event-time averages of the outcome relative to treatment onset.", fig.alt='Event-study line plot showing mean outcome over event time. The horizontal axis runs from several years before to after treatment, with a dashed vertical line at zero marking treatment onset. A solid blue line traces the average outcome, and a light-blue ribbon around it depicts SE confidence bands. The viewer can compare flat pre-treatment values to any jump or slope change after zero to gauge treatment effects.'}
base_stagg |>
    group_by(event_time = year - min(year[treat_stat == 1])) |>
    summarise(y_mean = mean(y),
              se     = sd(y) / sqrt(n())) |>
    ggplot(aes(event_time, y_mean)) +
    geom_line(color = "#377eb8", linewidth = 1) +
    geom_ribbon(aes(ymin = y_mean - se, ymax = y_mean + se),
                fill = "#377eb8",
                alpha = .2) +
    geom_vline(xintercept = 0, linetype = "dashed") +
    labs(x = "Years relative to treatment",
         y = "Mean outcome (y)",
         title = "Event-time plot: do outcomes change at treatment onset?") +
    theme_minimal()

```

A flat pre-trend (negative event times) and a noticeable jump at event-time 0 support the identifying assumptions for staggered DiD.

## Simple Difference-in-Differences {#sec-simple-difference-in-differences}

DID first emerged as a tool for [natural experiments](#sec-natural-experiments), settings where policy shocks or geographic quirks mimic random assignment. Its scope has since broadened: marketing A/B roll-outs, corporate ESG mandates, and the staggered release of a mobile-app feature now routinely call on DID for credible impact estimates.

At its computational heart lies the [Fixed Effects Estimator], which sweeps out any time-invariant heterogeneity across units and any shocks common to all periods, leaving the residual variation that identifies the treatment effect.

DID exploits *inter-temporal variation* between groups in **two** complementary ways to address omitted variable bias:

-   Cross-sectional comparison: Compares treated and control units *at the same point in time*, canceling bias from shocks that hit both groups equally (e.g., nationwide inflation). This helps avoid omitted variable bias due to common trends.
-   Time-series comparison: Tracks the same unit over time, purging bias from any fixed, unit-specific traits (e.g., a chain's brand equity, a region's climate). This helps mitigate omitted variable bias due to cross-sectional heterogeneity.

By taking the *difference of differences*, we simultaneously:

1.  **Remove common trends** that could confound a simple cross-sectional comparison.
2.  **Eliminate unit-specific constants** that would spoil a pure time-series analysis.

------------------------------------------------------------------------

### Basic Setup of DID

Consider a simple setting in Table \@ref(tab:did-potential-outcomes-simple) with:

-   **Treatment Group** ($D_i = 1$)
-   **Control Group** ($D_i = 0$)
-   **Pre-Treatment Period** ($T = 0$)
-   **Post-Treatment Period** ($T = 1$)

|                     | **After Treatment (**$T = 1$**)** | **Before Treatment (**$T = 0$**)** |
|---------------------|-----------------------------------|------------------------------------|
| Treated ($D_i = 1$) | $E[Y_{1i}(1)|D_i = 1]$            | $E[Y_{0i}(0)|D_i = 1]$             |
| Control ($D_i = 0$) | $E[Y_{0i}(1)|D_i = 0]$            | $E[Y_{0i}(0)|D_i = 0]$             |

Table: (\#tab:did-potential-outcomes-simple) Potential outcomes by treatment status and time.

The **fundamental challenge**: We cannot observe $E[Y_{0i}(1)|D_i = 1]$ (i.e., the **counterfactual outcome** for the treated group had they not received treatment).

------------------------------------------------------------------------

DID estimates the [Average Treatment Effect on the Treated] using the following formula:

$$
\begin{aligned}
E[Y_1(1) - Y_0(1) | D = 1] &= \{E[Y(1)|D = 1] - E[Y(1)|D = 0] \} \\
&- \{E[Y(0)|D = 1] - E[Y(0)|D = 0] \}
\end{aligned}
$$

::: rmdnote
What parallel trends really says. The identifying assumption is a statement about the *untreated potential outcome* of the treated group, an object we never observe post-treatment:

$$
E[Y(0)_{t=1} - Y(0)_{t=0} \mid D = 1] = E[Y(0)_{t=1} - Y(0)_{t=0} \mid D = 0].
$$

That is, had the treated group not been treated, its average outcome would have evolved in parallel with the control group's. This is intrinsically counterfactual and therefore untestable in the post-treatment period.

What pre-trend plots and placebo tests check is whether the *observed* pre-treatment trends were parallel. A clean pre-trend is supportive evidence for, but not proof of, the identifying assumption. @roth2022pretest shows that conventional pre-trend tests often have low power, so "failing to reject" is weak evidence at best. Modern practice couples pre-trend diagnostics with formal sensitivity analysis (e.g., @rambachan2023more; see Section \@ref(sec-modern-concerns-in-did)).
:::

This formulation differences out time-invariant unobserved factors, assuming the [parallel trends assumption](#prior-parallel-trends-test) holds.

-   For the treated group, we isolate the difference between being treated and not being treated.
-   If the control group would have experienced a different trajectory, the DID estimate may be biased.
-   Since we cannot observe treatment variation in the control group, we cannot infer the treatment effect for this group.

```{r}
# Load required libraries
library(dplyr)
library(ggplot2)
set.seed(1)

# Simulated dataset for illustration
data <- data.frame(
  time = rep(c(0, 1), each = 50),  # Pre (0) and Post (1)
  treated = rep(c(0, 1), times = 50), # Control (0) and Treated (1)
  error = rnorm(100)
)

# Generate outcome variable
data$outcome <-
    5 + 3 * data$treated + 2 * data$time + 
    4 * data$treated * data$time + data$error

# Compute averages for 2x2 table
table_means <- data %>%
  group_by(treated, time) %>%
  summarize(mean_outcome = mean(outcome), .groups = "drop") %>%
  mutate(
    group = paste0(ifelse(treated == 1, "Treated", "Control"), ", ", 
                   ifelse(time == 1, "Post", "Pre"))
  )

# Display the 2x2 table
table_2x2 <- table_means %>%
  select(group, mean_outcome) %>%
  tidyr::spread(key = group, value = mean_outcome)

print("2x2 Table of Mean Outcomes:")
print(table_2x2)

# Calculate Diff-in-Diff manually

# Treated, Post
Y11 <- table_means$mean_outcome[table_means$group == "Treated, Post"]  

# Treated, Pre
Y10 <- table_means$mean_outcome[table_means$group == "Treated, Pre"]   

# Control, Post
Y01 <- table_means$mean_outcome[table_means$group == "Control, Post"]  

# Control, Pre
Y00 <- table_means$mean_outcome[table_means$group == "Control, Pre"]   

diff_in_diff_formula <- (Y11 - Y10) - (Y01 - Y00)

# Estimate DID using OLS
model <- lm(outcome ~ treated * time, data = data)
ols_estimate <- coef(model)["treated:time"]

# Print results
results <- data.frame(
  Method = c("Diff-in-Diff Formula", "OLS Estimate"),
  Estimate = c(diff_in_diff_formula, ols_estimate)
)

print("Comparison of DID Estimates:")
print(results)
```

Figure \@ref(fig:did-simple-viz) shows a simple visualization of the DID in practice.

```{r did-simple-viz, fig.cap='DiD visualization of treated and control group changes pre- and post-intervention.', fig.alt='Scatter plot with two time points labeled 0 (pre) and 1 (post) on the x-axis and outcome values on the y-axis. The control group is shown in blue and the treated group in red. Both groups increase over time, but the treated group shows a larger rise. Dotted lines connect pre- and post-intervention points for each group, visually illustrating the DiD estimate. A legend distinguishes control and treated groups.'}
# Visualization
ggplot(data,
       aes(
           x = as.factor(time),
           y = outcome,
           color = as.factor(treated),
           group = treated
       )) +
    stat_summary(fun = mean, geom = "point", size = 3) +
    stat_summary(fun = mean,
                 geom = "line",
                 linetype = "dashed") +
    labs(
        title = "Difference-in-Differences Visualization",
        x = "Time (0 = Pre, 1 = Post)",
        y = "Outcome",
        color = "Group"
    ) +
    scale_color_manual(labels = c("Control", "Treated"),
                       values = c("blue", "red")) +
    causalverse::ama_theme()
```

|              | Control (0)        | Treated (1)         |
|--------------|--------------------|---------------------|
| **Pre (0)**  | $\bar{Y}_{00} = 5$ | $\bar{Y}_{10} = 8$  |
| **Post (1)** | $\bar{Y}_{01} = 7$ | $\bar{Y}_{11} = 14$ |

Table: (\#tab:did-group-time-average-outcomes) DiD table of group-time average outcomes.

Table \@ref(tab:did-group-time-average-outcomes) organizes the mean outcomes into four cells:

1.  Control Group, Pre-period ($\bar{Y}_{00}$): Mean outcome for the control group before the intervention.

2.  Control Group, Post-period ($\bar{Y}_{01}$): Mean outcome for the control group after the intervention.

3.  Treated Group, Pre-period ($\bar{Y}_{10}$): Mean outcome for the treated group before the intervention.

4.  Treated Group, Post-period ($\bar{Y}_{11}$): Mean outcome for the treated group after the intervention.

The DID treatment effect calculated from the simple formula of averages is identical to the estimate from an OLS regression with an interaction term.

The treatment effect is calculated as:

$\text{DID} = (\bar{Y}_{11} - \bar{Y}_{10}) - (\bar{Y}_{01} - \bar{Y}_{00})$

Compute manually:

$(\bar{Y}_{11} - \bar{Y}_{10}) - (\bar{Y}_{01} - \bar{Y}_{00})$

Use OLS regression:

$Y_{it} = \beta_0 + \beta_1 \text{treated}_i + \beta_2 \text{time}_t + \beta_3 (\text{treated}_i \cdot \text{time}_t) + \epsilon_{it}$

Using the simulated table:

$\text{DID} = (14 - 8) - (7 - 5) = 6 - 2 = 4$

This matches the **interaction term coefficient** ($\beta_3 = 4$) from the OLS regression.

Both methods give the same result!

------------------------------------------------------------------------

### Extensions of DID

#### DID with More Than Two Groups or Time Periods

DID can be extended to **multiple treatments, multiple controls**, and more than two periods:

$$
Y_{igt} = \alpha_g + \gamma_t + \beta I_{gt} + \delta X_{igt} + \epsilon_{igt}
$$

where:

-   $\alpha_g$ = Group-Specific Fixed Effects (e.g., firm, region).

-   $\gamma_t$ = Time-Specific Fixed Effects (e.g., year, quarter).

-   $\beta$ = DID Effect.

-   $I_{gt}$ = Interaction Terms (Treatment × Post-Treatment).

-   $\delta X_{igt}$ = Additional Covariates.

This is known as the [Two-Way Fixed Effects DID](#sec-two-way-fixed-effects) model. However, TWFE performs poorly under staggered treatment adoption, where different groups receive treatment at different times.

------------------------------------------------------------------------

#### Examining Long-Term Effects (Dynamic DID)

To examine the dynamic treatment effects (that are not under rollout/staggered design), we can create a centered time variable (Table \@ref(tab:did-event-time-coding)).

+------------------------+---------------------------------------------------------+
| Centered Time Variable | Interpretation                                          |
+========================+=========================================================+
| $t = -2$               | Two periods before treatment                            |
+------------------------+---------------------------------------------------------+
| $t = -1$               | One period before treatment                             |
+------------------------+---------------------------------------------------------+
| $t = 0$                | Last pre-treatment period right before treatment period |
|                        |                                                         |
|                        | (Baseline/Reference Group)                              |
+------------------------+---------------------------------------------------------+
| $t = 1$                | Treatment period                                        |
+------------------------+---------------------------------------------------------+
| $t = 2$                | One period after treatment                              |
+------------------------+---------------------------------------------------------+

Table: (\#tab:did-event-time-coding) Event-time coding around treatment.

##### Dynamic Treatment Model Specification

By interacting this factor variable, we can examine the dynamic effect of treatment (i.e., whether it's fading or intensifying). We index event time by $k = t - g_i$, where $g_i$ is the period when unit $i$ is first treated, so $k = 0$ is the treatment period and $k = -1$ is the period immediately before. Following the modern convention [@sun2021estimating; @callaway2021difference] we drop $k = -1$ as the reference period:

$$
\begin{aligned}
Y &= \alpha_0 + \alpha_1 Group + \alpha_2 Time  \\
&+ \sum_{k = -T_1}^{-2} \beta_k\, Treatment_k \\
&+ \sum_{k = 0}^{T_2} \beta_k\, Treatment_k + \varepsilon
\end{aligned}
$$

where:

-   $\beta_{-1}$ (the last pre-treatment period) is the reference group and is dropped from the model.

-   $T_1$ = number of pre-treatment periods retained (so leads run from $-T_1$ to $-2$).

-   $T_2$ = number of post-treatment periods retained (so lags run from $0$ to $T_2$).

-   Treatment coefficients ($\beta_k$) measure the effect at event time $k$ relative to $k = -1$.

##### Key Observations

-   Pre-treatment coefficients should be close to zero ($\beta_{-T_1}, \dots, \beta_{-1} \approx 0$), ensuring no pre-trend bias.

-   Post-treatment coefficients should be significantly different from zero ($\beta_1, \dots, \beta_{T_2} \neq 0$), measuring the treatment effect over time.

-   Higher standard errors with more interactions: Including too many lags can reduce precision.

------------------------------------------------------------------------

#### DID on Relationships, Not Just Levels

While DID is most commonly applied to examine treatment effects on outcome levels, it can also be used to estimate how treatment affects the **relationship between variables**. This approach treats estimated coefficients from first-stage regressions as outcomes in a second-stage DID analysis.

Standard DID examines whether treatment changes the level of an outcome $Y_{it}$. However, researchers may be interested in whether treatment changes how $Y$ responds to some predictor $X$, that is, whether treatment affects the coefficient $\beta$ in the relationship:

$$ Y_{it} = \alpha + \beta X_{it} + \epsilon_{it} $$

This requires a two-stage approach where regression coefficients themselves become the unit of analysis.

##### Two-Stage Estimation Procedure

**Stage 1: Estimate Group-Period-Specific Relationships**

For each combination of group $g$ and time period $t$, estimate:

$$ Y_{igt} = \alpha_{gt} + \beta_{gt} X_{igt} + \epsilon_{igt} $$

This yields a set of estimated coefficients $\{\hat{\beta}_{gt}\}$, where each $\hat{\beta}_{gt}$ captures the relationship between $X$ and $Y$ for group $g$ in period $t$.

**Stage 2: Apply DID to the Estimated Coefficients**

Treat the estimated coefficients $\hat{\beta}_{gt}$ as the outcome variable in a standard DID framework:

$$ \hat{\beta}_{gt} = \alpha_0 + \alpha_1 Treated_g + \alpha_2 Post_t + \delta (Treated_g \times Post_t) + u_{gt} $$

where:

-   $\hat{\beta}_{gt}$ = Estimated coefficient from Stage 1 (the "outcome").
-   $Treated_g$ = Indicator for treatment group.
-   $Post_t$ = Indicator for post-treatment period.
-   $\delta$ = **DID estimate of treatment effect on the relationship**.

The coefficient $\delta$ measures whether the relationship between $X$ and $Y$ changed differentially for the treated group after treatment.

The DID estimator can be expressed as:

$$ \begin{aligned}
\hat{\delta} &= (\hat{\beta}_{Treated}^{Post} - \hat{\beta}_{Treated}^{Pre}) - (\hat{\beta}_{Control}^{Post} - \hat{\beta}_{Control}^{Pre}) \\
&= \text{Change in relationship for treated} - \text{Change in relationship for control}
\end{aligned} $$

This captures the **causal effect of treatment on the structural parameter** $\beta$, controlling for secular trends that affect both groups.

##### Example: Price Sensitivity Before and After a Policy Change

Suppose we want to test whether a consumer protection law changes how price affects demand.

**Stage 1:** For each state $s$ and year $t$, estimate:

$$ \log(Quantity_{ist}) = \alpha_{st} + \beta_{st} \log(Price_{ist}) + \epsilon_{ist} $$

where $i$ indexes individual transactions. This gives us state-year specific price elasticities $\{\hat{\beta}_{st}\}$.

**Stage 2:** Use these elasticities as outcomes in a DID model:

$$ \hat{\beta}_{st} = \alpha_0 + \alpha_1 Treated_s + \alpha_2 Post_t + \delta (Treated_s \times Post_t) + u_{st} $$

If $\delta < 0$, the policy made consumers more price-sensitive (more elastic demand) in treated states. If $\delta > 0$, consumers became less price-sensitive.

##### Standard Error Correction

A critical issue is that Stage 2 uses estimated coefficients $\hat{\beta}_{gt}$ as the dependent variable, which introduces **generated regressor** problems. The standard errors from Stage 2 are incorrect because they don't account for estimation uncertainty in $\hat{\beta}_{gt}$.

**Solutions:**

1.  **Bootstrapping:** Resample at the individual level, re-estimate both stages, and calculate standard errors from the bootstrap distribution.

2.  **Weighted Least Squares:** Weight Stage 2 observations by the inverse of the variance of $\hat{\beta}_{gt}$:

$$ w_{gt} = \frac{1}{\text{Var}(\hat{\beta}_{gt})} = \frac{1}{SE(\hat{\beta}_{gt})^2} $$

This gives more weight to precisely estimated relationships.

3.  **Stacked Regression:** Pool all individual observations and include group-period fixed effects interacted with $X$:

$$ Y_{igt} = \sum_{g,t} \beta_{gt} (I_{gt} \times X_{igt}) + \text{controls} + \epsilon_{igt} $$

Then test whether $\{\beta_{gt}\}$ follow a DID pattern using an F-test or linear combinations.

##### Dynamic Effects on Relationships

The two-stage approach can be extended to examine how relationships evolve over time:

**Stage 1:** Estimate period-specific coefficients:

$$ Y_{igt} = \alpha_{gt} + \beta_{gt} X_{igt} + \epsilon_{igt} $$

**Stage 2:** Event study specification:

$$ \hat{\beta}_{gt} = \alpha_g + \gamma_t + \sum_{k \neq -1} \delta_k (Treated_g \times Period_{t=k}) + u_{gt} $$

where $Period_{t=k}$ are indicators for time relative to treatment (with $k=-1$ as the reference period).

**Key Observations:**

-   Pre-treatment coefficients ($\delta_k$ for $k < -1$) should be near zero, indicating parallel trends in the relationship.
-   Post-treatment coefficients ($\delta_k$ for $k \geq 0$) show how the relationship evolves after treatment.
-   This reveals whether effects on relationships are immediate, delayed, or fade over time.

##### Applications

**1. Advertising Effectiveness:**

Does a competitor's market entry change advertising elasticity?

-   Stage 1: Estimate $\frac{\partial \log(Sales)}{\partial \log(Advertising)}$ for each market-period.
-   Stage 2: DID on these elasticities comparing markets with/without competitor entry.

**2. Labor Supply Elasticity:**

Does a tax reform change labor supply responsiveness to wages?

-   Stage 1: Estimate $\frac{\partial Hours}{\partial Wage}$ for each region-year.
-   Stage 2: DID comparing regions with different tax policy changes.

**3. Educational Returns:**

Does a curriculum reform change the returns to study time?

-   Stage 1: Estimate $\frac{\partial GPA}{\partial StudyHours}$ for each school-year.
-   Stage 2: DID comparing schools that adopted vs. didn't adopt the reform.

**4. Platform Network Effects:**

Does a platform algorithm change affect network externalities?

-   Stage 1: Estimate $\frac{\partial UserValue}{\partial NetworkSize}$ for each market-quarter.
-   Stage 2: DID around the algorithm change.

##### Advantages and Limitations

**Advantages:**

1.  **Causal inference on mechanisms:** Identifies how treatment changes behavioral responses, not just outcomes.
2.  **Flexibility:** Can be applied to any estimable relationship (elasticities, marginal effects, coefficients).
3.  **Policy-relevant:** Directly tests whether policies alter structural parameters of interest.

**Limitations:**

1.  **Two-stage uncertainty:** Standard errors require careful correction for generated regressors.
2.  **Data requirements:** Needs sufficient observations within each group-period to estimate first-stage relationships precisely.
3.  **Parallel trends in relationships:** Requires that relationships (not just levels) would have trended similarly absent treatment, a stronger assumption.
4.  **Aggregation:** Loses individual-level variation when collapsing to group-period coefficients.

##### Relationship to Heterogeneous Treatment Effects

DID on relationships is conceptually related to, but distinct from, heterogeneous treatment effects. A three-way interaction $(X \times Treated \times Post)$ estimates whether treatment effects vary **across levels of** $X$. In contrast, DID on relationships estimates whether **the effect of** $X$ on $Y$ changes due to treatment, a fundamentally different question about structural parameters rather than treatment heterogeneity.

This two-stage approach transforms DID into a tool for testing **structural change hypotheses**, enabling researchers to make causal claims about how treatments alter the fundamental relationships governing economic, social, or behavioral systems.

------------------------------------------------------------------------

### Goals of DID

1.  **Pre-Treatment Coefficients Should Be Insignificant**
    -   Ensure that $\beta_{-T_1}, \dots, \beta_{-1} = 0$ (similar to a [Placebo Test](#sec-placebo-test-did)).
2.  **Post-Treatment Coefficients Should Be Significant**
    -   Verify that $\beta_1, \dots, \beta_{T_2} \neq 0$.
    -   Examine whether the trend in post-treatment coefficients is increasing or decreasing over time.

------------------------------------------------------------------------

```{r}
library(tidyverse)
library(fixest)

od <- causaldata::organ_donations %>%
    
    # Treatment variable
    dplyr::mutate(California = State == 'California') %>%
    # centered time variable
    dplyr::mutate(center_time = as.factor(Quarter_Num - 3))  
# where 3 is the reference period precedes the treatment period

class(od$California)
class(od$State)

cali <- feols(Rate ~ i(center_time, California, ref = 0) |
                  State + center_time,
              data = od)

etable(cali)
```

Figure \@ref(fig:did-estimated-effect-on-rate-overtime) shows the fixed effects estimates over time.

```{r did-estimated-effect-on-rate-overtime, fig.cap='Estimated effect on rate over time.', fig.alt='Line plot of treatment effect estimates over time centered at the intervention point (time 0). The y-axis represents the effect on rate with confidence intervals, and the x-axis shows relative time periods from −2 to +3.'}
iplot(cali, pt.join = T)
```

Figure \@ref(fig:did-interaction-effect-on-rate) shows the same plot with a different plotting function.

```{r did-interaction-effect-on-rate, fig.cap='Interaction effect on rate.', fig.alt='Line plot of estimated interaction effects between California and relative time periods from −2 to 3, excluding time 0. The y-axis shows effect estimates with vertical 95% confidence intervals. Pre-intervention estimates hover around zero; post-intervention estimates are negative, indicating a decline in the rate attributable to California-specific policy changes.'}
coefplot(cali)
```

## Empirical Research Walkthrough

### Example: The Unintended Consequences of "Ban the Box" Policies

@doleac2020unintended examine the unintended effects of "Ban the Box" (BTB) policies, which prevent employers from asking about criminal records during the hiring process. The intended goal of BTB was to increase job access for individuals with criminal records. However, the study found that employers, unable to observe criminal history, resorted to statistical discrimination based on race, leading to unintended negative consequences.

Three Types of "Ban the Box" Policies:

1.  Public employers only
2.  Private employers with government contracts
3.  All employers

Identification Strategy

-   If any county within a Metropolitan Statistical Area (MSA) adopts BTB, the entire MSA is considered treated.
-   If a state passes a law banning BTB, then all counties in that state are treated.

------------------------------------------------------------------------

The [basic DiD model](#sec-simple-difference-in-differences) is:

$$
Y_{it} = \beta_0 + \beta_1 \text{Post}_t + \beta_2 \text{Treat}_i + \beta_3 (\text{Post}_t \times \text{Treat}_i) + \epsilon_{it}
$$

where:

-   $Y_{it}$ = employment outcome for individual $i$ at time $t$
-   $\text{Post}_t$ = indicator for post-treatment period
-   $\text{Treat}_i$ = indicator for treated MSAs
-   $\beta_3$ = the DiD coefficient, capturing the effect of BTB on employment
-   $\epsilon_{it}$ = error term

**Limitations**: If different locations adopt BTB at different times, this model is not valid due to staggered treatment timing.

------------------------------------------------------------------------

For settings where different MSAs adopt BTB at different times, we use a **staggered DiD** approach:

$$
\begin{aligned} 
E_{imrt} &= \alpha + \beta_1 BTB_{imt} W_{imt} + \beta_2 BTB_{mt} + \beta_3 BTB_{mt} H_{imt} \\
&+ \delta_m + D_{imt} \beta_5 + \lambda_{rt} + \delta_m \times f(t) \beta_7 + e_{imrt} 
\end{aligned}
$$

where:

-   $i$ = individual, $m$ = MSA, $r$ = region (e.g., Midwest, South), $t$ = year
-   $W$ = White; $B$ = Black; $H$ = Hispanic
-   $BTB_{imt}$ = Ban the Box policy indicator
-   $\delta_m$ = MSA fixed effect
-   $D_{imt}$ = individual-level controls
-   $\lambda_{rt}$ = region-by-time fixed effect
-   $\delta_m \times f(t)$ = linear time trend within MSA

#### Fixed Effects Considerations

-   Including $\lambda_r$ and $\lambda_t$ separately gives broader fixed effects.

-   Using $\lambda_{rt}$ provides more granular controls for regional time trends.

------------------------------------------------------------------------

To estimate the effects for Black men specifically, the model simplifies to:

$$
E_{imrt} = \alpha + BTB_{mt} \beta_1 + \delta_m + D_{imt} \beta_5 + \lambda_{rt} + (\delta_m \times f(t)) \beta_7 + e_{imrt}
$$

------------------------------------------------------------------------

To check for pre-trends and dynamic effects, we estimate:

$$
\begin{aligned} 
E_{imrt} &= \alpha + BTB_{m (t - 3)} \theta_1 + BTB_{m (t - 2)} \theta_2 + BTB_{m (t - 1)} \theta_3 \\
&+ BTB_{mt} \theta_4 + BTB_{m (t + 1)} \theta_5 + BTB_{m (t + 2)} \theta_6 + BTB_{m (t + 3)} \theta_7 \\
&+ \delta_m + D_{imt} \beta_5 + \lambda_{r} + (\delta_m \times f(t)) \beta_7 + e_{imrt}
\end{aligned}
$$

Key points:

-   Leave out $BTB_{m (t - 1)} \theta_3$ as the reference category (to avoid perfect collinearity).
-   If $\theta_2$ is significantly different from $\theta_3$, it suggests pre-trend issues, which could indicate anticipatory effects before BTB implementation.

> Substantively, @shoag2021ban show that Ban-the-box policies increased employment in high-crime neighborhoods by up to 4%, especially in the public sector and low-wage jobs. This is the first nationwide evidence that such laws improve job access for areas with many ex-offenders.

------------------------------------------------------------------------

### Example: Minimum Wage and Employment

@card1993minimum famously studied the effect of an increase in the minimum wage on employment, challenging the traditional economic view that higher wages reduce employment.

-   [Philipp Leppert](https://rpubs.com/phle/r_tutorial_difference_in_differences) provides an R-based replication.
-   Original datasets are available at [David Card's website](https://davidcard.berkeley.edu/data_sets.html).

Setting

-   Treatment group: New Jersey (NJ), which increased its minimum wage.
-   Control group: Pennsylvania (PA), which did not change its minimum wage.
-   Outcome variable: Employment levels in fast-food restaurants.

The study used a Difference-in-Differences approach to estimate the impact (Table \@ref(tab:did-estimator-illustration-state-level)).

|           | State | After (Post) | Before (Pre) | Difference        |
|-----------|-------|--------------|--------------|-------------------|
| Treatment | NJ    | A            | B            | A - B             |
| Control   | PA    | C            | D            | C - D             |
|           |       | A - C        | B - D        | (A - B) - (C - D) |

Table: (\#tab:did-estimator-illustration-state-level) DiD estimator illustration using a state-level example.

where:

-   $A - B$ captures the treatment effect plus general time trends.
-   $C - D$ captures only the general time trends.
-   $(A - B) - (C - D)$ isolates the causal effect of the minimum wage increase.

For the DiD estimator to be valid, the following conditions must hold:

1.  [Parallel Trends Assumption](#prior-parallel-trends-test)
    -   The employment trends in NJ and PA would have been the same in the absence of the policy change.
    -   Pre-treatment employment trends should be similar between the two states.
2.  **No "Switchers"**
    -   The policy must not induce restaurants to switch locations between NJ and PA (e.g., a restaurant relocating across the border).
3.  **PA as a Valid Counterfactual**
    -   PA represents what NJ would have looked like had it not changed the minimum wage.
    -   The study focuses on bordering counties to increase comparability.

------------------------------------------------------------------------

The main regression specification is:

$$
Y_{jt} = \beta_0 + NJ_j \beta_1 + POST_t \beta_2 + (NJ_j \times POST_t)\beta_3+ X_{jt}\beta_4 + \epsilon_{jt}
$$

where:

-   $Y_{jt}$ = Employment in restaurant $j$ at time $t$
-   $NJ_j$ = 1 if restaurant is in NJ, 0 if in PA
-   $POST_t$ = 1 if post-policy period, 0 if pre-policy
-   $(NJ_j \times POST_t)$ = **DiD interaction term**, capturing the causal effect of NJ's minimum wage increase
-   $X_{jt}$ = Additional controls (optional)
-   $\epsilon_{jt}$ = Error term

Notes on Model Specification

-   $\beta_3$ (DiD coefficient) is the key parameter of interest, representing the causal impact of the policy.

-   $\beta_4$ (controls $X_{jt}$) is not necessary for unbiasedness but improves efficiency.

-   If we difference out the pre-period ($\Delta Y_{jt} = Y_{j,Post} - Y_{j,Pre}$), we can simplify the model:

    $$
    \Delta Y_{jt} = \alpha + NJ_j \beta_1 + \epsilon_{jt}
    $$

    Here, we no longer need $\beta_2$ for the post-treatment period.

------------------------------------------------------------------------

An alternative specification uses high-wage NJ restaurants as a control group, arguing that they were not affected by the minimum wage increase. However:

-   This approach eliminates cross-state differences, but
-   It may be harder to interpret causality, as the control group is not entirely untreated.

------------------------------------------------------------------------

A common misconception in DiD is that treatment and control groups must have the same baseline levels of the dependent variable (e.g., employment levels). However:

-   DiD only requires parallel trends, meaning the slopes of employment changes should be the same pre-treatment.
-   If pre-treatment trends diverge, this threatens validity.
-   If post-treatment trends converge, it may suggest policy effects rather than pre-trend violations.

Is Parallel Trends a Necessary or Sufficient Condition?

-   Not sufficient: Even if pre-trends are parallel, other confounders could affect results.
-   Not necessary: Parallel trends may emerge only after treatment, depending on behavioral responses.

Thus, we cannot prove DiD is valid. We can only present evidence that supports the assumptions.

------------------------------------------------------------------------

### Example: The Effects of Grade Policies on Major Choice

@butcher2014effects investigate how grading policies influence students' major choices. The central theory is that grading standards vary by discipline, which affects students' decisions.

The pattern that the highest-achieving students often concentrate in the hard sciences can be rationalized by two mechanisms.

1.  **Grading Practices Differ Across Majors**
    -   In STEM fields, grading is often stricter, meaning professors are less likely to give students the benefit of the doubt.
    -   In contrast, softer disciplines (e.g., humanities) may have more lenient grading, raising perceived utility from the major.
2.  **Labor Market Incentives**
    -   Degrees with lower market value (e.g., humanities) may compensate by offering a less demanding academic experience.
    -   STEM degrees tend to be more rigorous but provide higher job market returns.

------------------------------------------------------------------------

To examine how grades influence major selection, the study first estimates an OLS model:

$$
E_{ij} = \beta_0 + X_i \beta_1 + G_j \beta_2 + \epsilon_{ij}
$$

where:

-   $E_{ij}$ = Indicator for whether student $i$ chooses major $j$.
-   $X_i$ = Student-level attributes (e.g., SAT scores, demographics).
-   $G_j$ = Average grade in major $j$.
-   $\beta_2$ = Key coefficient, capturing how grading standards influence major choice.

Potential Biases in $\hat{\beta}_2$:

-   Negative Bias: 
    -   Departments with lower enrollment rates may offer higher grades to attract students.
    -   This endogenous response leads to a downward bias in the OLS estimate.
-   Positive Bias: 
    -   STEM majors attract the best students, so their grades would naturally be higher if ability were controlled.
    -   If ability is not fully accounted for, $\hat{\beta}_2$ may be upward biased.

------------------------------------------------------------------------

To address potential endogeneity in OLS, the study uses a difference-in-differences approach:

$$
Y_{idt} = \beta_0 + POST_t \beta_1 + Treat_d \beta_2 + (POST_t \times Treat_d)\beta_3 + X_{idt} + \epsilon_{idt}
$$

where:

-   $Y_{idt}$ = Average grade in department $d$ at time $t$ for student $i$.
-   $POST_t$ = 1 if post-policy period, 0 otherwise.
-   $Treat_d$ = 1 if department is treated (i.e., grade policy change), 0 otherwise.
-   $(POST_t \times Treat_d)$ = **DiD interaction term**, capturing the causal effect of grade policy changes on major choice.
-   $X_{idt}$ = Additional student controls.

------------------------------------------------------------------------

| Group             | Intercept ($\beta_0$) | Treatment ($\beta_2$) | Post ($\beta_1$) | Interaction ($\beta_3$) |
|-------------------|-----------------------|-----------------------|------------------|-------------------------|
| **Treated, Pre**  | 1                     | 1                     | 0                | 0                       |
| **Treated, Post** | 1                     | 1                     | 1                | 1                       |
| **Control, Pre**  | 1                     | 0                     | 0                | 0                       |
| **Control, Post** | 1                     | 0                     | 1                | 0                       |

Table: (\#tab:did-group-level-design-matrix) Group-level design matrix for difference-in-differences.

Table \@ref(tab:did-group-level-design-matrix) shows how we can think about the design matrix for DID.

-   The average pre-period outcome for the control group is given by $\beta_0$.
-   The key coefficient of interest is $\beta_3$, which captures the difference in the post-treatment effect between treated and control groups.

------------------------------------------------------------------------

A more flexible specification includes fixed effects:

$$
Y_{idt} = \alpha_0 + (POST_t \times Treat_d) \alpha_1 + \theta_d + \delta_t + X_{idt} + u_{idt}
$$

where:

-   $\theta_d$ = Department fixed effects (absorbing $Treat_d$).
-   $\delta_t$ = Time fixed effects (absorbing $POST_t$).
-   $\alpha_1$ = Effect of policy change (equivalent to $\beta_3$ in the simpler model).

Why Use Fixed Effects?

-   More flexible specification: 
    -   Instead of assuming a uniform treatment effect across groups, this model allows for department-specific differences ($\theta_d$) and time-specific shocks ($\delta_t$).
-   Higher degrees of freedom: 
    -   Fixed effects absorb variation that would otherwise be attributed to $POST_t$ and $Treat_d$, making the estimation more efficient.

Interpretation of Results

-   If $\alpha_1 > 0$, then the policy **increased** grades in treated departments.
-   If $\alpha_1 < 0$, then the policy **decreased** grades in treated departments.

------------------------------------------------------------------------

## One Difference

The regression formula is as follows @liaukonyte2023frontiers:

$$
y_{ut} = \beta \text{Post}_t + \gamma_u + \gamma_w(t) + \gamma_l + \gamma_g(u)p(t) + \epsilon_{ut}
$$

where

-   $y_{ut}$: Outcome of interest for unit u in time t.
-   $\text{Post}_t$: Dummy variable representing a specific post-event period.
-   $\beta$: Coefficient measuring the average change in the outcome after the event relative to the pre-period.
-   $\gamma_u$: Fixed effects for each unit.
-   $\gamma_w(t)$: Time-specific fixed effects to account for periodic variations.
-   $\gamma_l$: Dummy variable for a specific significant period (e.g., a major event change).
-   $\gamma_g(u)p(t)$: Group x period fixed effects for flexible trends that may vary across different categories (e.g., geographical regions) and periods.
-   $\epsilon_{ut}$: Error term.

This model can be used to analyze the impact of an event on the outcome of interest while controlling for various fixed effects and time-specific variations, but using units themselves pre-treatment as controls.

------------------------------------------------------------------------

## Two-Way Fixed Effects {#sec-two-way-fixed-effects}

A generalization of the [Difference-in-Differences](#sec-difference-in-differences) model is the two-way fixed effects (TWFE) model, which accounts for **multiple groups** and **multiple time periods** by including both unit and time fixed effects. In practice, TWFE is frequently used to estimate causal effects in panel data settings. However, it is **not** a design-based, non-parametric causal estimator [@imai2021use], and it can suffer from severe biases if the treatment effect is heterogeneous across units or time.

When applying TWFE to datasets with **multiple treatment groups** and **staggered treatment timing**, the estimated causal coefficient is a **weighted average** of all possible two-group, two-period DiD comparisons. Crucially, some of these weights can be **negative** [@goodman2021difference], which leads to potential biases. The weighting scheme depends on:

-   **Group sizes**
-   **Variation in treatment timing**
-   **Placement in the middle of the panel** (units in the middle tend to get the highest weight)

------------------------------------------------------------------------

### Canonical TWFE Model

The canonical TWFE model is typically written as:

$$
Y_{it} = \alpha_i + \lambda_t + \tau W_{it} + \beta X_{it} + \epsilon_{it},
$$

where:

-   $Y_{it}$ = Outcome for unit $i$ at time $t$

-   $\alpha_i$ = Unit fixed effect

-   $\lambda_t$ = Time fixed effect

-   $\tau$ = Causal effect of treatment

-   $W_{it}$ = Treatment indicator ($1$ if treated, $0$ otherwise)

-   $X_{it}$ = Covariates

-   $\epsilon_{it}$ = Error term

An illustrative TWFE event-study model [@stevenson2006bargaining]:

$$ \begin{aligned} Y_{it} &= \sum_{k} \beta_{k} \cdot Treatment_{it}^{k} + \eta_{i} + \lambda_{t} + Controls_{it} + \epsilon_{it}, \end{aligned} $$

where:

-   $Treatment_{it}^k$: Indicator for whether unit $i$ is in its $k$-th year relative to treatment at time $t$.

-   $\eta_i$: Unit fixed effects, controlling for time-invariant unobserved heterogeneity.

-   $\lambda_t$: Time fixed effects, capturing overall macro shocks.

-   Standard Errors: Typically clustered at the group or cohort level.

Usually, researchers drop the period **immediately before treatment** ($k=-1$) to avoid collinearity. However, dropping this or another period inappropriately can shift or bias the estimates.

When there are only two time periods $(T=2)$, TWFE simplifies to the [traditional DiD model](#sec-simple-difference-in-differences). Under **homogeneous treatment effects** and if the [parallel trends assumption](#prior-parallel-trends-test) holds, $\hat{\tau}_{OLS}$ is unbiased. Specifically, the model assumes [@imai2021use]:

1.  **Homogeneous treatment effects** across units and time periods, meaning:
    -   No dynamic treatment effects (i.e., treatment effects do not evolve over time).
    -   The treatment effect is constant across units [@goodman2021difference; @de2020two; @sun2021estimating; @borusyak2024revisiting].
2.  [Parallel trends assumption](#prior-parallel-trends-test)
3.  **Linear additive effects** are valid [@imai2021use].

However, in practice, **treatment effects are often heterogeneous**. If effects vary by cohort or over time, then standard TWFE estimates can be biased, particularly when there is staggered adoption or dynamic treatment effects [@goodman2021difference; @de2020two; @sun2021estimating; @borusyak2024revisiting]. Hence, to use the TWFE, we actually have to argue why the effects are homogeneous to justify TWFE use:

-   Assess treatment heterogeneity: If heterogeneity exists, TWFE may produce biased estimates. Researchers should:
    -   Plot treatment timing across units.
    -   Decompose the treatment effect using the [Goodman-Bacon decomposition](#sec-goodman-bacon-decomposition) to identify negative weights.
    -   Check the proportion of never-treated observations: When 80% or more of the sample is never treated, TWFE bias is negligible.
    -   Beware of bias worsening with long-run effects.
-   Dropping relative time periods: 
    -   If all units eventually receive treatment, two relative time periods must be dropped to avoid multicollinearity.
    -   Some software packages drop periods randomly; if a post-treatment period is dropped, bias may result.
    -   The standard approach is to drop periods -1 and -2.
-   Sources of treatment heterogeneity: 
    -   Delayed treatment effects: The impact of treatment may take time to manifest.
    -   Evolving effects: Treatment effects can increase or change over time (e.g., phase-in effects).

------------------------------------------------------------------------

TWFE compares different types of treatment/control groups:

-   Valid comparisons: 
    -   Newly treated units vs. control units
    -   Newly treated units vs. not-yet treated units
-   Problematic comparisons: 
    -   Newly treated units vs. already treated units (since already treated units do not represent the correct counterfactual).
    -   Strict exogeneity violations: 
        -   Presence of time-varying confounders
        -   Feedback from past outcomes to treatment [@imai2019should]
    -   Functional form restrictions: 
        -   Assumes treatment effect homogeneity.
        -   No carryover effects or anticipation effects [@imai2019should].

------------------------------------------------------------------------

### Limitations of TWFE

TWFE inherits its appeal from a deceptively simple promise: absorb unit and time effects, and the residual variation that survives identifies the causal coefficient. That promise holds when treatment effects are constant. Once effects vary across units or evolve over time, the residual variation no longer maps cleanly onto a single causal quantity. The price of using a one-coefficient model on a many-coefficient world shows up as bias, not as obvious model failure, which is why these limitations were under-appreciated for years.

The strong assumptions that TWFE rides on can be enumerated as follows:

-   No dynamic treatment effects: The model requires that the treatment effect not evolve over time.
-   No unit-level differences: The treatment effect must be constant across all units.
-   Linear additive effects: TWFE assumes that the underlying data-generating process is captured by additive fixed effects plus a constant treatment effect [@imai2021use].

If any of these assumptions are violated, TWFE can produce biased estimates. The mechanics of the bias are worth pausing on, because they motivate every modern remedy discussed below. When treatment is staggered, the regression coefficient is an implicit average of many smaller two-by-two comparisons, and the weights attached to those comparisons are determined by panel structure rather than by what the researcher cares about. Symptoms include:

-   Negative weights and biased estimates: With multiple groups and staggered timing, the TWFE estimate becomes a complicated average of "two-group, two-period" DiD comparisons, some of which can receive **negative weights** [@goodman2021difference].
-   Bias from dropped relative-time periods: If all units eventually get treated, software often drops a reference period (or periods) to avoid multicollinearity. If the dropped period is post-treatment, the bias can worsen. Researchers often drop relative time $-1$ or $-2$.
-   Delayed or evolving treatment effects: If the effect of treatment takes time to manifest or changes over time, TWFE's single coefficient $\tau$ can be misleading.

When **two time periods only** exist, TWFE collapses back to the [traditional DiD model](#sec-simple-difference-in-differences), making these problems far less severe. But as soon as one moves beyond a single treatment period or has variation in treatment timing, these issues become critical. The same logic surfaces in [Multiple Periods and Variation in Treatment Timing](#sec-multiple-periods-and-variation-in-treatment-timing), where staggered adoption is the rule rather than the exception, and these failure modes have spawned a whole generation of estimators (see [Modern Estimators for Staggered Adoption](#sec-modern-estimators-for-staggered-adoption) and [Modern Concerns in DiD](#sec-modern-concerns-in-did)).

Several authors [@sun2021estimating; @callaway2021difference; @goodman2021difference] have catalogued concrete pathologies of TWFE DiD regressions under staggered adoption:

-   Cohort mixing: The regression unintentionally compares newly treated units to already treated units, conflating post-treatment behavior of early adopters with the pre-treatment trends of later adopters.
-   Negative weights: Some group comparisons receive negative weights, which can reverse the sign of the overall estimate.
-   Spurious pre-treatment leads: Leads may appear non-zero if earlier-treated groups remain in the sample as implicit "controls" while later adopters are still untreated.
-   Compounding long-run bias: Heterogeneity in lagged (long-run) effects accumulates as the panel lengthens, so longer windows do not necessarily produce cleaner estimates.

The empirical stakes are substantial. In fields such as finance and accounting, [newer estimators](#sec-modern-estimators-for-staggered-adoption) often reveal **null or much smaller** effects than standard TWFE once bias is properly accounted for [@baker2022much]. Substantial portions of these literatures have had to revisit headline results, which underscores the importance of the diagnostics in the next subsection.

------------------------------------------------------------------------

### Diagnosing and Addressing Bias in TWFE

Before reaching for a sophisticated alternative estimator, it pays to ask how badly TWFE is actually misbehaving in the design at hand. The diagnostics below answer that question with progressively heavier machinery, beginning with a picture of the data and ending with a formal decomposition of the regression coefficient itself. Researchers should treat these as a sequence: a quick visual check tells you whether to worry, the share of never-treated units tells you how much room there is for negative weights, and the [Goodman-Bacon Decomposition](#sec-goodman-bacon-decomposition) tells you which two-by-two comparisons are doing the damage.

1.  [Goodman-Bacon Decomposition](#sec-goodman-bacon-decomposition)

-   Purpose: Decomposes the TWFE DiD estimate into the sum of all two-group, two-period comparisons.
-   Insight: Reveals which comparisons have negative weights and how much each comparison contributes to the overall estimate [@goodman2021difference].
-   Implementation: Identify subgroups by treatment timing, then examine each group-time pair to see how it contributes to the aggregate TWFE coefficient.
-   When it is decisive: If the decomposition shows that "already-treated as control" comparisons receive substantial weight, TWFE is structurally compromised and a remedy from the next subsection is warranted.

2.  [Plotting Treatment Timing](#sec-visualization-did)

-   Visual inspection: Always plot the distribution of treatment timing across units.
-   High risk of bias: If treatment is staggered and many units differ in their adoption times, standard TWFE will often be biased.
-   Cheap and informative: This step takes minutes, costs nothing, and frequently changes which estimator a careful researcher reaches for next.

3.  **Assessing Treatment Heterogeneity Directly**

-   Check for variation in effects: If there is a theoretical or empirical reason to believe that treatment effects differ by subgroup or over time, TWFE might not be appropriate.
-   Size of never-treated sample: When 80% or more of the sample is never treated, the potential for bias in TWFE is smaller because the regression leans heavily on clean treated-versus-never-treated comparisons. Large shares of treated units with varied adoption times raise red flags.
-   Long-run effects: Bias can worsen if the treatment effect accumulates or changes over time, since the coefficient on the treatment indicator averages early and late dynamics with weights chosen by the panel rather than by the researcher.

If the diagnostics suggest that TWFE is mildly biased, an event-study specification with carefully chosen reference periods may suffice. If they suggest a structural problem, the remedies in the next subsection become necessary rather than optional.

------------------------------------------------------------------------

#### Goodman-Bacon Decomposition {#sec-goodman-bacon-decomposition}

The Goodman-Bacon decomposition [@goodman2021difference] is a powerful diagnostic tool for understanding the [TWFE](#sec-two-way-fixed-effects) estimator in settings with [staggered treatment adoption](#sec-staggered-difference-in-differences). This approach clarifies how the TWFE DiD estimate is a weighted average of many **2×2 difference-in-differences comparisons** between groups treated at different times (or never treated).

<!-- For an excellent set of explanatory slides by the author, [see here](https://www.stata.com/meeting/chicago19/slides/chicago19_Goodman-Bacon.pdf). -->

Key Takeaways

-   A pairwise DiD estimate ($\tau$) receives **more weight** when:
    -   The treatment happens **closer to the midpoint** of the observation window.
    -   The comparison involves **more observations** (e.g., more units or more years).
-   Comparisons between early-treated and later-treated groups can produce **negative weights**, potentially biasing the aggregate TWFE estimate.

We illustrate the decomposition using the `castle` dataset from the `bacondecomp` package:

```{r}
library(bacondecomp)
library(tidyverse)

# Load and inspect the castle dataset
castle <- bacondecomp::castle %>% 
  dplyr::select(l_homicide, post, state, year)
head(castle)
```

Running the Goodman-Bacon Decomposition

```{r}
# Apply Goodman-Bacon decomposition
df_bacon <- bacon(
  formula = l_homicide ~ post,
  data = castle,
  id_var = "state",
  time_var = "year"
)

# Display weighted average of the decomposition
weighted_avg <- sum(df_bacon$estimate * df_bacon$weight)
weighted_avg
```

Comparing with the TWFE Estimate

```{r}
library(broom)

# Fit a TWFE model
fit_tw <- lm(l_homicide ~ post + factor(state) + factor(year), data = castle)
tidy(fit_tw)
```

> **Interpretation**: The TWFE estimate (approx. 0.08) equals the weighted average of the Bacon decomposition estimates, confirming the decomposition's validity.

------------------------------------------------------------------------

Visualizing the Decomposition (Figure \@ref(fig:did-decomposition-treatment-effects-type-weight))

```{r did-decomposition-treatment-effects-type-weight, fig.cap='Decomposition of treatment effects by comparison type and weight.', fig.alt='Scatter plot of treatment effect estimates versus their weights. Points are colored by comparison type: red (earlier vs later treated), green (later vs earlier treated), and blue (treated vs untreated). Most comparisons cluster near zero weight, but a few blue points have large weights and high positive estimates, indicating that untreated comparisons drive much of the overall effect. A legend in the top right explains the color coding.'}
library(ggplot2)

ggplot(df_bacon) +
  aes(
    x = weight,
    y = estimate,
    color = type
  ) +
  geom_point() +
  labs(
    x = "Weight",
    y = "Estimate",
    color = "Comparison Type"
  ) +
  causalverse::ama_theme()
```

> **Insight**: This plot shows the contribution of each 2×2 DiD comparison, highlighting how estimates with large weights dominate the overall TWFE coefficient.

------------------------------------------------------------------------

Interpretation and Practical Implications

-   Purpose: Decomposes the TWFE DiD estimate into the sum of all two-group, two-period comparisons.
-   Insight: Reveals how much each comparison contributes to the overall estimate and whether any have negative or misleading effects.
-   Implementation: 
    -   Identify subgroups by treatment timing.
    -   Compute DiD for each 2×2 comparison (early vs. late, late vs. never, etc.).
    -   Evaluate how these contribute to the final TWFE estimate.

When time-varying covariates are included that allow for identification within treatment timing groups, certain problematic comparisons (like "early vs. late") may no longer influence the TWFE estimator directly. These scenarios may collapse into simpler within-group estimates, improving identification (Table \@ref(tab:did-goodman-bacon-comparison)).

| Comparison Type     | Description                                    | Common Issue          |
|---------------------|------------------------------------------------|-----------------------|
| Treated vs. Never   | Clean comparisons if never-treated units exist | Often reliable        |
| Early vs. Late      | Later group is control in earlier period       | May introduce bias    |
| Late vs. Early      | Early group is control in later period         | May reverse causality |
| Treated vs. Treated | Within-treatment variation by timing           | Sensitive to dynamics |

Table: (\#tab:did-goodman-bacon-comparison) Goodman-Bacon comparison types.

------------------------------------------------------------------------

### Remedies for TWFE's Shortcomings

The estimators surveyed here all share a single ambition: recover an interpretable average treatment effect when treatment is staggered, treatment effects are heterogeneous, and dynamics matter. They differ in what data they need, what they assume, and what they deliver. Rather than treat the list as a menu of independent tools, it helps to keep three questions in mind while reading. First, does the design have a meaningful pool of never-treated (or not-yet-treated) units? Second, do treatment effects evolve with exposure length, and is that dynamic itself the object of interest? Third, can units toggle treatment on and off, or is treatment an absorbing state? The answers narrow the field quickly and motivate the cross-references at the end of the subsection (see also [Modern Estimators for Staggered Adoption](#sec-modern-estimators-for-staggered-adoption) and [Modern Concerns in DiD](#sec-modern-concerns-in-did)).

The core idea uniting the estimators below is to disaggregate the single TWFE coefficient into well-defined building blocks (typically group-time or cohort-time effects), estimate each block from comparisons that are guaranteed to be clean, and only then aggregate using transparent weights. Stated this way, modern DiD looks less like a zoo of competing methods and more like one strategy with several flavors.

1.  [Group-Time Average Treatment Effects](#sec-group-time-average-treatment-effects-callaway2021difference)

@callaway2021difference propose a two-step approach:

1.  Group-time treatment effects: In each time period, estimate the effect for the cohort that first received treatment in that period (compared to a never-treated group).
2.  Aggregate: Use a bootstrap procedure to account for autocorrelation and clustering, then aggregate across groups.

-   Advantages: Allows for **heterogeneous treatment effects** across groups and over time; compares treated groups only with never-treated units (or well-chosen controls).
-   When it dominates: This is often the default first move when the panel contains a credible never-treated (or not-yet-treated) comparison group and the researcher wants to summarize effects flexibly across cohorts and exposure lengths.
-   What its assumptions cost: It needs parallel trends to hold conditional on the chosen comparison group and is most informative when each cohort has enough mass to estimate cohort-specific effects with reasonable precision.
-   Implementation: `did` package in R.

2.  [Event-Study Design with Cohort-Specific Estimates](#sec-cohort-average-treatment-effects-sun2021estimating)

@sun2021estimating build on @callaway2021difference to handle event-study settings:

-   Lags and leads: Capture dynamic treatment effects by including time lags and leads relative to the event (treatment).
-   Cohort-specific estimates: Estimate separate paths of outcomes for each cohort, controlling for other cohorts carefully.
-   Interaction-weighted estimator: Adjusts for differences in when treatment began.
-   When it dominates: Choose this when the dynamic path of the effect (the shape of the [event study](#sec-event-studies)) is the substantive question and standard event-study coefficients in TWFE are contaminated by other cohorts' treatment effects bleeding into the leads and lags.
-   Implementation: `fixest` package in R.

3.  [Panel Match DiD Estimator with In-and-Out Treatment Conditions](#sec-panel-match-did-estimator-with-in-and-out-treatment-conditions)

@imai2021use develop methods allowing units to **switch in and out** of treatment:

-   **Matching** to create a weighted version of TWFE, addressing some of the bias from heterogeneous effects.
-   When it dominates: This is the natural choice when treatment is reversible (units enter and exit) so absorbing-state estimators do not apply, and when the researcher is comfortable arguing for [conditional ignorability](#sec-conditional-ignorability-assumption) given an observed history. It connects DiD to the broader logic of [matching](#sec-matching-methods) on pre-treatment trajectories.
-   Implementation: `wfe` and `PanelMatch` R packages.

4.  Two-Stage Difference-in-Differences (DiD2S)

@gardner2022two propose **two-stage DiD**:

-   Idea: Partial out fixed effects first, then perform a second-stage regression that focuses on within-group/time variation.
-   Strength: Handles heterogeneous treatment effects well, especially when never-treated units are present.
-   When it dominates: It offers a computationally light path to consistent staggered-DiD estimates with familiar regression output, useful when researchers want to incorporate covariates flexibly without committing to the full machinery of cohort-time aggregation.
-   Implementation: `did2s` R package.

5.  [Switching DiD Estimator](#sec-switching-difference-in-differences-estimator-de2020two)

-   If a study has **never-treated units**, @de2020two suggest an [switching DiD estimator](#sec-switching-difference-in-differences-estimator-de2020two) to recover the [average treatment effect](#sec-average-treatment-effect).
-   When it dominates: Particularly useful when researchers care about a single, period-specific average effect among switchers rather than a full event-time profile.
-   Caveat: This approach still fails to detect heterogeneity if treatment effects vary with **exposure length** [@sun2022linear], so pair it with a dynamic specification when long-run effects are plausible.

6.  [Matrix Completion Estimator](#sec-matrix-completion-estimator)

This estimator imputes missing potential outcomes for treated cells by fitting a low-rank structure to the panel, much like the [synthetic control](#sec-synthetic-control) and [synthetic difference-in-differences](#sec-synthetic-difference-in-differences) approaches it is conceptually related to. It comes into its own when the panel is wide (many units, many periods), parallel trends across simple cohort groupings looks implausible, and a flexible counterfactual model is preferable to a strict comparison-group design (see also [counterfactual estimators](#sec-counterfactual-estimators)).

7.  [Reshaped Inverse Probability Weighting-TWFE Estimator](#sec-reshaped-inverse-probability-weighting-twfe-estimator)

-   Design-based approaches: @arkhangelsky2024design offer further refinements that incorporate inverse probability weighting.
-   Goal: Improve balance and reduce bias from non-random treatment timing.
-   When it dominates: Useful when researchers want to keep the convenience of a TWFE-style regression but fear that the weights TWFE implicitly applies are misaligned with the target estimand. The reweighting recasts the estimand as a [design-based](#sec-design-based) object, trading some efficiency for a more credible interpretation.

8.  [Stacked DiD](#sec-stacked-difference-in-differences) (simpler but biased)

-   Build stacked datasets for each treatment cohort, running separate regressions for each "event window."
-   This approach is simpler but can still carry biases if the underlying assumptions are violated [@gormley2011growing; @cengiz2019effect; @deshpande2019screened].
-   When it dominates: It is often the right pragmatic first pass in industry settings or when communicating to audiences uncomfortable with cohort-time aggregation, since each stacked panel reduces to a familiar two-by-two comparison. Treat it as a cousin of the formal staggered-DiD estimators above rather than a substitute, and always cross-check against one of them.

9.  [Doubly Robust Difference-in-Differences Estimators](#sec-counterfactual-estimators) (DR-DID) [@sant2020doubly]

-   DR-DID estimators combine outcome regression and propensity score weighting to identify treatment effects, remaining consistent if either model is correctly specified.
-   They achieve local efficiency under joint correctness and can be applied to both panel and repeated cross-section data.
-   When it dominates: Ideal when covariate adjustment is essential (the [conditional ignorability](#sec-conditional-ignorability-assumption) and [overlap](#sec-overlap-positivity-assumption) assumptions are doing real work) and the researcher wants insurance against misspecification of either the outcome or the treatment model.

10.  [Nonlinear Difference-in-Differences](#sec-nonlinear-did)

-   When the outcome is binary, count, or otherwise bounded, additive parallel trends can be implausible because shifts at one part of the distribution mechanically constrain shifts elsewhere. Nonlinear DiD addresses this by working on a transformed scale or by directly modeling the distribution. See also [Changes-in-Changes](#sec-changes-in-changes) and the comparison in [CIC vs Quantile DiD](#sec-cic-vs-qdid).

A practical decision logic ties these threads together. If the panel has a reliable never-treated group and the question is "what is the average effect across cohorts and exposure lengths," start with @callaway2021difference and use @sun2021estimating when the dynamic path is the focus. If treatment toggles on and off, switch to the panel-matching family of @imai2021use. If covariates carry most of the identifying weight, prefer a doubly robust estimator. If the panel is wide and parallel trends is hard to defend on raw data, lean on counterfactual estimators such as matrix completion or [synthetic difference-in-differences](#sec-synthetic-difference-in-differences). Stacked DiD and TWFE remain useful as exposition tools and pragmatic baselines, but should not stand alone as the headline estimator under staggered adoption.

------------------------------------------------------------------------

### Best Practices and Recommendations

The recommendations below collapse the preceding diagnostic and remedy discussions into a workflow. The order matters: confirm whether TWFE is even defensible for the design, diagnose bias before reaching for alternatives, tune the event-study specification to avoid self-inflicted artifacts, and only then graduate to a modern estimator if the diagnostics warrant it. The same logic underlies the more detailed treatment in [Modern Estimators for Staggered Adoption](#sec-modern-estimators-for-staggered-adoption).

1.  **When is TWFE Appropriate?**
    -   Single treatment period: TWFE DiD works well if there is **only one** treatment period for all treated units (no variation in timing). In that special case it reduces to the [simple DiD](#sec-simple-difference-in-differences) and the staggered-adoption pathologies vanish.
    -   Homogeneous effects: If strong theoretical or empirical reasons suggest **constant treatment effects** across cohorts and over time, TWFE remains a reasonable choice. The argument for homogeneity should be made explicitly and defended, not assumed by default.
2.  **Diagnosing and Addressing Bias with Staggered Adoption**
    -   Plot treatment timing: Examine the distribution of treatment timing across units (see [Visualization](#sec-visualization-did)). If treatment adoption is highly staggered, TWFE is likely to produce biased estimates.
    -   Decomposition methods: Use the [Goodman-Bacon Decomposition](#sec-goodman-bacon-decomposition) [@goodman2021difference] to see how TWFE pools comparisons (and whether negative weights emerge). If decomposition is infeasible (e.g., unbalanced panels), the share of never-treated units can indicate potential bias severity.
        -   Decomposes the TWFE DiD estimate into two-group, two-period comparisons.
        -   Identifies which comparisons receive negative weights, which can lead to biased estimates.
        -   Helps determine the influence of specific groups on the overall estimate.
    -   Discuss heterogeneity: Explicitly state the likelihood of treatment effect heterogeneity and incorporate it into the research design rather than treating it as a footnote.
3.  **Event-Study Specifications within TWFE**
    -   Avoid arbitrary binning: Do not collapse multiple time periods into a single bin unless you can justify **homogeneous** effects within that bin.
    -   Full relative-time indicators: Include flexible event-time indicators, carefully choosing a reference period (commonly $-1$, the year before treatment). Specifically, include fully flexible relative time indicators and justify the reference period (usually $l = -1$ or the period just prior to treatment).
    -   Beware of multicollinearity: Including leads and lags can cause multicollinearity and artificially produce significant "pre-trends."
    -   Drop the right periods: If all units eventually get treated, dropping post-treatment periods accidentally can bias results.
4.  [**Consider Alternative Estimators**](#sec-modern-estimators-for-staggered-adoption): If the diagnostics flag staggered adoption with non-trivial heterogeneity, move to one of the modern estimators discussed above. Robustness checks across several modern estimators (see also [Robustness Checks](#sec-robustness-checks-quasi-exp) and [Modern Concerns in DiD](#sec-modern-concerns-in-did)) are now the norm in careful applied work, since agreement across methods that lean on different assumptions is the strongest evidence the result is not an artifact of any one specification.

------------------------------------------------------------------------

## Multiple Periods and Variation in Treatment Timing {#sec-multiple-periods-and-variation-in-treatment-timing}

TWFE has been extended beyond the simple DiD setup to **multiple periods** and **staggered adoption** (where treatment occurs at different times for different units). Such designs are common in applied economics, public policy, and longitudinal research. However, standard TWFE regressions can be biased in these contexts when treatment effects are heterogeneous across groups or over time.

### Staggered Difference-in-Differences {#sec-staggered-difference-in-differences}

In **staggered treatment adoption** (also called event-study DiD or dynamic DiD):

-   Different units adopt the treatment at **different time periods**.
-   Standard TWFE often produces biased estimates because it "pools" all treated units (regardless of when they started treatment) together, implicitly comparing newly treated units to already treated ones.
-   Treatments that occurred earlier may contaminate the counterfactual for later adopters if the model does not properly handle dynamic or heterogeneous effects [@wing2024designing; @baker2022much].
-   For applied guidance, see [@wing2024designing] and recommendations in [@baker2022much].

Researchers should be aware that standard TWFE can **mix treatment effects** of early adopters (long-exposed) with later adopters (newly exposed), potentially assigning **negative weights** to particular group comparisons [@goodman2021difference].

When using staggered adoption, the following assumptions are critical:

1.  **Rollout Exogeneity**\
    Treatment assignment and timing should be uncorrelated with potential outcomes.

    -   Evidence: Regress adoption on pre-treatment variables. And if you find evidence of correlation, include linear trends interacted with pre-treatment variables [@hoynes2009consumption]
    -   **Evidence** [@deshpande2019screened, p. 223]:
        -   Treatment is random: Regress treatment status at the unit level to all pre-treatment observables. If you have some that are predictive of treatment status, you might have to argue why it's not a worry. At best, you want this.
        -   Treatment timing is random: Conditional on treatment, regress timing of the treatment on pre-treatment observables. At least, you want this.

2.  **No Confounding Events**\
    Ensure no other policies or shocks coincide with the staggered treatment rollout.

3.  **Exclusion Restrictions**

    -   No Anticipation: Treatment timing should not affect outcomes prior to treatment.
    -   Invariance to History: Treatment duration shouldn't matter; only the treated status matters (often violated).

4.  **Standard DID Assumptions**

    -   **Parallel Trends** (Conditional or Unconditional)
    -   **Random Sampling**
    -   **Overlap** (Common Support)
    -   **Effect Additivity**

------------------------------------------------------------------------