6.4 Naive estimator

A naive estimate of ATT would just be the difference in means between the treated and control groups in the period following the expansion.

naive  <- mean(dat_canonical$sahieunins138[dat_canonical$expand == 1 & dat_canonical$year >= 2014]) - 
          mean(dat_canonical$sahieunins138[dat_canonical$expand == 0 & dat_canonical$year >= 2014])

print(naive)
## [1] -13.65797

The naive estimate suggests that uninsured rate dropped by -13.66 percentage points following the Medicaid expansion in 2014. But can we trust this estimate? Not really!

Here are some reasons why the naive estimate fails:

  1. One way to assess the validity of naive estimate is to compare the (natural) experiment on hand with the randomized control case.
    Note that we are very far away from the randomized controlled trial in this case. The treatment (decision to expand Medicaid) is not random. Note that states voluntarily decided to expand Medicaid. This means that expansion versus non-expansion states may be very different in terms of pre-treatment characteristics. For example, many of the southern states did not expand Medicaid. Also, pre-treatment uninsured rates of southern states are generally higher compared to non-southern states. The naive comparison can simply be capturing the difference in pre-treatment characteristics correlated with the treatment assignment.

  2. The baseline outcome among the treatment group may differ significantly from the control group. For example, southern states have higher population of Blacks compared to non-South. Typically, uninsured rate is higher among Blacks. Hence, it is difficult to disentagle the influence of democraphic composition versus Medicaid expansion.

Let’s evaluate the difference in uninsured rate between the expansion versus the non-expansion states in the pre-treatment year (2013).

naive_pre  <- mean(dat_canonical$sahieunins138[dat_canonical$expand == 1 & dat_canonical$year < 2014]) - 
           mean(dat_canonical$sahieunins138[dat_canonical$expand == 0 & dat_canonical$year < 2014])

print(naive_pre)
## [1] -7.844546

Note that treatment units on average have 7.68 percentage points lower uninsured rate compared to the control units even prior to the treatment. Hence, the naive estimator captures the pre-existing differerences in outcome; something that we don’t want.

  1. Since the treatment is not randomized a lot of baseline characteristics that influence the outcome across the treated and control groups may differ dramatically. If these variables are thought to influence the dynamics of the outcome variable, then we will be capturing the influence of such variables rather than the treatment itself.