MatchIt: Getting Started

Introduction

MatchIt implements the suggestions of Ho, Imai, King, and Stuart (2007) for improving parametric statistical models for estimating treatment effects in observational studies and reducing model dependence by preprocessing data with semi-parametric and non-parametric matching methods. After appropriately preprocessing with MatchIt, researchers can use whatever parametric model they would have used without MatchIt and produce inferences that are more robust and less sensitive to modeling assumptions. MatchIt reduces the dependence of causal inferences on commonly made, but hard-to-justify, statistical modeling assumptions using a large range of sophisticated matching methods. The package includes several popular approaches to matching and provides access to methods implemented in other packages through its single, unified, and easy-to-use interface.

Matching is used in the context of estimating the causal effect of a binary treatment or exposure on an outcome while controlling for measured pre-treatment variables, typically confounding variables or variables prognostic of the outcome. Here and throughout the MatchIt documentation we use the word “treatment” to refer to the focal causal variable of interest, with “treated” and “control” reflecting the names of the treatment groups. The goal of matching is to produce covariate balance, that is, for the distributions of covariates in the two groups to be approximately equal to each other, as they would be in a successful randomized experiment. The importance of covariate balance is that it allows for increased robustness to the choice of model used to estimate the treatment effect; in perfectly balanced samples, a simple difference in means can be a valid treatment effect estimate. Here we do not aim to provide a full introduction to matching or causal inference theory, but simply to explain how to use MatchIt to perform nonparametric preprocessing. For excellent and accessible introductions to matching, see Stuart (2010) and Austin (2011).

A matching analysis involves four primary steps: 1) planning, 2) matching, 3) assessing the quality of matches, and 4) estimating the treatment effect and its uncertainty. Here we briefly discuss these steps and how they can be implemented with MatchIt; in the other included vignettes, these steps are discussed in more detail.

We will use Lalonde’s data on the evaluation of the National Supported Work program to demonstrate MatchIt’s capabilities. First, we load MatchIt and bring in the lalonde dataset.

library("MatchIt")
data("lalonde")

head(lalonde)

##      treat age educ   race married nodegree re74 re75       re78
## NSW1     1  37   11  black       1        1    0    0  9930.0460
## NSW2     1  22    9 hispan       0        1    0    0  3595.8940
## NSW3     1  30   12  black       0        0    0    0 24909.4500
## NSW4     1  27   11  black       0        1    0    0  7506.1460
## NSW5     1  33    8  black       0        1    0    0   289.7899
## NSW6     1  22    9  black       0        1    0    0  4056.4940

The statistical quantity of interest is the causal effect of the treatment (treat) on 1978 earnings (re78). The other variables are pre-treatment covariates. See ?lalonde for more information on this dataset. In particular, the analysis is concerned with the marginal, total effect of the treatment for those who actually received the treatment.

In what follows, we briefly the four steps of a matching analysis and how to implement them in MatchIt. For more details, we recommend reading the other vignettes, vignette("matching-methods"), vignette("assessing-balance"), and vignette("estimating-effects"), especially for users less familiar with matching methods. For the use of MatchIt with sampling weights, also see vignette("sampling-weights"). It is important to recognize that the ease of using MatchIt does not imply the simplicity of matching methods; advanced statistical methods like matching that require many decisions to be made and caution in their use should only be performed by those with statistical training.

Planning

The planning phase of a matching analysis involves selecting the type of effect to be estimated, selecting the target population to which the treatment effect is to generalize, and selecting the covariates for which balance is required for an unbiased estimate of the treatment effect. Each of these are theoretical steps that do not involve performing analyses on the data. Ideally, they should be considered prior to data collection in the planning stage of a study. Thinking about them early can aid in performing a complete and cost-effective analysis.

Selecting the type of effect to be estimated. There are a few different types of effects to be estimated. In the presence of mediating variables, one might be interested in the direct effect of the treatment that does not pass through the mediating variables or the total effect of the treatment across all causal pathways. Matching is well suited for estimating total effects, and specific mediation methods may be better suited for other mediation-related quantities. One may be interested in a conditional effect or a marginal effect. A conditional effect is the effect of a treatment within some strata of other prognostic variables (e.g., at the patient level), and a marginal effect is the average effect of a treatment in a population (e.g., for implementing a broad policy change). Different types of matching are well suited for each of these, but the most common forms are best used for estimating marginal treatment effects; for conditional treatment effects, typically modeling assumptions are required or matching must be done within strata of the conditioning variables. Matching can reduce the reliance on correct model specification for conditional effects.

Selecting a target population. The target population is the population to which the effect estimate is to generalize. Typically, an effect estimated in a sample generalizes to the population from which the sample is a probability sample. If the sample is not a probability sample from any population (e.g., it is a convenience sample or involves patients from an arbitrary hospital), the target population can be unclear. Often, the target population is a group of units who are eligible for the treatment (or a subset thereof). Causal estimands are defined by the target population to which they generalize.

The average treatment effect in the population (ATE) is the average effect of the treatment for all units in the target population. The average treatment effect in the treated (ATT) is the average effect of the treatment for units like those who actually were treated. The most common forms of matching are best suited for estimating the ATT, though some are also available for estimating the ATE. Some matching methods distort the sample in such a way that the estimated treatment effect corresponds neither to the ATE nor to the ATT, but rather to the effect in an unspecified population (sometimes called the ATM, or average treatment effect in the remaining matched sample). When the target population is not so important (e.g., in the case of treatment effect discovery), such methods may be attractive; otherwise, care should be taken in ensuring the effect generalizes to the target population of interest. Different matching methods allow for different target populations, so it is important to choose a matching method that allows one to estimate the desired effect.

Selecting covariates to balance. Selecting covariates carefully is critical for ensuring the resulting treatment effect estimate is free of confounding and can be validly interpreted as a causal effect. To estimate total causal effects, all covariates must be measured prior to treatment (or otherwise not be affected by the treatment). Covariates should be those that cause variation in the outcome and selection into treatment group; these are known as confounding variables. See VanderWeele (2019) for a guide on covariate selection. Ideally these covariates are measured without error and are free of missingness.

Check Initial Imbalance

After planning and prior to matching, it can be a good idea to view the initial imbalance in one’s data that matching is attempting to eliminate. We can do this using the code below:

# No matching; constructing a pre-match matchit object
m.out0 <- matchit(treat ~ age + educ + race + married + 
                   nodegree + re74 + re75, data = lalonde,
                 method = NULL, distance = "glm")

The first argument is a formula relating the treatment to the covariates used in estimating the propensity score and for which balance is to be assessed. The data argument specifies the dataset where these variables exist. Typically, the method argument specifies the method of matching to be performed; here, we wet it to NULL so we can assess balance prior to matching¹. The distance argument specifies the method for estimating the propensity score, a one-dimensional summary of all the included covariates, computed as the predicted probability of being the treated group given the covariates; here, we set it to "glm" for generalized linear model, which implements logistic regression by default² (see ?distance for other options).

Below we assess balance on the unmatched data using summary():

# Checking balance prior to matching
summary(m.out0)

## 
## Call:
## matchit(formula = treat ~ age + educ + race + married + nodegree + 
##     re74 + re75, data = lalonde, method = NULL, distance = "glm")
## 
## Summary of Balance for All Data:
##            Means Treated Means Control Std. Mean Diff. Var. Ratio eCDF Mean eCDF Max
## distance          0.5774        0.1822          1.7941     0.9211    0.3774   0.6444
## age              25.8162       28.0303         -0.3094     0.4400    0.0813   0.1577
## educ             10.3459       10.2354          0.0550     0.4959    0.0347   0.1114
## raceblack         0.8432        0.2028          1.7615          .    0.6404   0.6404
## racehispan        0.0595        0.1422         -0.3498          .    0.0827   0.0827
## racewhite         0.0973        0.6550         -1.8819          .    0.5577   0.5577
## married           0.1892        0.5128         -0.8263          .    0.3236   0.3236
## nodegree          0.7081        0.5967          0.2450          .    0.1114   0.1114
## re74           2095.5737     5619.2365         -0.7211     0.5181    0.2248   0.4470
## re75           1532.0553     2466.4844         -0.2903     0.9563    0.1342   0.2876
## 
## 
## Sample Sizes:
##           Control Treated
## All           429     185
## Matched       429     185
## Unmatched       0       0
## Discarded       0       0

We can see severe imbalances as measured by the standardized mean differences (Std. Mean Diff.), variance ratios (Var. Ratio), and empirical cumulative density function (eCDF) statistics. Values of standardized mean differences and eCDF statistics close to zero and values of variance ratios close to one indicate good balance, and here many of them are far from their ideal values.

Matching

Now, matching can be performed. There are several different classes and methods of matching, described in vignette("matching-methods"). Here, we begin by briefly demonstrating 1:1 nearest neighbor (NN) matching on the propensity score, which is appropriate for estimating the ATT. One by one, each treated unit is paired with an available control unit that has the closest propensity score to it. Any remaining control units are left unmatched and excluded from further analysis. Due to the theoretical balancing properties of the propensity score described by Rosenbaum and Rubin (1983), propensity score matching can be an effective way to achieve covariate balance in the treatment groups. Below we demonstrate the use of matchit() to perform nearest neighbor propensity score matching.

# 1:1 NN PS matching w/o replacement
m.out1 <- matchit(treat ~ age + educ + race + married + 
                   nodegree + re74 + re75, data = lalonde,
                 method = "nearest", distance = "glm")

We use the same syntax as before, but this time specify method = "nearest" to implement nearest neighbor matching, again using a logistic regression propensity score. Many other arguments are available for tuning the matching method and method of propensity score estimation.

The matching outputs are contained in the m.out1 object. Printing this object gives a description of the type of matching performed:

m.out1

## A matchit object
##  - method: 1:1 nearest neighbor matching without replacement
##  - distance: Propensity score
##              - estimated with logistic regression
##  - number of obs.: 614 (original), 370 (matched)
##  - target estimand: ATT
##  - covariates: age, educ, race, married, nodegree, re74, re75

The key components of the m.out1 object are weights (the computed matching weights), subclass (matching pair membership), distance (the estimated propensity score), and match.matrix (which control units are matched to each treated unit). How these can be used for estimating the effect of the treatment after matching is detailed in vignette("estimating-effects").

Assessing the Quality of Matches

Although matching on the propensity score is often effective at eliminating differences between the treatment groups to achieve covariate balance, its performance in this regard must be assessed. If covariates remain imbalanced after matching, the matching is considered unsuccessful, and a different matching specification should be tried. MatchIt offers a few tools for the assessment of covariate balance after matching. These include graphical and statistical methods. More detail on the interpretation of the included plots and statistics can be found in vignette("assessing-balance").

In addition to covariate balance, the quality of the match is determined by how many units remain after matching. Matching often involves discarding units that are not paired with other units, and some matching options, such as setting restrictions for common support or calipers, can further decrease the number of remaining units. If, after matching, the remaining sample size is small, the resulting effect estimate may be imprecise. In many cases, there will be a trade-off between balance and remaining sample size. How to optimally choose among them is an instance of the fundamental bias-variance trade-off problem that cannot be resolved without substantive knowledge of the phenomena under study. Prospective power analyses can be used to determine how small a sample can be before necessary precision is sacrificed.

To assess the quality of the resulting matches numerically, we can use the summary() function on m.out1 as before. Here we set un = FALSE to suppress display of the balance before matching for brevity and because we already saw it. (Leaving it as TRUE, its default, would display balance both before and after matching.)

# Checking balance after NN matching
summary(m.out1, un = FALSE)

## 
## Call:
## matchit(formula = treat ~ age + educ + race + married + nodegree + 
##     re74 + re75, data = lalonde, method = "nearest", distance = "glm")
## 
## Summary of Balance for Matched Data:
##            Means Treated Means Control Std. Mean Diff. Var. Ratio eCDF Mean eCDF Max Std. Pair Dist.
## distance          0.5774        0.3629          0.9739     0.7566    0.1321   0.4216          0.9740
## age              25.8162       25.3027          0.0718     0.4568    0.0847   0.2541          1.3938
## educ             10.3459       10.6054         -0.1290     0.5721    0.0239   0.0757          1.2474
## raceblack         0.8432        0.4703          1.0259          .    0.3730   0.3730          1.0259
## racehispan        0.0595        0.2162         -0.6629          .    0.1568   0.1568          1.0743
## racewhite         0.0973        0.3135         -0.7296          .    0.2162   0.2162          0.8390
## married           0.1892        0.2108         -0.0552          .    0.0216   0.0216          0.8281
## nodegree          0.7081        0.6378          0.1546          .    0.0703   0.0703          1.0106
## re74           2095.5737     2342.1076         -0.0505     1.3289    0.0469   0.2757          0.7965
## re75           1532.0553     1614.7451         -0.0257     1.4956    0.0452   0.2054          0.7381
## 
## Sample Sizes:
##           Control Treated
## All           429     185
## Matched       185     185
## Unmatched     244       0
## Discarded       0       0

At the top is a summary of covariate balance after matching. Although balance has improved for some covariates, in general balance is still quite poor, indicating that nearest neighbor propensity score matching is not sufficient for removing confounding in this dataset. The final column, Std. Pair Diff, displays the average absolute within-pair difference of each covariate. When these values are small, better balance is typically achieved and estimated effects are more robust to misspecification of the outcome model (King and Nielsen 2019; Rubin 1973).

Next is a table of the sample sizes before and after matching. The matching procedure left 244 control units unmatched. Ideally, unmatched units would be those far from the treated units and would require greater extrapolation were they to have been retained. We can visualize the distribution of propensity scores of those who were matched using plot() with type = "jitter":

plot(m.out1, type = "jitter", interactive = FALSE)

(m.13MEHH+wxe+bF67sDaN4y+skzzJVTs1�pZqhHsqZDXJD7H�6IiIAL1EPQXPMAZP�/A1F3000s �cUXr39JDEkdulq2cyussEKNPk��v47+l24c6tttt2zfPPPMk613NNN7uKL4c6#800080; } wSRM{Orl�CAc7aC4CyyyRwGpAAAHR��a60AN4+NRydn.�jEAERE{f/~d06{Orl�:tXHr4gjd2/RludedF1Q7HZRC4Plgd46s8NCiLBXX35OABFrF�Q�F+ieb2Tuo6X��zn3v4uufr6�ɻ�uPsZHdxlqe5k6s8N 0.0sjEAERE{f5OABTMNn4013Ne7iE��/z4CLUiylWXfVFGk.g219��.g8xi/ri��ǽ=�fr�7747i1c } /*3Ne7iE��/z4CLUiylWXfVFGk.g219��. { color:liaDm9J46XpAAAHR��a60t2zfPMnette("assesYEERGD8AsWZS�8AsWMQrlQwagjd2/RludedF2��BERABBERAF1Q7HZR/�}YqwrF3Tan>ften in�CAcpZZZrEoiv32 (tr0uU+rFFo each tren class="fu"EF0j66+/LkyL/dznqFY+u5o�|VMR��vOuV0IN��GQ�~47n0xaW8/EH�=d+��n4��')8a�4JcsLjGaMwdGkRddkrwH10a} /* Exten>(m.out1, (m.out0)


�t5nA+rFF6; }"img src="dat;
}
table td//2/8r�Dni*s7B��^haq31rqJOYYoRFwViWDOe+211+LpdFmnIRWf7McHNQ2IVEZVExBuBLqoiwLHEbp0R6cBMY3Pab1CKm7xefzyyy/dzTffHP/y10BgE3AJwNf2kUceiaKThgDnjjHGGPlT4nYl6X7zzTdRdNPtnoZ8eabHWOcZItB4KAo0TvLPTX676DkqSsv24X7wj++8Iz4zTP6�to be estimat44+liIgAt1L4Aa> ePi0heXeffectivb�Wjkadiusrl�CAxVqABGPsj, KinecccTP6�to be estn: un/VERE{f/~d0Ea>m+z�{+dth: 1px;
border-color: #DDDDDD;
border-style: atched sample). Wo8i1/oe�H��Asvo6d8olor:x�nb�;
f�~Li7:(1FA�&B(e.g., in 0hpn/+4l4C, and Stuartscore
�7                  nodegree + re74 + re75, data =19/jiE� lalonde,   4
qwLQ7MZLa6/PjGxword *SX�so�@t�4+z61w	x�DEj��)1hecad.OKMes:
## �eneOo��:Nl4+EEXbable 9��aKThe�o