Federal District Court Reversal Rates: Experience And Age

It seems almost axiomatic that, among the hundreds of federal district court judges, some are better at their job than others.  One possible measure of the quality of their work is the frequency with which their decisions are reversed on appeal.  A defining feature of our court hierarchy is that higher courts have the power to correct lower ones.  Thus, a reversal of a lower court decision means, in some sense, that the lower court judge made an error, the correction of which delayed the resolution of the case and incurred the considerable expense of an appeal. 

Drawing and expanding on the methodology described by Royce de R. Barondes in his 2010 essay Federal District Judge Gender and Reversals, I collected Westlaw Litigation Analytics Appeals reports for 1052 district court judges commissioned between 1989 and 2020.  After limiting to judges with at least 50 appeals in the Westlaw data and excluding extremely early decisions, some of which were likely appeals from state court decisions, I was left with 252,703 observations covering 877 judges.  Due to export limitations in Westlaw, one judge with more than 1000 observations was limited to the first 1000.  Following Barondes, “reversed” is defined as reversed, reversed in part, or vacated.  In this series of posts I will be exploring the dataset in a broad way. 

At the outset I should be clear that reversal rate is an imperfect proxy for quality.  Sometimes the district court decision was actually correct at the time, as when a lower court judge correctly applies a precedent which is then overturned on appeal.  And circuit courts make mistakes as well, hence such mechanisms as rehearing en banc and appeal to the Supreme Court.  However, only a small fraction of appellate decisions are appealed further and many of those are ultimately affirmed.  Still, we must recognize that a circuit court’s decision is not the “ground truth” and that not all reversals are equal (consider a complete reversal versus a reversal-in-part that affirms every substantive aspect of a case with only a minor change in the calculation of damages).

For scale, the histogram of the number of observations per judge:

We can see that there are some outliers with a large number of appeals, but overall this is a fairly typical half-normal distribution, as expected given that a judge cannot have fewer than zero appeals.  Here is the QQ plot against a half-normal distribution:

With that in mind, let us begin in perhaps the most obvious place: experience.  Do judges get better at their jobs over time?  First a histogram of the number of observations by the number of years since the judge’s commission.

(“Years Since Commission” is the number of years between the district judge’s commissioning and the date of the appellate decision.  I would prefer to use the date of the lower court decision being appealed, but that was not available in the Westlaw data.)

We see some positive skew here caused by the fact that judges leave the bench over time.  There is also a clear “startup” period from years 0-2 likely primarily due to the lag between cases progressing enough for a decision to be made and then again for an appeal to be made. 

Now we can get to the main question: How does the reversal rate vary with experience?

I have excluded the extremes of the data due to the small number of observations and small number of judges.  As Barondes found in 2010, there is a modest positive correlation between years on the bench and reversal rate, our case R2=0.133.  On its own this is not enough to show significance, but that will wait for a more complete model.

One reason why this is not enough to show significance is that experience does not exist in a vacuum.  Experience comes only with age, and age could also be associated with a change in reversal rates.  First, the number of observations by age:

We can see there is very little data outside of the 1st and 99th percentiles of age 45 and age 80.  Looking at the reversal rate by age shows a fairly clear relationship.

This could be caused by many factors: perhaps older judges making bolder decisions or feel more confident in disagreeing with established precedent or perhaps it reflects a loss of mental acuity.  It is well known that there are differences in aging between men and women, and this leads naturally to the question of whether there is a difference in reversal rates as well, both across time and in general.  For more on that, the next post in this series will focus on gender.

Acknowledgements: I would like to thank Victoria Henige for asking the questions that led to this series of posts and Professor Lee Epstein for sharpening my thinking on it. 

Read more at The Juris Lab …