Inside the Opinions — Judicial Text & Sentiment Measures

01 · Start here · no other reading required

What the opinions actually look like

This page lets you read real federal appeals-court opinions and see, next to each one, the scores our automated text methods gave it. It is self-contained — you don't need any slide deck or other document open to use it. Pick a case, read it, and judge for yourself whether the scores match what's on the page.

The basics

Why does this page exist?	We've built automated ways to score an opinion's stance toward business and regulation. Before trusting them, we want to read actual opinions and ask: are the scores capturing what matters in the text — and is there a better way to do the text analysis? That's the whole task here.
What is a "judicial opinion"?	The written decision a court issues after deciding a case. It states who won and walks through the legal reasoning. When three judges hear an appeal, one writes the majority opinion — that authored text is what we show and score.
Which court? Which judges?	The U.S. Courts of Appeals — the federal "circuit" courts, one level below the Supreme Court. The authors are federal circuit judges (occasionally a district judge sitting by designation).
What is the full dataset?	About 376,000 majority opinions, 1891–2013, drawn from the project's text corpus. The Corpus over time tab summarizes the whole thing.
What are these 18 cases?	A small, hand-picked illustrative sample (not random) — chosen to span different eras, lengths, and especially the cases where our measures agree vs. disagree. They are examples to read, not a representative sample to draw conclusions from.
What do we want from you?	Read a few. Tell us where the scores feel right, where they feel wrong, and whether the text suggests a better way to measure a judge's lean. That feedback is the point.
Is there a bigger document?	No — this page stands on its own. Start here.

What you'll see attached to each opinion

Every case shows the same scores at the top. In plain terms:

+1/−1 LLM ruling	An AI model's read of who the holding favors — business (+1) vs. worker / consumer / government (−1).
+1/−1 LLM reasoning	The same model's read of the tone and framing of the writing — separate from who won. Ruling and reasoning are scored independently and can point opposite ways.
Doc2Vec sentiment & similarity	An older, word-pattern method's read of the same opinion. "Sentiment" is its positive/negative word-tone near business language; "similarity" is how business-focused the text looks.
Voting outcome (regulatory cases only)	For the six Labor/EPA cases: did the judge rule for or against the government regulator. This is the outcome variable from the underlying study.

Two optional highlight layers

Inside each opinion you can switch on highlights. Both are off by default; they're reading aids, not required.

GPT-cited. The exact phrases the AI model quoted as its evidence when it scored the case.

Claude's diagnostic read. Phrases that an AI (after the fact, for this page) thinks look diagnostic. A reading aid only — not how any score was actually computed.

Next step: open the Case gallery, pick any case from the list, and read it. The cases are grouped: general business opinions from 1910 and 1940, then six regulatory (Labor/EPA) cases that also carry the for/against-regulator vote. If you want a reminder of what the basic regression results look like first, jump to Where things stand.

Optional: exactly how the 18 cases were chosen

Set A — twelve general business opinions (1910 & 1940). For each era, one opinion in each of four profiles, plus a short and a long outlier:

LLM pro-biz · Doc2Vec agrees	Both measures read it as favorable to business.
LLM anti-biz · Doc2Vec agrees	Both read it as protective of workers / consumers / the public.
diverge LLM pro-biz · Doc2Vec negative	LLM calls the ruling pro-business; Doc2Vec's word-tone is negative.
diverge LLM anti-biz · Doc2Vec positive	LLM calls it anti-business; Doc2Vec's word-tone is positive.
Short / long	Length outliers — a few-sentence per curiam vs. a multi-thousand-word opinion.

Set B — six regulatory cases vs. the vote. The for/against-regulator outcome only exists for Labor/EPA cases (~1934+), so there are no 1910 regulatory cases. These span agreement and divergence between the LLM read and the recorded vote, and deliberately include a union and an environmental challenger — where the vote and the LLM most cleanly come apart (a union or green group beating the agency counts as "anti-regulator" by the variable but reads as anti-business to the LLM).

One caveat on the vote. The recorded vote is coded from the case disposition. For cases where the government is the petitioner (e.g. the NLRB seeking to enforce an order), that coding has known issues — it can mark the government as "losing" even where the court enforced its order. So some apparent LLM-vs-vote divergences reflect the measure, not the judge. See the “How scores are made” tab, § 3, for the full explanation.

02 · Read the full text

Case gallery

Pick a case from the lists below, then read it — the scores sit at the top, the full opinion is underneath. The cases come in three groups, shown as the small headers in the list:

General business · 1910 and General business · 1940 — everyday business opinions from each era, used to compare the LLM against Doc2Vec.
Regulatory cases · Labor/EPA — six cases that also carry the "voting against the regulator" outcome and the challenger's identity, so you can see where the LLM agrees or diverges with the recorded vote.

Reading a card. The bold pro-business / anti-business label is the LLM's read of the opinion's stance — the other scores (Doc2Vec, and the regulator vote) appear once you open the case. The AGREE / DIVERGE tag compares two measures: for general cases it's LLM vs. Doc2Vec; for regulatory cases it's LLM vs. the recorded vote. The remaining text is opinion length (short / modal / long and word count) or, for regulatory cases, the challenger type and whether the vote went anti-reg / pro-reg.

03 · The corpus, zoomed out

How the text changes over time

Computed over all opinions in the corpus (1891–2013), one streaming pass. These describe the raw material every measure is built on — before any scoring.

Opinion length by decade

Average words per majority opinion. Opinions shrink through mid-century, then grow sharply after 1970.

Sentence length by decade

Average words per sentence — a rough proxy for prose style. A steady decline from ~36 to ~21: sentences get shorter and more numerous.

How business is named, by decade

Occurrences per 10,000 words. Early opinions overwhelmingly say company; corporation peaks in the 1930s; regulation is essentially absent before 1940, then climbs. The vocabulary for talking about business is not stable across the century — something any text-similarity measure trained on one era inherits.

04 · From text to number

How each score is made

Three measurements appear in this project. Each turns opinion text (or a case disposition) into a number a different way.

1 · The Doc2Vec business-sentiment score

This is the construct from Ash, Chen & Galletta (2021), "Measuring Judicial Sentiment" (Economica) — the earlier paper this project originally extended, before the regulatory-voting work. Doc2Vec embeds every sentence; for each sentence we take its cosine similarity to a business word list (that is simi) and its position on a positive–negative attribute axis, then aggregate to the case:

sentiment_case = Σ_{d ∈ case} ( S_d × W^business_d )

S_d  = sentence d's positive-minus-negative attribute loading
W_d  = sentence d's similarity weight to the "business" target
simi = average cosine similarity of the case to "business"

So sentiment can be positive even when a case rules against a business (if the language near business terms is positively toned), which is exactly where it diverges from the LLM's ruling score.

2 · The LLM business-sentiment score

A two-stage GPT-4o-mini pipeline, tuned against a 100-case Claude ground truth. Stage 1 asks whether the case is business-relevant; Stage 2 scores two independent dimensions:

Ruling direction — the objective direction of the holding: +1 favors business, 0 procedural/mixed, −1 favors worker / consumer / government.
Reasoning direction — the tone of the opinion's reasoning, independent of who won: +1 pro-business framing, −1 protective framing, 0 neutral doctrinal analysis (the most common).

The model also returns a 1–10 confidence, a short justification, and up to three verbatim quotes — the "GPT-cited" highlight layer in the gallery. Every gallery case shows these exact outputs.

A worked example: where Doc2Vec and the LLM disagree

3 · Voting against the regulator

This outcome comes from Ash, Chen & Naidu (2026), "Ideas Have Consequences: The Impact of Law and Economics on American Justice" (QJE) — it is the paper's headline outcome variable, not something built for this page. Unlike the two above, it's not a text measure — it reads the case disposition.

The government is flagged as a party; govt_wins is built from who petitioned and whether the court affirmed or reversed; the regulator outcome is its complement on Labor/EPA cases.

# the disposition-based outcome (simplified)
govt_wins = (govt_respondent & Affirmed) | (govt_petitioner & Reversed)
govt_wins = 1 - govt_wins        # if the voting judge dissented
govt_lose = 1 - govt_wins        # when govt is a party
y_labor_epa_lose = govt_lose     # on Labor/EPA cases only

The six regulatory cases in the gallery (the bottom group) carry this outcome; the twelve general business opinions do not, since the variable only exists for Labor/EPA cases (~1934+). On each regulatory case the gallery shows the coded vote next to the LLM read, plus the raw disposition signals (govt role, affirmed/reversed) so you can judge the coding for yourself.

One thing to note about this variable. About 82% of Labor/EPA votes are stays / dismissals / remands, and the QJE coding treats all of these as the government losing the case — government losing is the default residual category. As a result, the headline statistic that the government loses 89% of these cases is partly driven mechanically by that default. This may be worth revisiting if we use this variable seriously. While investigating this, I also noticed an issue for the 1,099 votes where the government was the petitioner and the disposition was Affirmed (e.g., the NLRB seeking to enforce an order, and the court enforces it — arguably a government win, but coded as a loss). I've fixed those in an alternate version of the variable; the regression results are fairly similar to sticking with the original QJE coding.

All eighteen opinions, scores, and corpus statistics are reproduced from the project dataset (opinion_texts.parquet, llm_sentiment_final.parquet, the 1891-extended Doc2Vec file, and overlap_classified.csv for the regulatory vote). Doc2Vec values are the 1891-extension reconstruction. Built 2026-05-20.

05 · A quick reminder of the regressions

Where things stand

We've tested whether father's wealth — the project's predictor of interest — shifts each of our outcome measures. Nulls across the board, pooled and weighted. Tables below are computed from the dataset linked at the bottom of this tab so you can reproduce or modify them in Stata.

The basic specification

The same reduced-form regression runs against each outcome:

reghdfe y_outcome father_wealth_100pp                             ///
        republican female BirthYear                              ///
        [aw=wjt],                                                  ///
        absorb(circuit##year) cluster(jid)

father_wealth_100pp is the judge's father's predicted-income percentile rescaled to a 0–1 scale, so the coefficient reads directly as "effect of moving from the 0th to the 100th percentile." It is the same variable as father_wealth (the QJE's rank_predinc_farmROI, 0–100 scale), just divided by 100 for interpretability. wjt stands for Weight per Judge-Year (W subscripted by j for judge and t for year) — defined as 1 / (# cases that judge had in that year), so each judge-year contributes equal weight regardless of caseload. The judge-year is the unit Daniel's QJE paper treats as equal-weighted.

What each column dependent variable means

Column	What it is	Sample
LLM rul dir	GPT-4o-mini's read of the opinion's ruling direction, −1 / 0 / +1 (anti / neutral / pro-business).	LLM
LLM rul bin	Binary version of the above: 1 if the ruling is pro-business, 0 otherwise.	LLM
LLM rea dir	GPT-4o-mini's read of the tone/framing of the writing — scored independently of who won.	LLM
LLM rea bin	Binary version of reasoning direction.	LLM
D2V sent	Ash–Chen–Galletta Doc2Vec business-sentiment score (positive–negative word-tone near business language).	both samples
D2V simi	Doc2Vec cosine similarity of the opinion to the "business" target embedding.	both samples
y_lose (orig)	The QJE outcome `y_labor_epa_lose` as originally coded: 1 if the judge voted against the regulator (Labor / EPA).	Labor/EPA
y_lose (Alt)	The same outcome with one coding fix applied — toggles y_lose for the 1,099 votes where the government was the petitioner and the disposition was "affirmed" (an NLRB enforcement order being upheld means the government won, not lost). See § 3 of "How scores are made" for the full reasoning.	Labor/EPA

Reproduce or modify these tables. One dataset, one do-file:

analysis_sample.dta (12 MB, 133,212 vote-level rows) — the union of the project's two existing analysis samples (LLMSentiment's analysis_sample.dta + RegulatorVoting's analysis_sample.dta) with Doc2Vec sentiment + similarity merged in by caseid. Unused QJE-archive columns dropped to stay under Cloudflare's 25 MB limit; row counts and key variables match the originals exactly. Outcomes are populated on their natural sample — LLM measures on the 115K LLM rows, y_labor_epa_lose on the 23K Labor/EPA rows, Doc2Vec on both.
coauthor_table.do — loads the dataset, builds in-sample WJT (separately for each analytic sample, via preserve / restore), runs the same spec on every outcome, exports the LaTeX tables. Required Stata packages: ssc install reghdfe ftools estout.

Output files (for download): pooled .tex, WJT .tex, summary stats .tex, CSV of point estimates, Stata log.

How we get to the regression samples

The corpus the Corpus over time tab summarizes is ~376,000 majority opinions (1891–2013). The two analytic samples below sit on that corpus differently:

Cols 1–6 (text measures) work at the author level — one row per case, attached to the judge who wrote the majority opinion. The text scores (LLM and Doc2Vec) are case-level; we credit them to the authoring judge and use that judge’s father-wealth as the predictor.
Cols 7–8 (y_lose) work at the panel-vote level — one row per judge per case, with the case-level disposition attached to each panel member and flipped for any judge who dissented. This matches the QJE paper's approach.

Restriction	Rows remaining
LLM analytic sample · author-level · one row per case (cols 1–6)
All majority opinions in the corpus	375,794
+ Authoring judge has FJC ID and father-wealth match (Santiago’s census linkage)	190,255
+ Case is business-relevant per LLM Stage 1 → LLM analytic sample	115,613
Labor/EPA analytic sample · panel-vote level · multiple rows per case (cols 7–8)
All panel-vote observations in the corpus (~3 judges per case)	~ 1,155,000
+ Labor/EPA case (NLRB / OWCP / FLRA / OSHA / EPA; 1934+)	~ 33,900
+ Judge has father-wealth match → Labor/EPA analytic sample	23,156

On the author vs. panel-vote distinction. A federal appeals case is decided by a 3-judge panel, but one of those judges writes the majority opinion. For the text measures (cols 1–6) we use the writing judge — one row per case, with that judge’s father-wealth as the predictor. For the vote outcome (cols 7–8) we follow the QJE paper and use all panel members — ~2.2 judges per case (panels are sometimes smaller; per-curiam votes are excluded). Same predictor (father’s wealth), different unit of observation. The dataset file unions both: 133,212 total rows, with the LLM and Doc2Vec columns populated on the 115K author-level rows and y_lose populated on the 23K panel-vote rows.

Why 190K → 115K on the LLM side? Doc2Vec scores every opinion case-wide, so Doc2Vec values exist for all 190,255 cases whose authoring judge has a father-wealth match. But the LLM ruling and reasoning scores were only computed on cases that LLM Stage 1 flagged as business-relevant (Stage 2 wasn’t run on the ~46% of cases Stage 1 marked not-business-relevant). So ruling_direction and friends only exist on the 115,613 business-relevant subset. We restrict the Doc2Vec columns (5 and 6) to the same 115K so cols 1–6 sit on the same observations — apples-to-apples across the text measures. If we ran Doc2Vec on the full 190K instead, those columns’ N would be ~190K and the coefficients would be similar but not identical.

A coding caveat on the y_lose columns: both versions inherit the QJE convention that treats stays / dismissals / remands as the government losing — see the “How scores are made” tab, § 3, for the full explanation of why the column-7 mean of 0.89 is partly mechanical.

Summary statistics

Variable	Mean	SD	N
Outcomes · each populated only on its natural analytic sample
LLM ruling direction	0.063	0.981	115,612
LLM ruling binary	0.515	0.500	115,612
LLM reasoning direction	−0.129	0.515	115,612
LLM reasoning binary	0.076	0.265	115,612
Doc2Vec sentiment (business)	0.114	0.116	132,552
Doc2Vec similarity (business)	0.001	0.030	132,552
Vote against regulator (Labor/EPA)	0.894	0.308	23,156
Predictor & controls · populated on all rows
Father's wealth (0–1 percentile)	0.764	0.258	133,212
Republican	0.527	0.499	133,212
Female	0.026	0.160	133,212
Birth year	1903	27.2	133,212

Computed from analysis_sample.dta via tabstat over the full 133,212 rows. The LLM measures live on the 115K LLM-bridge analytic sample (business-relevant cases); y_labor_epa_lose on the 23K Labor/EPA analytic sample; Doc2Vec on all rows where the underlying case is in the 1891-extension corpus.

Panel A · Main results, pooled (unweighted)

	(1) LLM rul. dir.	(2) LLM rul. bin.	(3) LLM rea. dir.	(4) LLM rea. bin.	(5) D2V sent.	(6) D2V simi.	(7) y_lose (orig)	(8) y_lose (Alt)
Father's wealth	−0.001	−0.003	0.008	0.005	0.001	0.000	0.017	0.020
	(0.017)	(0.009)	(0.010)	(0.006)	(0.004)	(0.000)	(0.011)	(0.012)
Republican	0.013	0.008	0.027	0.012	−0.003	0.000	0.003	0.003
	(0.010)	(0.005)	(0.006)	(0.003)	(0.002)	(0.000)	(0.005)	(0.005)
Female	0.015	0.013	−0.013	0.002	0.003	0.000	0.017	0.017
	(0.042)	(0.022)	(0.022)	(0.011)	(0.008)	(0.001)	(0.013)	(0.014)
Birth year	0.001	0.000	0.001	0.000	0.000	0.000	0.000	0.000
	(0.001)	(0.000)	(0.000)	(0.000)	(0.000)	(0.000)	(0.000)	(0.000)
Sample / fit
Observations	115,605	115,605	115,605	115,605	115,604	115,604	23,128	23,128
R²	0.026	0.025	0.028	0.022	0.172	0.023	0.127	0.107
Judges (clusters)	1,207	1,207	1,207	1,207	1,207	1,207	807	807

Panel B · Main results, WJT-weighted (judge-year equal weight)

	(1) LLM rul. dir.	(2) LLM rul. bin.	(3) LLM rea. dir.	(4) LLM rea. bin.	(5) D2V sent.	(6) D2V simi.	(7) y_lose (orig)	(8) y_lose (Alt)
Father's wealth	−0.021	−0.016	−0.008	−0.010	0.003	0.000	0.013	0.017
	(0.023)	(0.012)	(0.013)	(0.007)	(0.004)	(0.001)	(0.013)	(0.014)
Republican	0.002	0.002	0.018	0.009	−0.005	0.000	0.008	0.009
	(0.013)	(0.006)	(0.007)	(0.004)	(0.002)	(0.000)	(0.006)	(0.006)
Female	0.007	0.007	−0.015	−0.002	0.008	0.001	0.015	0.014
	(0.044)	(0.022)	(0.021)	(0.010)	(0.009)	(0.001)	(0.020)	(0.021)
Birth year	0.001	0.000	0.001	0.000	0.000	0.000	−0.001	−0.001
	(0.001)	(0.000)	(0.000)	(0.000)	(0.000)	(0.000)	(0.000)	(0.000)
Sample / fit
Observations	115,605	115,605	115,605	115,605	115,604	115,604	23,128	23,128
R²	0.046	0.044	0.044	0.041	0.183	0.045	0.168	0.151
Judges (clusters)	1,207	1,207	1,207	1,207	1,207	1,207	807	807

SEs in parentheses, clustered at judge level. All columns include circuit×year fixed effects. Cols 1–6 use the LLM analytic sample; cols 7–8 use the canonical Labor/EPA sample. Column 7 (original coding) reproduces the QJE-style headline (+0.017 pooled, +0.013 WJT — already null under WJT). Column 8 applies the coding fix described in the “How scores are made” tab (toggles y_lose for the 1,099 votes where the government was petitioner and disposition was affirmed) — the coefficient nudges up to +0.020 pooled (marginal, p ≈ 0.09) and +0.017 WJT (still null).

Status, per thread

Phase 0 · Doc2Vec sentiment extension (1891)	done null on wealth; strong judge signature (ICC 0.16)
Thread 1 · expanded regulator sample (31 agencies)	done coef shrinks to ~0
Thread 2 · extended bio file (Santiago)	done sign flip is spurious (driven by 31 reassigned judges)
Thread 3 · LLM bridge (375,794 cases, ~$177)	done null on wealth
Coding & weighting sensitivity (QJE outcome)	done null under every permutation under WJT
Slide deck on LLM bridge	done available on the documentation site
Measurement-error correction (SSRN 4913179)	open Daniel's suggestion, not yet pursued
Restrict QJE outcome to business-as-party cases	open the variable currently pools cases brought by unions, environmental groups, and individuals — a tighter "business vs. regulator" outcome would be a cleaner test
Independent code audit by a co-author	open especially on the Stata/Python pipeline that builds y_labor_epa_lose

Regression tables computed by coauthor_table.do on analysis_sample.dta — both downloadable above. Last computed 2026-06-01.