Operations & Systems

Simpson's Paradox in Business Reports: Inspect the Group Mix Before Explaining a Reversal

Both request groups show higher completion rates. The overall completion rate falls.

The totals can be correct.

Inspect how much of each group enters the denominator before explaining what changed.

Penn State's lesson on three-way tables describes Simpson's paradox as a reversal between marginal and conditional associations. Its discussion also separates association from causation.

The fictional report below creates that reversal with service-request counts. Every record group, period, classification rule, and result is synthetic. The worksheet proposes a review method. It describes no organization or demonstrated process improvement.

Hold the Reporting Rules Still

One observation represents one eligible request received during a fictional ten-business-day intake period. Period A and Period B are separate cohorts. Every request has reached its assessment deadline before its period's extract closes.

Completion means first completion within five business days of receipt. A later completion remains outside the success count for that deadline measure. Repeated messages and status events do not create additional requests.

At intake, each request receives one mutually exclusive complexity label. Simple means the fictional checklist requires one standard review. Complex means it requires that review plus an external-evidence review. Preserve that dictionary in both periods.

The label records required review steps. It does not measure employee effort, customer value, or the quality of the completed work. Calling one group complex supplies no explanation for an individual request's outcome.

This example contains no duplicate requests, missing labels, unknown outcomes, transfers between periods, or cancellations. All eligible requests enter their assigned period's denominator once. Every outcome is either complete by deadline or not complete by deadline.

Real extracts need those checks before comparison. If a category rule changes, reconcile the definitions before presenting the periods as comparable. The dashboard-reconciliation method provides a separate procedure for that task.

Read Every Count Before the Rates

The following table contains the entire synthetic report. The not-complete column preserves the other side of each denominator.

Period Request group Complete by deadline Not complete by deadline Eligible requests Completion rate
A Simple 90 10 100 90%
A Complex 20 30 50 40%
A All requests 110 40 150 73.3%
B Simple 19 1 20 95%
B Complex 65 65 130 50%
B All requests 84 66 150 56%

Start with Period A. Simple requests reconcile as 90 + 10 = 100. Complex requests reconcile as 20 + 30 = 50. Together, 110 complete and forty do not complete by deadline, among 150 eligible requests.

Period B has nineteen completions and one deadline miss among twenty simple requests. Its complex group has sixty-five completions and sixty-five deadline misses among 130 requests. Together, eighty-four complete and sixty-six miss the deadline, again among 150 requests.

The all-requests rows summarize the groups. Do not add them to the group rows when totaling an extract. Doing so would count every request twice.

Within simple requests, the rate rises from 90 / 100 = 90% to 19 / 20 = 95%. That is an increase of five percentage points.

Within complex requests, it rises from 20 / 50 = 40% to 65 / 130 = 50%. That is an increase of ten percentage points.

The pooled comparison goes the other way. Period A's 110 / 150 is approximately 73.3%. Period B's 84 / 150 is 56%. The pooled rate falls by approximately 17.3 percentage points.

Keep the fractions available. The repeating decimal in 110 / 150 is rounded for display. Calculate the difference from the unrounded fractions, then round the displayed result.

See the Reversal Without Hiding the Denominators

This chart uses a common zero-to-one-hundred-percent scale. Read each pair with the counts in the table. The bars show descriptive shares, not uncertainty or a causal effect.

Synthetic completion rates by period and request group Simple requests rise from 90 to 95 percent. Complex requests rise from 40 to 50 percent. All requests fall from about 73.3 to 56 percent. Blue bars represent Period A. Orange bars represent Period B. Every pair uses a common scale from zero to one hundred percent. Completion within five business days, synthetic data Period APeriod B SimpleComplexAll requests 0%50%100% 90%95%40%50%73.3%56%
Both within-group comparisons rise. The pooled comparison falls. All underlying counts appear in the preceding table.

The simple-request completion count also falls, from ninety to nineteen. That count change coexists with a higher simple-request completion rate because the simple denominator falls from one hundred to twenty.

Keep counts and rates together when a reader needs workload context. A rate alone cannot tell the reader how many requests contributed to it. Here, both total denominators are 150, while their internal composition changes.

Rebuild the Aggregate With Its Group Weights

Each period's group weight is that group's eligible requests divided by the period's eligible total. The weights describe composition before they describe outcomes.

Period Simple weight Complex weight Weight total
A 100 / 150, approximately 66.7% 50 / 150, approximately 33.3% 100%
B 20 / 150, approximately 13.3% 130 / 150, approximately 86.7% 100%

Period A contains mostly simple requests. Period B contains mostly complex requests. In both periods, the complex group has the lower completion rate.

Reconstruct Period A from its group rates and exact weights: (90 / 100) × (100 / 150) + (20 / 50) × (50 / 150) = 110 / 150.

Reconstruct Period B the same way: (19 / 20) × (20 / 150) + (65 / 130) × (130 / 150) = 84 / 150.

Those identities reproduce the pooled rates. They also expose how different weights can reverse the direction seen inside each group. The weighted-average guide explains the pooling arithmetic in more detail.

Do not take the unweighted average of the two group percentages and call it the actual overall result. For Period A, (90% + 40%) / 2 = 65%. For Period B, (95% + 50%) / 2 = 72.5%.

Those averages assign equal weight to the groups. Neither matches the observed mix in its period. They answer an equal-weighted comparison question that must be named separately.

Choose the Question Before Choosing the View

For “what share of all eligible requests completed by deadline in each period,” report the pooled rates. They describe the actual synthetic cohorts under the stated eligibility rule.

For “how do the observed completion shares compare within each intake group,” show the stratified rates and counts. They describe the simple and complex groups separately.

Both questions are valid descriptive tasks. Select the task before deciding which result belongs in the headline. Do not search for a grouping that produces the preferred direction.

A third question can hold the weights fixed for a descriptive comparison. State the chosen reference mix and retain the observed pooled rates beside the result.

For example, apply Period A's exact mix to Period B's group rates: (95% × 100 / 150) + (50% × 50 / 150) = 80%.

Under those reference weights, Period A remains approximately 73.3% and Period B becomes 80%. The calculation answers how the reported group rates combine under one chosen composition.

It does not estimate what Period B would have achieved if its real workload had changed. Changing the workload could change the group rates themselves. Label the eighty percent result “descriptive rate using Period A weights.” Keep it out of the actual completion-count column.

Before comparing a chosen reference mix, ask who selected it and why. Period A is convenient here because its group counts are available. That convenience gives it no special authority as a target workload. A planning question may require a different documented mix.

Keep the reference weights in a separate column from actual counts. Record whether the weights sum to one and whether each required group has a defined rate. If one rate is missing, leave the combined reference result unresolved. Do not substitute another period's rate without naming the assumption and the question it changes.

Keep Causal Claims Outside the Comparison

A reversal is a reason to inspect composition. It does not by itself identify the cause of the composition change or the cause of either group rate.

Penn State's discussion of marginal and conditional associations provides the statistical boundary. Accounting for a third variable can change an association. Association alone does not establish causation.

The synthetic table contains period, intake group, and deadline outcome. It contains no randomized assignment, staffing record, intervention history, quality assessment, or measurement audit. It cannot establish that a new process improved either group.

Even the simple category may contain different work across periods. A stable label does not guarantee identical requests. The smaller Period B simple group also contains fewer observations. This descriptive worksheet provides no uncertainty calculation.

List alternative explanations as questions requiring evidence. Did intake classification change? Did the deadline clock change? Did the mix within each category shift? Were outcomes recorded consistently? Did a backlog rule alter eligibility?

Do not turn any of those questions into a finding until its source is inspected. The existing scatter-plot article applies the same separation between a visible association and its explanation to paired numerical measurements.

Copy the Reversal-Review Worksheet

This worksheet is a proposed review aid. Fill it from counts before writing an interpretation.

Field Filled synthetic example
Overall comparison 110 / 150, about 73.3%, versus 84 / 150, 56%
Strata Simple and complex, classified at intake under one dictionary
Counts A: 90 / 100 simple and 20 / 50 complex; B: 19 / 20 simple and 65 / 130 complex
Weights A: 100 / 150 and 50 / 150; B: 20 / 150 and 130 / 150
Within-group results Simple rises five percentage points; complex rises ten
Composition change Complex requests rise from about 33.3% to about 86.7% of eligible requests
Alternative explanations Classification, clock, within-group mix, outcome recording, and eligibility require evidence
Bounded conclusion Both observed group rates rise while the observed pooled rate falls; composition differs

Attach the eligibility rule, extract dates, grouping dictionary, and unresolved evidence requests. Name who can inspect each source. Keep the calculation reproducible without exposing private request details.

If a group has no eligible observations in a period, leave its rate undefined. Do not replace an absent denominator with zero percent. A blank subgroup comparison cannot support a statement that both group rates moved together.

If unknown outcomes exist, disclose their counts and the denominator rule before comparing periods. This example excludes that uncertainty by construction. A real report must preserve it.

Keep the original counts. State the reporting question. Explain only what the comparison supports.

Sources

  • Penn State STAT 504, Lesson 5: Three-Way Tables, Types of Independence. Marginal and conditional associations, Simpson's paradox, and the distinction between association and causation. Checked 2026.10.28. The service-request example, chart, weights, reference-mix calculation, and worksheet are original synthetic editorial material. No causal effect or statistical significance is estimated.

Topic map: Business reporting and statistics

Articles in this cluster, then the 3 clusters that share the most topics with it.

This cluster: Business reporting and statistics

Neighbor: Spreadsheets, CSV files, and data handoffs

Neighbor: Records, access, and continuity

Neighbor: APIs, webhooks, and automation