Chi-Square Test Explained: Formula, How to Calculate It, and When to Use It

The Millionaire's Guessing Game

Every weekend, someone looks at a lottery draw and is certain the machine is rigged. "Six even numbers? Impossible." It is a feeling almost everyone has had, and it is exactly the instinct the chi-square test was built to discipline. Karl Pearson introduced it in 1900 to answer a question that sounds simple and is anything but: when do observed counts disagree with expected counts enough to be suspicious?

The chi-square test is the most widely used tool in statistics for data that comes in counts — categories, not measurements. Is this die fair? Do these colors appear in the advertised proportion? Did this website redesign change click behavior? Is this drug's effect independent of the patient's age group? All of these questions reduce to one move: compare what you observed against what you expected, and ask whether the gap is bigger than random noise would usually produce.

This guide explains what a chi-square test does, the formula behind it, how to run one by hand, and — crucially — which of the three main flavors you need, because picking the wrong one is one of the most common errors in applied statistics.

What the Chi-Square Test Actually Does

A chi-square test asks a deceptively narrow question: do observed categorical counts match expected counts? It works whenever your data is a set of counts sorted into categories, and it answers whether the mismatch between observation and expectation is bigger than sampling luck would typically explain[khan-chisq].

Imagine rolling a six-sided die 60 times. A fair die should produce about 10 of each face. You get: 8, 12, 9, 11, 10, 10. Nothing extreme — the counts wander a little, as counts always do. Now imagine you instead get: 2, 3, 4, 5, 22, 24. Those counts are wildly off, and you would rightly suspect the die. The chi-square test quantifies exactly this judgment: it turns the whole pattern of discrepancies into a single number that says "this much mismatch would happen by chance X% of the time."

The magic ingredient is the expected count. The test never works on raw observations alone — it always needs a hypothesis that predicts what counts should be. For the die, the hypothesis is "fair die," predicting equal counts. For a genetics experiment, the hypothesis is a Mendelian ratio. For a survey, the hypothesis is equal preference. The expected counts encode the null hypothesis, which is why getting them right matters as much as recording the observations correctly[nist-chisq].

The Chi-Square Formula

The chi-square statistic is a sum over categories of the squared difference between observed and expected, standardized by the expected count:

Where $O_i$ is the observed count in category $i$, $E_i$ is the expected count in that category, and $k$ is the number of categories.

The structure is worth understanding rather than memorizing. Each term does three things at once:

• Squaring removes the sign — a shortfall and an excess both contribute positively, so deviations do not cancel out.
• Dividing by E standardizes — an excess of 5 matters more when only 10 were expected (50% off) than when 1,000 were expected (0.5% off).
• Summing combines all categories into one total measure of discrepancy.

The result is a number that is small when observation tracks expectation and large when it does not. The question is always: how large is "too large to be chance"? That boundary depends on the degrees of freedom, which is why the chi-square test has its own distribution rather than the normal curve[wikipedia-chisq].

Degrees of Freedom, Explained Simply

Degrees of freedom (df) are the chi-square test's control knob, and they are the source of endless confusion — largely because they work differently depending on which chi-square test you run.

For the goodness-of-fit test (one set of categories against a fixed expectation), the degrees of freedom are:

Why minus one? Because the expected counts are constrained to sum to the total number of observations. Once you know the total and all but one expected count, the final one is forced. That single constraint costs you one degree of freedom. With four categories, you have three free choices, so df = 3.

For a test of independence on a two-way table with R rows and C columns, the degrees of freedom are:

A 2×2 table — say, "clicked vs did not click" by "red button vs blue button" — has 1 degree of freedom. That is why the 2×2 case gets special treatment everywhere in statistics, including its own continuity corrections.

Why does df matter? Because the chi-square distribution gets sharper as df grows. With few degrees of freedom the curve is heavily skewed and the critical values are high; with many degrees of freedom it approaches a normal-shaped curve. A chi-square of 7.81 is significant at 3 df but unremarkable at 10 df — the same number means different things at different df, which is why every chi-square result must report its degrees of freedom alongside the statistic[stathow-chisq].

A Worked Example: Goodness of Fit by Hand

Let us run a goodness-of-fit test on a concrete dataset so the machinery is visible. A shop claims that customer colors in a sample of 100 products come in the proportions 25% red, 25% blue, 30% green, 20% yellow. The observed counts are: red 20, blue 30, green 34, yellow 16.

Step 1 — the expected counts. 25% of 100 = 25 red, 25 blue, 30 green, 20 yellow. Observed and expected:

Step 2 — the per-category contributions. For each color, compute (O − E)² / E:

• Red: (20 − 25)² / 25 = 25 / 25 = 1.00
• Blue: (30 − 25)² / 25 = 25 / 25 = 1.00
• Green: (34 − 30)² / 30 = 16 / 30 ≈ 0.53
• Yellow: (16 − 20)² / 20 = 16 / 20 = 0.80

Step 3 — sum the contributions.

Step 4 — interpret. With df = 4 − 1 = 3, the 5% critical value is 7.81. Our 3.33 falls well below it, so there is no evidence the observed colors depart from the advertised proportions. The Chi-Square Calculator does the whole calculation instantly — you enter observed and expected lists and it returns χ² and df directly.

The per-category contributions matter as much as the total. In this example blue is the category driving the result (it alone accounts for 1.00 of the 3.33). If a chi-square ever rejects the null, looking at which category contributed the most tells you exactly where reality broke from expectation — a chi-square of 10 could mean one wildly off category or a uniform drift across all of them, and those two stories lead to very different follow-up actions.

Three Flavors of Chi-Square: Which One Do You Need?

There is not one chi-square test — there are three closely related ones, and using the wrong version invalidates the result. The distinction depends entirely on how the data is structured.

Goodness of fit. You have one set of categories and a fixed expectation — from theory, an advertised proportion, or a known standard. Example: does a die behave fairly? Does the observed color mix match the stated mix? One row of counts against a fixed hypothesis[nist-chisq].

Test of independence. You have two categorical variables and want to know whether they are related. Example: is click behavior independent of button color? Is a disease outcome independent of whether a patient got the treatment? The data is a two-way contingency table, and the expected counts are derived from the row and column totals — they are not fixed in advance. This is the test people mean when they say "chi-square test of independence"[jmp-chisq].

Test of homogeneity. You have several groups and want to know whether they share the same distribution. Example: do men and women prefer the same three smartphone brands? The setup looks identical to independence — a two-way table — and the arithmetic is the same, but the question is subtly different: not "are these two variables related" but "do these populations follow the same distribution."

The rule of thumb: one list of counts against a fixed expectation → goodness of fit; a two-way table asking "are these related?" → independence; a two-way table asking "do these groups match?" → homogeneity. The formula is the same; the meaning of the expected counts and the degrees of freedom is what changes.

How the Expected Counts Are Built in a Test of Independence

The trickiest part of a chi-square test of independence is where the expected counts come from, because they are not given in advance — they are manufactured from the data itself. This is also where the most fundamental misunderstanding lives.

In a two-way table, each cell's expected count under the assumption of independence is:

Intuitively: if the two variables were unrelated, each cell should hold a share of its row total proportional to its column total. The row and column totals are real — they are the actual counts — but the individual cells are what independence would predict. The chi-square test then asks whether the actual cells deviate from those predicted values more than chance would explain[wikipedia-chisq].

Concretely: suppose 200 people are shown a red or blue button, and 120 click. If color made no difference, the clicks would be distributed across both colors in proportion to how many people saw each color. A large deviation from that proportionality is evidence that color does matter. The degrees of freedom — (R−1)×(C−1) — reflect how many cells are free once the row and column totals are fixed. In a 2×2 table, fixing the totals pins down three cells and leaves only one free, which is why a 2×2 chi-square has exactly 1 df.

The Chi-Square Distribution and Critical Values

Once you have χ² and df, you still need to know whether the number is big. That is the job of the chi-square distribution — a family of curves, one for each degree of freedom, that describes how large χ² gets when the null hypothesis is true and the discrepancies are pure noise.

Key critical values at the conventional 5% level:

Reading the table is simple: if your computed χ² exceeds the critical value for your df, you reject the null hypothesis of a good fit. If it falls below, the data do not contradict the expectation strongly enough to reject. Because the chi-square statistic is a sum of squared terms, it is always positive, and the distribution is skewed right — small values are common, huge values are rare — which is exactly the shape you want for a "surprise" detector[stathow-chisq].

Why the Chi-Square Test Matters Everywhere

Chi-square tests are woven through the most common decisions made with categorical data. In genetics, they test whether observed offspring ratios match Mendelian predictions — the original motivation for Pearson's work, and still standard in cross-breeding experiments. In market research, they test whether preferences differ across demographics. In quality control, they test whether defect rates differ across production shifts or machines. In medicine, they test whether outcomes are independent of treatment, age, or other risk factors.

The test also anchors one of the most important ideas in all of statistics: a result is only meaningful relative to the expectation it is testing against. A chi-square result is always "observed counts versus expected counts under hypothesis H" — and that framing forces you to state H explicitly before you look at the data. That discipline is why the chi-square test, despite being over a century old, still underlies modern A/B testing and clinical trial analysis: whenever the world produces counts, chi-square is waiting to ask whether they are surprising[khan-chisq].

Chi-Square vs the T-Test: Counting vs Measuring

Chi-square and the t-test are both workhorses of significance testing, but they answer different questions for different data types — and choosing between them is one of the most important decisions in applied statistics.

The t-test works with measured (continuous) data: heights, times, scores, prices. It asks whether the means of two groups differ. The chi-square test works with counted (categorical) data: how many fell into each bucket. It asks whether the counts deviate from expectation. If your data is "87 people chose A, 113 chose B," that is chi-square territory. If it is "the average response time was 2.4 seconds for group 1 and 1.9 for group 2," that is a t-test.

The deeper difference is in sensitivity. Measurements carry more information than counts — a score of 87 versus 86 is a different measurement, while two people in the same category are indistinguishable. That is why, for the same sample size, a t-test can detect smaller effects than a chi-square test. It is also why converting continuous data into categories (say, "passed/failed" instead of the actual scores) is generally wasteful: it throws away information the t-test would have used[jmp-chisq].

The decision rule, condensed: two numeric groups → t-test; counts in categories → chi-square; three or more numeric groups → ANOVA; two numeric variables → correlation or regression. When in doubt, ask whether your data is measured or counted — that single question points to the right test.

Common Mistakes That Invalidate a Chi-Square Test

The chi-square test is easy to run and easy to break, and the breakage usually comes from how the data is prepared rather than from the arithmetic.

Small expected counts. The chi-square approximation to the p-value relies on expected counts being reasonably large — conventionally at least 5 in each category. When a cell has an expected count of 1 or 2, its (O−E)²/E term behaves erratically and the p-value becomes untrustworthy. The fix is to merge small categories or collect more data, not to ignore the warning.

Expected counts derived from the observed data. In a goodness-of-fit test, the expected counts must come from theory or an external standard, never from the sample itself. If you set expected equal to the observed proportions, the test becomes circular and can never reject. This is the single most subtle and damaging misuse of the test.

Non-independent observations. Each observation must fall into exactly one category, and categories must be mutually exclusive. If the same individual contributes to multiple counts, or categories overlap, the chi-square statistic overstates the evidence. This is why "respondents could select all that apply" questions are notoriously hard to test with chi-square.

Using percentages instead of counts. The test requires raw counts. Percentages lose the sample size, and without n there is no way to know whether the counts could have arisen by chance. Always convert percentages back to counts before testing[nist-chisq].

Confusing the three tests. Running a goodness-of-fit when the data is actually a two-way table, or treating a test of homogeneity as independence, produces a chi-square with the wrong degrees of freedom — and the wrong degrees of freedom can flip the verdict.

Practical Tips for Using a Chi-Square Test

• Decide which flavor first. One list of counts against a fixed expectation → goodness of fit. Two variables in a table → independence. Multiple groups sharing a distribution → homogeneity. The wrong choice invalidates the test.
• State the expected counts' source. They encode your hypothesis. If they come from theory, a previous study, or an advertised proportion, say which — and never derive them from the observed data.
• Check the degrees of freedom. k−1 for goodness of fit, (R−1)×(C−1) for a two-way table. The same χ² means different things at different df.
• Keep expected counts above 5. Merge sparse categories rather than trusting a p-value computed from tiny expected frequencies.
• Verify the categories are exclusive. Each observation in exactly one bucket; "select all that apply" data breaks the independence assumption.
• Use counts, not percentages. The test is meaningless without the sample size behind the proportions.
• Inspect the per-category contributions. A large χ² is diagnosable only if you can see which category drove it.
• Verify with the Chi-Square Calculator when the numbers matter — hand calculations with squares and division are error-prone, and the calculator reports df alongside χ² so you do not misread the critical value.

Limitations and Edge Cases

Chi-square tests have honest limitations. The approximation degrades with small expected counts, so sparse tables need merging or an exact test (like Fisher's exact test for 2×2 tables). The test flags that counts depart from expectation but not why — it is a screen, not a model builder. It cannot distinguish a single wildly off category from a uniform drift without inspecting the contributions. And a non-significant result does not prove the null is true: with a tiny sample, even a badly misspecified expectation can produce a χ² too small to reject. The test has low power when n is small, so a "good fit" from a handful of observations means little. Report the sample size, the expected counts' source, and the degrees of freedom alongside the statistic so readers can judge whether the test had the power to detect a real discrepancy.

Frequently Asked Questions

❓ What is a chi-square test?

✅ A statistical test that compares observed categorical counts against expected counts and decides whether the gap is bigger than random sampling noise would typically produce.

❓ What is the chi-square formula?

✅ χ² = Σ (Oᵢ − Eᵢ)² / Eᵢ, summed over all categories. O is the observed count, E is the expected count, and each term is a squared, standardized deviation.

❓ What is the difference between goodness of fit and test of independence?

✅ Goodness of fit compares one set of counts against a fixed expectation (df = k−1). A test of independence uses a two-way table and derives expected counts from the row and column totals (df = (R−1)(C−1)).

❓ How do I calculate chi-square by hand?

✅ Find the expected counts, compute (O−E)²/E for each category, then sum them. Compare the total to the critical value for your degrees of freedom at your chosen significance level.

❓ What is a good chi-square value?

✅ There is no universal good value. Compare your χ² to the critical value for your exact degrees of freedom; if it exceeds the critical value, the data depart from expectation more than chance explains.

❓ What sample size do I need?

✅ Enough that most expected counts are at least 5. With smaller expected frequencies the chi-square approximation to the p-value becomes unreliable.

❓ Can I use percentages instead of counts?

✅ No. The test requires raw counts. Percentages hide the sample size, and without n there is no way to judge whether the deviation could be chance.

❓ Who invented the chi-square test?

✅ Karl Pearson introduced the chi-square test in 1900, making it one of the earliest formal methods for analyzing categorical data — still essential in genetics, quality control, and survey research.

❓ When should I use a t-test instead?

✅ When your data is measured (continuous) rather than counted. A t-test compares two group means; a chi-square test compares observed counts against expected counts. Measurements carry more information, so t-tests detect smaller effects at the same sample size.