Sample Ratio Mismatch Calculator

Sample ratio mismatch, or SRM, is what happens when the actual number of visitors in each arm of your A/B test does not match the split you configured. Set up a 50/50 test and end up with 10,245 users in variant A against 9,758 in variant B, and the question becomes whether that gap is normal noise or a sign your experiment is broken. The calculator above runs a chi-square goodness-of-fit test on your numbers and gives you a p-value, the standard way experimentation teams answer that question before they trust a test result.

How SRM is calculated

The test compares your observed counts against the counts you would expect if the split had gone exactly as planned. For each variant, you take the squared difference between observed and expected, divide by the expected count, then sum across variants to get a chi-square statistic. That statistic converts into a p-value, the probability of seeing an imbalance this large or larger purely by chance if your randomization were working correctly. A very small p-value means the imbalance is very unlikely to be chance.

What p-value means SRM

Most experimentation teams use a stricter bar for SRM than the 0.05 cutoff common in significance testing. Because an SRM check runs on every experiment you launch, a 0.05 threshold would flag false alarms constantly just from normal variation. The convention that has become standard across the industry, popularized by Microsoft's experimentation platform team, is to flag SRM at p below 0.001, with the 0.001 to 0.01 range treated as a borderline zone worth a second look rather than an automatic fail.

Why sample ratio mismatch matters

When SRM is present, the two groups you are comparing are no longer the same kind of user in aggregate, which breaks the core assumption behind an A/B test. A skewed split is often a symptom of something upstream: bot traffic hitting one variant unevenly, a redirect or caching layer leaking users before they are logged, broken randomization in the assignment code, or one variant loading slower and losing impatient visitors before the experiment records their exposure. Any of these can quietly bias your results in the same direction as, or opposite to, the effect you are trying to measure, which is how a broken test ships a confidently wrong decision. If your team is running experiments on landing pages or funnels and wants that program built on a solid measurement foundation, a free SEO audit is a good place to start the conversation.

Want this done for you? Rankite offers monthly SEO management and SEO strategy session.