Mistakes that creep in when comparing two figures

Two raw figures are rarely enough to say which is larger in the sense that matters. Comparisons go wrong when the figures are measured against different bases, cover different periods or use different definitions, and when a difference is described in absolute or relative terms without saying which.

Compare against the same base

A raw count depends on the size of what it counts. Suppose two depots each report late deliveries for a single month. One reports 84 and the other reports 210. The larger number belongs to the larger depot, and it says nothing about which depot is late more often relative to its work. If the first depot handled 1,400 deliveries and the second handled 5,000, the late rates are 6 percent and 4.2 percent. The depot with fewer late deliveries in total has the higher late rate.

The same trap appears with time. A total for a three-week period looks smaller than a total for a five-week period even when the weekly pace is identical, so compare weekly or daily amounts when the periods differ in length. Small bases create a separate problem. A rate of 50 percent based on two cases tells you far less than the same rate based on two hundred cases, because a handful of extra or missing cases can move a small base dramatically.

The fix is to divide by a base before comparing. Choose a base that is the same for both figures, such as sales, customers, households or population, depending on the question. If the right base is not available, say so rather than comparing raw totals as though they were equivalent. A rate with a weak base is still more informative than a total with none.

Absolute and relative differences

A difference can be stated in absolute terms, which is the simple subtraction, or in relative terms, which is the difference divided by the starting figure. Both are correct, but they can create very different impressions. Going from 2 to 4 is an absolute increase of 2 and a relative increase of 100 percent, because the figure doubled. Going from 90 to 92 is also an absolute increase of 2, but the relative increase is about 2.2 percent.

A practical habit is to write both forms down before judging how large a change is. The relative figure shows how the change compares with where it started, and the absolute figure shows how many units actually moved. A large relative change on a tiny starting figure may involve only a few units, while a small relative change on a very large starting figure may involve thousands. Many disagreements about the size of a change come from one side quoting the absolute figure and the other quoting the relative one, so ask which form the story chose and whether the other form would leave a different impression.

Other mismatches and a short check

Beyond the base, several things can make two figures incomparable. The periods may differ: a quarter is not a year, and a partly complete year is not the same as a full one. The definitions may differ: a "household" in one survey may not match a "family" in another. The method may have changed, so a jump may reflect a new way of counting rather than a real change. When you cannot tell whether two figures match, look for the methodology note, which usually names the base, the period and the definitions in a few sentences.

Revisions need their own check. Suppose a monthly count is first published as 4,200 and later revised to 4,600, and the following month's first release is 4,500. Setting that new figure beside the revised 4,600 shows a fall of 100, while setting it beside the first-release 4,200 shows a rise of 300. The two comparisons point in opposite directions because one month has been revised and the other has not. If earlier first releases were revised upward, as the 4,200 was, the new 4,500 may be revised upward too, so treat the direction as unsettled until the new month is revised. Whenever you compare two figures, state which version each one is, and check the publisher's revision note, which usually explains when and how figures are updated.

Before you draw a conclusion from a comparison, answer four questions. Are the two figures measuring the same thing? Do they cover the same period? Are they measured against the same base? And is the difference larger than the normal variation you would expect from one period to the next? Counts of around 90 can rise or fall by several units from one month to the next without any underlying change, so a rise of 2 means little on its own. A difference that survives all four questions is far more likely to be real.

If the check fails on any question, the honest conclusion is that the two figures cannot be compared yet. Say that plainly, and look for a matching base, period or definition before making a claim. A comparison that holds up under these four questions is worth making, and one that does not is better held back until the figures line up.