Setting a baseline before you measure a change

To judge whether a change in your own numbers is real, you need a record of what those numbers looked like before the change. That record is the baseline: a period of ordinary, measured behavior that the new period can be compared against. Without one, a new reading is judged only against memory, and memory tends to favor the story you already believe.

The approach works for changes you make on purpose, such as a new commuting route or a tighter grocery budget, and for changes that happen to you, such as a move or a shift in work hours. The rule underneath every step is to compare like with like: the same measure, recorded the same way, across periods of comparable length.

Define the change before the baseline starts

Write down exactly what will change and what result you expect. "Cycle to work instead of driving" is specific enough to test. "Feel better about my commute" is not, because there is no measure attached to it. A clear definition tells you which measures to track during the baseline and which to track during the change.

Settle how long the change will run before you judge it, and write that down before you start. A new habit may need several weeks before its effect shows, and a one-off event such as a holiday week may need to be excluded from both periods. Writing the planned duration down in advance prevents you from stopping early when the first results look convenient, or from continuing indefinitely when they do not.

Before the change starts, note the result you would expect if the change works and the result you would expect if it makes no difference. A prediction written in advance is much harder to reinterpret afterward, and it tells you in advance what kind of evidence would count against the change.

Collect a baseline long enough to show normal variation

A baseline should cover the usual ups and downs, not just one good or bad stretch. When a measure varies a lot from day to day, a few weeks of entries is a more reliable starting point than a few days. If the measure follows a weekly rhythm, such as heavier grocery spending on weekends, make sure the baseline includes at least one full cycle of it, so the comparison is not tilted by which days happened to fall inside the window.

Look at the baseline before the change begins. Plot it, or calculate its typical range and how widely the values spread. If the baseline is extremely erratic, the measure may be too noisy to detect a modest change, and you may need a different measure or a longer baseline. Knowing the spread in advance is what lets you tell a meaningful shift from ordinary noise once the change is underway.

If the baseline is short because the change could not wait, for example because a new schedule began on a fixed date, record the start date and treat the early comparison as provisional. Add any older records of the same measure that you can reconstruct, and revisit the comparison once the change period is longer.

Keep the measurement method fixed

Use the same definition, unit, device and timing in the baseline and in the change period. A switch from a timed commute to an estimated one, or from a monthly total to a weekly average, can create an apparent change that comes entirely from the method. If you must change the method, run the old and new versions side by side for a short time, so the difference between them can be measured instead of guessed.

Watch for other changes in the same period

A change rarely arrives alone. If you start cycling to work and also begin a new schedule that shifts your hours, the commute measure will reflect both. You may not be able to separate them completely, but noting the second change in the log helps you avoid crediting the whole result to the first. Write down anything else that changed in the same stretch, such as a new job responsibility, a different season or a move, so that later you can see the list of possible explanations.

Regression to the mean can make a change look like a success. If you started because a number had reached an unusually bad level, the next few readings will often look better even with no change at all, simply because extreme values tend to be followed by values nearer the usual level. Compare the change period with the baseline average, not with the value that prompted the change.

Compare the periods without overreading

Compare the average and the typical range of the change period with those of the baseline. A shift that falls well outside the baseline range is more persuasive than one that sits comfortably inside it. Remember that the periods you pick can shape the story: comparing your best week of the change with your worst week of the baseline is not a fair comparison, and neither is choosing the dates after you have seen which window looks best.

A simple summary is often enough. Write down the typical value in each period, how widely the values spread and the number of entries in each. Setting these side by side keeps the comparison honest and reduces the pull toward the single most dramatic day.

Give the result an honest reading. If the change falls within the baseline's usual range, the fair conclusion is that you have not yet detected a clear effect, which is not the same as showing that the change did nothing. If the shift is clear, keep recording for a while, because early differences sometimes fade. Write down the conclusion and the reasoning behind it, so that a future you can see how the judgment was reached and whether it still holds.