What a daily diary study can and cannot show

A large share of the capitalization literature rests on one design: the daily diary. Participants record events and reactions once or more per day across a period of days or weeks. Understanding what this design can support is more useful than memorising any particular result obtained with it.

The design

Niall Bolger, Angelina Davis and Eshkol Rafaeli set out the methodology in a 2003 Annual Review of Psychology paper that remains the standard reference. The central argument is that measuring close to the moment reduces reliance on retrospective summary, which is subject to known distortions, and that repeated measurement within the same person allows a person to serve as their own comparison.

That second property is the important one. When a diary study reports an association, it is frequently a within-person association: on days when one thing was higher for a given individual, another thing also tended to be higher for that same individual. This is a different claim from a between-person association, and the two are often reported in the same paper.

What within-person association does not establish

Two variables can co-vary within a person for reasons that have nothing to do with one causing the other. A third factor may move both. The direction may run opposite to the one a reader assumes. And in most diary designs both measures are collected in the same entry, so temporal ordering is not established either.

Careful papers state this. Reports written about those papers frequently do not, which is the main reason a claim can drift from the study that produced it.

Compliance, and the paper-versus-electronic problem

Diary data depend on entries being made when they are supposed to be made. Alexandra Green, Eshkol Rafaeli, Niall Bolger, Patrick Shrout and Harry Reis addressed this directly in a 2006 Psychological Methods paper comparing paper and electronic diaries, examining whether the two produce equivalent data. Timestamped electronic collection made it possible to check something that paper diaries could not: whether entries were completed at the intended time or reconstructed later.

The methodological literature that followed treats compliance as a design parameter to be measured and reported rather than assumed. Studies vary in how thoroughly they do so.

Why the design was adopted here

The events this literature studies are ordinary, frequent, and easy to forget. Asking someone months later how a partner generally responds to good news produces a summary judgment shaped by mood at the time of asking and by the general state of the relationship. Recording closer to the event trades that problem for others — participant burden, reactivity to being measured, dropout across the study period — and the trade is made deliberately.

Reading a diary study

Several things are worth locating in the paper itself:

• The number of participants and the number of days — both affect precision, and they are not interchangeable.
• Whether reported associations are within-person, between-person, or both.
• Whether the two variables came from the same diary entry or from entries at different times.
• How compliance was measured, and what proportion of intended entries were completed.
• Who the participants were, which in this literature is often a university sample.

None of this makes diary studies weak. The design was developed precisely because the alternatives — one-off retrospective questionnaires, or laboratory sessions detached from daily life — carry problems of their own. It is a design with a defined scope of inference, and the scope is narrower than summaries of it tend to suggest.