Most studies linking media use with wellbeing use self-reported estimates of media use, but a new analysis of data from over 50 000 individuals suggests that self-reported media use does not accurately reflect objective measures of device-logged media use. This implies that caution is needed when drawing wide-reaching conclusions from studies relying solely on self-reported media use.

“The screen-time discrepancies highlight that we simply do not know enough yet about the actual effects – both positive and negative – of our media use. Researchers, journalists, members of the public and crucially policy makers need to question the quality of evidence when they consider research on media uses and effects. We can no longer simply take claims of harmful effects at face value,” says Dr Douglas Parry, a researcher and lecturer in the Department of Information Science at Stellenbosch University (SU) in South Africa. Parry is the lead author of an international study of which the results are published in the journal Nature Human Behaviour.

Researchers from South Africa, Norway, the United Kingdom and the United States analysed data from over 50 000 individuals reported across 47 studies that included both self-reported and actual use via device logs.

In the study, it was reported that the relationship between self-reported and actual media use is too unreliable to draw meaningful conclusions. In fact, participants’ estimates of their usage were only accurate in about 5% of the studies that were evaluated. What’s more, analysis revealed that participants under- and over-report their usage in equal measure.

Comments Dr Brit Davidson from the University of Bath’s School of Management: “These highly flawed studies are over-inflating the relationships between digital media use and typically negative outcomes, such as mental health symptoms and cognitive impairments, which of course explains the pervading view that smartphones among other technologies are bad for us.”

She adds that phone and technology use takes the blame for everything from increases in teenage depression and suicide to higher incidence of Attention Deficit Hyperactivity Disorder (ADHD) and violence. “If we want to properly investigate harms associated with smartphone use, we should first tackle assumptions about screen-time and disentangle how people are actually using their phones or other technologies of interest.”

The COVID-19 pandemic has shifted most education, work and leisure activities online. While digital technologies have been a critical tool during the pandemic, many have raised concerns regarding the link between increased digital media use, such as smartphones, social media and video games, with poor well-being. Even before the pandemic moved most day-to-day activities online, some scholars went as far as to say that excessive media use has “destroyed a generation”.

Most studies that have reported that increased digital media use is associated with poor well-being acquired data by asking participants to estimate how much they use digital media. For example, participants have been asked questions like, “On average, how many minutes do you spend on social media on a typical day?” as well as answering questionnaires that measure well-being. With this data, researchers then estimate the relationship between media use and well-being.

However, self-reported behaviour tends to be inaccurate, as it reflects what people believe they do, rather than what they actually do. An alternative to asking people to estimate their digital media use is to capture objective logged information via media devices. For example, Apple’s “Screen Time” feature for iPhones and iPads can provide precise media usage reports for a given time period.

Says Parry: “Not only does this finding have far reaching implications for researchers across fields like communication, psychology and information systems but, perhaps of equal importance, it suggests caution before implementing policies that restrict digital media use. Given the quality of current evidence, we simply do not know enough yet about the actual effects (both positive and negative) of our media use.”

University of Oslo researcher and study co-author Dr Daniel Quintana explains further: “If self-report estimates of media use are not accurate measures of actual behaviour, we need to question the validity of the findings of studies that have solely relied on self-reported measures of media use, which have been most studies up to this point.”

The research team also investigated whether measures of so-called “social media addiction” were suitable substitutes for logged usage. These analyses revealed an even smaller association between self-reported measures and usage logs. This aligns with work that suggests that self-reported measures are more reflective of various mental health outcomes than media use itself.

The researchers hope that their study will lead to a shift in measurement practices regarding technology, alongside starting to correct the narrative regarding technology and society. Only by better understanding what people actually do with their technologies, they say, can we can start to genuinely understand the impact of them on people and society.

“We hope that our research will encourage policymakers to use accurate measures of media use when making decisions and recommendations regarding media usage,” says Quintana.