Science Reading: Cognitive narrative helps us understand science better

Take a look at the picture below. What do you see? Think for a moment and write down your observations or say them out loud.

Did you see tracks in the snow and the shadow of trees in the background? Maybe you were more specific, noting there are different kinds of tracks going in different directions, and at least two trees casting shadows. Or you may have even thought it looks like a large bird of prey caught a small animal and flew off with it.

In fact, there are no animals in this picture. There are no tracks. There isn’t even any color.

All that we actually see are gray-scale pixels on a two-dimensional space. Yet, when asked what we see, many of us begin constructing and describing an event: an animal caught by a bird of prey. We might even include elements of motivation (the bird is hunting for survival) and emotion (feeling sorry for the critter) in our observations. What this means is that we infer from these shades of gray a narrative – characters (the predator and prey) interacting in a place over time (the bird capturing the critter and flying off with it).

Narrative is key to how we interpret the world.

We should clarify now that the narrative we are referring to is cognitive narrative. This is narrative as an information-organizing principle in human-type cognition. It is enmeshed in how humans think and interact. When we chat with our friends to catch up, we’re sharing stories about our lives. When we go on a job interview, we’re telling the story of our prior experiences as positively as possible. When we read a newspaper article, we’re reading stories that happened in the world as documented by those who observed them or at least gathered information from others about what happened. When we watch a stranger storm off, then ask our friend what happened, we are engaging in gossip – stories about people that are none of our business but that interest us anyway.

Stories shape our lives – and they are immensely powerful. Consider what happened with the television series Firefly, a space western imagined by Joss Whedon. When the show was canceled after one season, fans raised money and pressured the studios to complete the story. Think about that for a moment. Strangers the world over organized just to get the ending of a story.

Stories shape our thoughts, our comprehension of what happens in the world, and they move us to action.

What does this have to do with science? Science is a human activity dedicated to gathering knowledge and understanding. We record what we learn and how we learned it in papers. Our goal when we read a scientific journal article is what we call ‘Science Reading’: the act of conceptually reconstructing a scientific process that researchers, with all of their human biases and limitations, have conducted. We should clarify that we’re not talking about the type of narrative commonly used in science communication, where a science communicator superimposes a story designed for consumption by a general audience. The narrative that interests us here is the one inherent to the representation of a scientific process.

We narrate from the available documentation such that we can assess the reliability of the scientific process, its products, and the inferences the researchers make. In other words, it means that we, as readers, want to be able to understand what actions were performed and observed, and why, where, when, how, and what it all means. For Science Reading to be effective, the researchers must have created documentation structured for comprehension from these processes for us to be able to answer all these questions. Thus, the documentation must be both physically and conceptually accessible.

It is not enough to be able to get the information, we must also be able to understand it.

Let’s think about how we eat ice cream.

To understand what we mean, let’s first consider something that most of us are well familiar with: eating ice cream. We’re not referring to the mechanics of eating ice cream (i.e., can you put a spoonful of creamy, icy cold deliciousness in your mouth?).

The real question is what skills we need to understand how to break down an ice cream eating event in terms of understanding. What entities, actions, behaviors, attitudes, etc., are involved. Those are the skills you need for Science Reading: the skills to break down events to understand how they happen.

Let’s deconstruct.

What do we need for an ice cream eating event? A list might look something like this:

1. People who want ice cream
2. A place to get ice cream
3. To order ice cream
4. The ability to order ice cream
5. Tools to serve and eat the ice cream (scooper, bowl, and spoon)
6. Time to eat the ice cream
7. To engage in actually eating the ice cream
8. People who no longer want to eat ice cream because they've had some (yes, we know. Extremely unlikely, but bear with us.)

All of these points can be positioned in a narrative arc, framed by time. This arc contains 3 key points. At the beginning is Belief/Physical State A, in the middle something that happens to challenge the status of Belief/Physical State A, and at the end the resulting Belief/Physical State now labeled B. The arc can look something like this:

Note that the arc is not quantitative. Being higher up does not indicate a greater quantity of anything, it signifies progression towards the point where a shift becomes possible from A to B.

More information needed to understand an ice cream eating event than to list its sub-events and place them on a narrative arc. We still don’t know who, or how, or when. Every language has a set of terms it uses specifically to get the details of a narrative. They’re the universal interrogatives, labeled as such because they occur in every single language in some form. In English they are:

1. Who
2. What
3. Where
4. When
5. Why
6. How

You’ll notice that some answers in our list are actually answers to an interrogative with a preposition (e.g., “tools to serve and eat ice cream” answers the question “with what?”). The interrogation process is about learning to ask the kinds of questions that can get you the whole story. That said, one of the helpful things about positioning your events in an arc is that you can see where there are gaps in your information.

You’ll notice above that there is nothing between ‘eating ice cream’ and ‘no longer wanting ice cream’. What happened here?

The universal interrogatives help you obtain the missing information. The more you practice actively working on getting the whole story, the more specific your questions will become (i.e. was the ice cream eaten slowly? Did anyone get second servings?). Where the questions stop depends on your goal for the story you are building in your head.

In a science paper, your goal is enough comprehension to be able to reproduce the events conceptually. The important thing is to understand that you already have this tool set. Now you need to refine those skills for science.

Mapping to science article

What happens when we apply the same narrative process, guided by universal interrogatives to a scientific journal article?

Let’s begin by looking at a short paper: See the ball, Hit the Ball (Witt & Profitt, 2005). From this paper, we may get a list of events that look like this:

1. Made an observation about the subjectivity of ball size estimation
2. Did a literature review about size perception
3. Produced a hypothesis proposing there is a relationship between skill and size perception
4. Designed an experiment to test whether this was the case using softball players
5. Designed and produced the materials for the experiment
6. Developed a recruitment plan for participants
7. Recruited participants softball player
8. Ran the experiment
9. Prepared data for analysis
10. Analysed the data
11. Interpreted the data
12. Made a decision about what the data said in relation to the hypothesis
13. Wrote up the paper

And positioning into a narrative arc may look like this:

But there is still much information missing.

Enter the universal interrogatives. The first event is ‘made an observation […]’ but who made the observation? When? In what context? The next event is ‘Did a literature review […]’ but who did it, how was it done? How were the sources gathered? Were some already familiar to the authors? What criteria were used to select specifically these papers?

Many of the answers to these questions will be found in the text. Unfortunately many more won’t. The power of narrative here is that it sheds light on the unknown unknowns. It helps us understand what we did not realize we did not know. This is crucially important to improving the quality of the scientific record.

A disturbing proportion of peer-reviewed papers do not contain enough information to replicate the studies they describe. The people who reviewed them did not realize that key information was missing or what key information was needed to ensure that the science they contained was replicable.

The systematicity of this problem is blown wide open in meta-analyses. Again and again researchers attempting meta-analyses find that papers in their current form lack the information needed to appropriately compare them with similar studies. This points to the need for an improved scientific documentation system, based on how humans organize and understand information. We wish to stress that it is no individual author’s or reviewer’s personal fault that our science documentation is unreliable. Only that it is our collective responsibility to fix it.

Take home message

A scientific journal article is  the documentation of a sequence of events aimed at finding an answer to a question – a scientific process. It’s very similar to historical writing in that the events are interpreted from the author’s perspective. However, unlike with historical writing, scientific events are typically recorded by the very people who designed and orchestrated them. This is key to conceptualizing that neither the events, their documentation, nor their interpretation can ever be objective. They are all constrained by the inherent biases involved with being human, the same as with writing histories and the logic behind the adage that ‘History is written by the victors’. This humanity, warts and all, is not a bug in the system, but a feature of being humans who do science. Does this mean that the products of this process cannot be trusted? No. It means they must be understood as products of rational minds in the context of specific arguments in specific contexts. There are no shortcuts. Improving Science Reading (and writing) will require collaboration between science writers, readers, and reviewers

Copyright © 2022 Monica Gonzalez-Marquez, Anouschka Foltz, Jeffrey K. Bye, Ali Fulsher. Distributed under the terms of the Creative Commons Attribution 4.0 License.