Article Critique model text

Some have requested a mentor text to guide your article reviews.  So here is an idea: I will post one that I have written and allow you to critique my critique.  The goal for this form of peer review is to enhance the writing.  Please offer questions and comments in that vein.  I like this open-source process for rooting out bad arguments, inconsistent thoughts, incomprehensible language, mistaken assumptions, poor analysis and other forms of writing habits.

Positives might be nice, as well.

(For those of you uninterested in reading my work in progress, here is a just-too-brief example of an article critique).

Thanks for your help.

************************************************************************

Kahan, D, Braman, D. and Jenkins-Smith, H. (2010) “Cultural cognition of scientific consensus” in Journal of risk research (in press)

*

Abstract: (from authors)Why do members of the public disagree—sharply and persistently—about facts on which expert scientists largely agree? We designed a study to test a distinctive explanation: the cultural cognition of scientific consensus. The “cultural cognition of risk” refers to the tendency of individuals to form risk perceptions that are congenial to their values. The study presents both correlational and experimental evidence confirming that cultural cognition shapes individuals’ beliefs about the existence of scientific consensus, and the process by which they form such beliefs, relating to climate change, the disposal of nuclear wastes, and the effect of permitting concealed possession of handguns. The implications of this dynamic for science communication and public policy-making are discussed.

*

“Give me a place to stand, and I will move the earth”

Archimedes

Why do segments of the population so easily disagree on public policy despite the presence of “steady and massive accumulation of scientific evidence” and consensus among experts?  The authors of “Cultural cognition of scientific evidence,” from a yet to be released issue of Journal of risk research, present a “novel explanation” for the tendency of individuals to assimilate new information to their preexisting beliefs.  They provide correlational and experimental evidence that “cultural cognition,” one’s background worldview and normative values, represents how new information is understood by individuals in a way more determinate than logic and reason.  They conclude,

“Individuals systematically overestimate the degree of scientific support for positions they are culturally predisposed to accept as a result of a cultural availability effect that influences how readily they can recall instances of expert endorsement of those positions.”

In other words, perception is always and already biased.  Let us unpack the background, methodology and results of this study to get a stronger sense of why this is so.

Background

The authors start by framing a problem of public dialogue regarding science,

“The problem, it seems, is not that members of the public are unexposed or indifferent to what scientists say, but rather that they disagree about what scientists are telling them.”

They go on to explain why they take up the lens of “cultural cognition” as a hypothesis for this reflexive position-taking.  This theory, “a collection of psychological mechanisms that dispose individuals selectively to credit or dismiss evidence … that fit values they share with others,” emerges from a nexus of three existing ideas.  The availability heuristic refers to the tendency of individuals to “more readily recall instances of experts taking the position that is consistent with their cultural predisposition.”  Working memory has a limited size.  When thinking about an issue, people juggle only so much information at once.  The availability of certain kinds of information thus weights their decision-making more heavily.

Add to this factor a disposition for biased assimilation of information.  Thinking about an issue may result in a kind of evidence seeking, for data to fit into existing schema.  One reason for this narrowing of acceptable data concerns identity-protective motivations “to conform one’s belief to those of like-mind others in order to avoid dissonance and protect social standing.”  Think of identity as a kind of normal state of being, without conscientious attempt to perform for others.  This more or less unreflective thinking implicitly pulls into view information that does not shock the mind.  Dissonance-creating information is ignored, allowing a kind of cognitive restfulness.  Put another way, it hurts too much to think.

Together, these three dimensions of perception lead to over-representing information sources in individual’s mental inventories that align with a preexisting belief.  The predisposition accounts for a disproportionate share of confirming information.

Methodology and Results

The authors pose this thesis of cultural cognition as the reason for public dissensus.  To test this, they set up a study with two components.  Using a sample of 1500 individuals who roughly represent American socio-economic demographics, the authors use a correlational survey on preexisting beliefs among their subjects and then an experimental survey to assay responses to scenarios aligned with and diametrical to these beliefs.  The results are analyzed in two ways. First, a preliminary quick scan of the surveys provides a macro-view on their thesis.  Then a multivariate analysis (basically, a computer-based algorithm that transforms the survey results into numbers and crunches them into statistics) detects micro-level significance.

In all instances of analysis, the results confirm the original thesis.  While the range and degree of confirmation varies widely, all outcomes provide statistically significant evidence (the outcomes could not have happened by chance) of  “a strong correlation between individuals’ cultural values and their perceptions of scientific consensus.”

Commentary

“Cultural cognition of scientific consensus” underscores the view that dialogue is riven from the start with obstacles of perception.  This article has several serious threats to its validity.  One could argue that the evidence from this study is itself subject to the same forces this study attempts to grapple, demonstrating the obstacles of perception by failing as a piece of research (a fact that is philosophically intriguing, though of limited practical use).  Or, one could demonstrate how surveys used might over-determine the authors’ beliefs (by allowing for only two diametrical points of view; the middle vanishes, along with nuance).  And the failure of the researchers to address, much less explain, how prior understanding of their subjects majorly skew the survey responses.  Further, one could point out how the use of statistics to articulate the process of human thinking inherently misses the actual scope of human experience.  This means that responses are a third order of thinking - original thought, explanation of original thought, response to survey in a way to present original thought in line with researchers aims (the so-called Hawthorne effect).  All these criticisms (and more: consider also he unquestioned representativeness of the sample) do put a drag on the reach of the conclusions presented in the study.  However, the underlying workings of cultural cognition, confirmed with at least a face valid confidence as they appear in this study, do provide a graspable structure for ways to approach the dynamics of understanding. 

This aspect is vital for educators and others working in the public sphere. The challenge for the teacher is immense.  As the authors write of a typical student,

“… because of culturally biased information search and culturally biased assimilation, she is likely to attend to the information in a way that reinforces her prior beliefs and affective orientation … the same predisposition that informs her prior (beliefs) will also be unconsciously shaping her ability to recognize an assign weight to all manner of evidence, including the opinions of scientists.”

Cognitive tendencies filter perceptions. To reach each other, individuals must take overt steps to negotiate the bearings of their common world; shared experience has to be explicitly named and agreed upon at a fundamental level.  For teachers, this means structuring learning environments that invite students to investigate the ways they (both student and teacher) bias information.  Statements of fact need to be broken down into components of normative and subjective experience (Carspecken, 1990).  From simple reflexive opinions to layered rational replies, student dialogue represents the topsoil of understanding.  It is the effortful digging into the cultural valence of thinking – “communicators must attend to the cultural meaning as well as the scientific content of information” – that provides a platform for the kinds of critical thinking ostensibly sought in classrooms today.  Put another way, the very values of judgment that produce meaning must be kept open lest individuals close down onto piece-meal, fragmentary beliefs (Readings, 1996).

The authors do not intend so, but their work further erodes the conception of schooling based on the ideal of accumulation.  In practice, this conception treats students as self-activating vacuum cleaners; when they are ready to get to the business of knowledge consumption, an internal switch starts the sucking.  The effects of cultural cognition brings awareness to the shakiness of the ground underneath.  Communities – including districts, schools, even classrooms – must first recognize how divergently the world spins for its members.  Attempts to harmonize the common space each member occupies must precede knowledge acquisition.  Put differently, educators must seek to understand the experiences of their students from within the students’ point of view (van Manen, 1990), and help each do the same with all others.  They must start with the proposition that unpacked (and perhaps un-unpackable) belief structures carry students, not the other way around.  Without giving reason to students and taking for granted the inviolable presence of an active subjective rationality (Schön), teachers will compound the dissensus at the heart of the study under review.

   Archimedes never found a place to stand.  Schools have the potential to give up his futile search, and instead lead students to examine the taken-for-granted ground of their individual experience.  

References

Carspecken, P. (1996). Critical ethnography in educational research. New York: Routledge

Readings, B. (1996). The university in ruins. Cambridge, MA: Harvard University Press

Schön, D. (1987). “Educating the reflective practitioner.” 1987 AERA presentation.  (http://resources.educ.queensu.ca/ar/schon87.htm)

van Manen, M. (1990). “Beyond assumptions: shifting the limits of action research.” Theory into Practice, vol. xxix, no. 3