25-Cognitive-Assessment.Rmd

# Assessment of Cognition {#cognition}

## Overview of Cognition {#overview-cognition}

Cognitive-behavioral therapy (CBT) is the most effective treatment we have for many mental health conditions, especially depression and anxiety.
The cognitive therapy component of CBT assumes that thoughts mediate the influences of the environment on emotions and behavioral responses.\index{cognition}
In general, cognitions and cognitive processes play a key role in many psychological theories.\index{cognition}
Therefore, cognitions and distorted cognitions are assumed to be a key construct to assess.\index{cognition}
However, there are many challenges to cognitive assessment.\index{cognitive assessment}
Some consider cognitions to be fictitious entities or epiphenomenal to people's experience—i.e., that is, they are not a causal process, but rather they are a by-product of neural and biological processes.\index{cognition}

## Aspects of Cognition Assessed {#cognitionAspects}

Multiple aspects of cognition can be assessed, including cognitive products, processes, and structures or organization.\index{cognition}
Cognitive products include a person's conscious thoughts or imaginal images.\index{cognition}
Cognitive processes include how a person transforms the environmental input and how they take meaning from it.\index{cognition}
Cognitive structures and organization include the hypothesized structures that a person has to guide information processing.\index{cognition}

Consider an example of a person with an anxiety disorder.\index{cognition}
Cognitive products for the person may include having the thought, "I'm going to make a fool of myself, and everyone will think I'm stupid".\index{cognition}
The person's cognitive processes may involve the over-estimation of personal risk.\index{cognition}
The person's cognitive structures may include "danger" schemas that, when activated, lead to attention [bias](#bias) toward threat stimuli.\index{cognition}\index{bias!attentional}
Though, this finding is under question because of challenges to the [reliability](#reliability) of the dot-probe task, which has been frequently used to examine attentional [bias](#bias).\index{cognition}\index{reliability}\index{bias!attentional}

## Approaches to Assessing Cognition {#cognitionApproaches}

The general approaches to assessing cognition include endorsement methods and production methods.\index{cognitive assessment}
Endorsement methods for assessing cognition contain a predetermined set of thoughts that participants identify or rate, such as a checklist.\index{cognitive assessment}
Using production methods, participants generate or recall their thoughts, i.e., the method involves free response.\index{cognitive assessment}
The psychometric status is currently stronger for endorsement methods; however, increasing attention is being given to production methods.\index{cognitive assessment}
Approaches to assessing cognition are reviewed by @Dunkley2019.\index{cognitive assessment}

### Self-Reports of Cognition {#cognitionSelfReport}

Self-reports of cognition can either be in written or interview format and comprise most of the cognitive assessment techniques that are regularly used in clinical practice.\index{cognitive assessment}\index{self-report}
Few of the other approaches to assessing cognition are used routinely in clinical practice.\index{cognitive assessment}\index{self-report}
In a self-report of cognition, the participant must reflect on and report about their own cognitive style.\index{cognitive assessment}\index{self-report}
Thus, self-report of cognition involves introspection.\index{cognitive assessment}\index{self-report}

Self-reports of cognition are still used in CBT, to learn about people's cognitive styles, their attributions to life events or a specific negative event, and their expectancies.\index{cognitive assessment}\index{self-report}
Expectancies include, for instance, what the person anticipates happening.\index{cognitive assessment}\index{self-report}

#### Pros {#cognitionSelfReportPros}

Pros of self-reports of cognition include:\index{cognitive assessment}\index{self-report}

 - They are cheap, easy to score and administer, and are able to be useful in clinical practice.\index{cognitive assessment}\index{self-report}
 - They are sensitive to treatment effects.\index{cognitive assessment}\index{self-report}
Cognitive therapy advocates would argue that changes in cognitions mediate treatment changes (i.e., they are thought to be a causal mechanism that explains how therapy works).\index{cognitive assessment}\index{self-report}
 - They are standardized, [normed](#norm), and have been psychometrically validated.\index{cognitive assessment}\index{self-report}\index{norm}\index{standardization}

#### Cons {#cognitionSelfReportCons}

Despite the fact that these techniques are so popular, there are still some major drawbacks to these approaches.\index{cognitive assessment}\index{self-report}
Cons of self-reports of cognition include:\index{cognitive assessment}\index{self-report}

- They require a participant to have insight into their cognitions that they can report on—such insight is not always typical.\index{cognitive assessment}\index{self-report}
@Nisbett1977 published a classic study titled, "Telling more than we can know: Verbal reports on mental processes".\index{cognitive assessment}\index{self-report}
The study was based on the idea that people believe they were thinking particular things; however, they reported thoughts on things that would not have been possible for them to know.\index{cognitive assessment}\index{self-report}
Findings such as these challenge the [validity](#validity) of self-reported cognition.\index{cognitive assessment}\index{self-report}\index{cognitive assessment!validity}
 - They require a person to reflect upon their cognitions, retrospectively recalling what they were thinking in various situations.\index{cognitive assessment}\index{self-report}\index{retrospective}
 - At least in the written format, which is often used as an endorsement method, participants are limited to responding to the experimenter-generated choice options—which can lead people into having certain thoughts and make people endorse thoughts that they did not actually have.\index{cognitive assessment}\index{self-report}

### Think-Aloud Approaches (Thought Listing) {#cognitionThinkAloud}

In think-aloud approaches, the participant is asked to verbalize their cognitions in the form of a continuous monologue while performing some task, and responses are recorded for later evaluation.\index{cognitive assessment}
In this way, thoughts are assessed concurrently with their occurrence.\index{cognitive assessment}
In some approaches, such as Articulated Thoughts in Simulated Situations (ATSS), thoughts are assessed in simulated situations that are geared towards eliciting certain emotions—the "situations" can be presented in video, audiotape, or virtual reality format.\index{cognitive assessment}
In think-aloud approach, the participant lists their thoughts, so the approach is also called "thought listing".\index{cognitive assessment}

The purpose of think-aloud approaches is to avoid retrospection, and to create a situation in the lab that resembles a real-world situation to see what thoughts or what stream of thoughts are generated.\index{cognitive assessment}
Think-aloud approaches are reviewed by @Davison1997.\index{cognitive assessment}

#### Examples {#cognitionThinkAloudExamples}

Examples of think-aloud assessments of cognition include video replay and private speech.\index{cognitive assessment}

##### Video Replay {#cognitionThinkAloudVideoReplay}

Video replay is also called videotape thought reconstruction.\index{cognitive assessment}
In videotape thought reconstruction, the examiner video records the interaction, then replays the video for the examinee, and asks the examinee what they were thinking during that moment.\index{cognitive assessment}
It is more proximal than retrospective report.\index{cognitive assessment}\index{retrospective}
The act of re-watching a video of oneself can be strange, and there is commonly [reactivity](#reactivity) to that.\index{cognitive assessment}

##### Private Speech {#cognitionThinkAloudPrivateSpeech}

Another example of think-aloud cognition is children's private speech.\index{cognitive assessment}
Young children sometimes talk out loud to themselves to guide their behavior, and the speech is not serving a communicative function.\index{cognitive assessment}
This is known as private or self-directed speech.\index{cognitive assessment}
Private speech is most common among 2- to 7-year-olds, and it provides insight into their cognition, often while performing a challenging task.\index{cognitive assessment}

#### Pros {#cognitionThinkAloudPros}

Pros of think-aloud approaches include:\index{cognitive assessment}

- Participants are asked to report on cognitions at the same time or just after the thoughts are occurring, so they are less sensitive to retrospective recall [bias](#bias).\index{cognitive assessment}\index{retrospective}\index{bias}
- Responses are not limited to experimenter-selected choices—all cognitions can be assessed.\index{cognitive assessment}
The constraints are made in terms of how the thoughts are coded and analyzed by the experimenter, not in terms of the unstructured and open-ended thoughts that the participant can provide.\index{cognitive assessment}
- Specific situations of interest to the research team can be explored.\index{cognitive assessment}
For example, the examiner can examine the respondent in situations that are thought to evoke particular cognitive processes, including situations that might be impractical due to low frequency or that are unethical to assess in their natural context.\index{cognitive assessment}
For example, an examiner could examine a person's self-reported thoughts in response to social criticism for someone with social anxiety.\index{cognitive assessment}

#### Cons {#cognitionThinkAloudCons}

Cons of think-aloud approaches include:\index{cognitive assessment}

- Performance is susceptible to observer effects.\index{cognitive assessment}\index{observer effect}
Observer effects are changes in the person's response because they are being watched or recorded.\index{cognitive assessment}\index{observer effect}
That is, response biases may be present because the person's responses are not anonymous.\index{cognitive assessment}\index{response style}
- Social desirability [bias](#bias): people may censor negative or disturbing thoughts and may describe their thoughts in a way that portrays themselves in a more positive light.\index{cognitive assessment}\index{bias!social desirability}
- It can be difficult to generate the thoughts.\index{cognitive assessment}
There are individual differences in people's awareness of their cognitions.\index{cognitive assessment}
- Think-aloud approaches can interrupt the thought flow—we think faster than we can speak, so verbalizing thoughts can alter the thoughts.\index{cognitive assessment}
That is, talking about thoughts interferes with the task itself.\index{cognitive assessment}
- It can be difficult to get a whole narrative of thoughts from think-aloud approaches.\index{cognitive assessment}
- It can be difficult to code the qualitative thoughts and reduce them down to meaningful data.\index{cognitive assessment}
And coders may attribute different meaning to a thought than was intended by the respondent.\index{cognitive assessment}
- Think-aloud approaches tend to be lower in [ecological validity](#ecologicalValidity) because they typically occur in the lab rather than in a naturalistic context.\index{cognitive assessment}\index{validity!ecological}

### Random Thought Sampling {#cognitionRandomThoughtSampling}

The purpose of thought sampling is to quantify characteristics or aspects of thinking in an [ecologically valid](#ecologicalValidity) context.\index{cognitive assessment}
By [ecologically valid](#ecologicalValidity), we mean that the situation closely represents the specific situations under which the person usually exists.\index{cognitive assessment}\index{validity!ecological}
In a random thought sampling procedure, participants are given a beeper or some other device that beeps randomly.\index{cognitive assessment}
When the person hears the beep, they must record what they are doing and what they are thinking—either quantitatively or qualitatively.\index{cognitive assessment}
The person hears a beep, and jots down the thought they were having when the beep occurred, so it is retrospective back a second.\index{cognitive assessment}\index{retrospective}

In the experience sampling method (ESM), the examiner can also ask participants to report the context in which the experience occurred.\index{cognitive assessment}
This can be helpful for those with schizophrenia who are hallucinating to help link the hallucinations to particular situations or contexts.\index{cognitive assessment}
Thought sampling and ESM are described by @Hurlburt1997.\index{cognitive assessment}

#### Pros {#cognitionRandomThoughtSamplingPros}

Pros of random thought sampling include:\index{cognitive assessment}

- The delay of recall is instantaneous—hence participants are not forced to recall what they were thinking.\index{cognitive assessment}
- The situations are [ecologically valid](#ecologicalValidity) in that they are occurring in situations that the person actually experiences, rather than a lab.\index{cognitive assessment}\index{validity!ecological}
- Random thought sampling gets at what people are actually thinking about rather than what they think they are thinking about.\index{cognitive assessment}
- Random thought sampling is better able to detect fluctuations in thoughts than retrospective approaches, and it is better able to detect co-variation with other processes, such as mood.\index{cognitive assessment}\index{retrospective}
- Random thought sampling involves less interference with the interaction.
Presumably, the "flow" of a situation is not interrupted...although it very well might be.\index{cognitive assessment}
- A client's [reactivity](#reactivity) could show therapeutic benefit.\index{cognitive assessment}\index{reactivity}

#### Cons {#cognitionRandomThoughtSamplingCons}

Cons of random thought sampling include:\index{cognitive assessment}

- It assesses narrow slices of time.\index{cognitive assessment}
- Because it assesses narrow slices of time, it is hard to capture the whole sequence of events.\index{cognitive assessment}
- It is odd to be interrupted by a beeper.\index{cognitive assessment}
- Random thought sampling could elicit [reactivity](#reactivity) such that thinking about one's thoughts could change thoughts or create thoughts.\index{cognitive assessment}\index{reactivity}
- Random thought sampling only accesses the contents of the consciousness, like the approaches discussed earlier.\index{cognitive assessment}
That is, the respondent has to be aware of the thoughts in order to write them down.\index{cognitive assessment}
There are likely many cognitive processes that nobody is aware of.\index{cognitive assessment}
- The written-down thoughts have to be coded.\index{cognitive assessment}
- Because random thought sampling typically occurs in an [ecologically valid](#ecologicalValidity) context, the experimenter has less experimental control of the situation, context, and stimuli.\index{cognitive assessment}\index{validity!ecologically valid}
Consequently, the experimenter is less likely to discover relations between rare but theoretically important processes.\index{cognitive assessment}

### Cognitive Science Approaches {#cognitionCognitiveScience}

Cognitive science approaches to assessing cognition include a range of non-mutually exclusive approaches, including performance-based measures, cognitive modeling, and cognitive neuroscience approaches.\index{cognitive assessment!modeling}\index{performance-based assessment}\index{cognitive neuroscience}
These cognitive science approaches are different from the approaches described earlier in that they do not rely on respondents reporting their thoughts.\index{cognitive assessment!modeling}\index{performance-based assessment}\index{cognitive neuroscience}\index{self-report}
Cognitive science approaches try to assess a person's cognitions based on behavior or neural functioning.\index{cognitive assessment!modeling}\index{performance-based assessment}\index{cognitive neuroscience}
For instance, using performance-based measures and cognitive modeling, the approaches are based on behavioral performance and then use a model to try to estimate what is happening cognitively—the participants just have to engage and process information.\index{performance-based assessment}\index{cognitive assessment!modeling}

#### Performance-Based Measures {#performanceBased}

Performance-based measures are used to assess many domains and constructs, including attention, executive functions, inhibitory control, memory, decision-making, categorization, etc.\index{performance-based assessment}
Examples of performance-based measures include the Stroop test, Iowa Gambling Task, and implicit cognition tasks such as the Implicit Association Test.\index{performance-based assessment}
In many performance-based tasks, respondents' accuracy and reaction times are evaluated when there are competing demands.\index{performance-based assessment}

#### Cognitive Modeling {#cognitionModeling}

Cognitive modeling attempts to decompose behavioral performance into different indices that reflect different sub-processes, such as working memory and processing speed.\index{cognitive assessment!modeling}
Behavior is complex and is influenced by many different processes, such as cognitive, motivational, and response processes.\index{cognitive assessment!modeling}
If you observe that a person shows behavioral deficits, it can be helpful to know what specific process is responsible for the behavioral deficits.\index{cognitive assessment!modeling}

It can be difficult to use multiple behavioral tasks to decompose basic processes because any task taps multiple cognitive processes, including likely overlapping processes, and each is affected by [measurement error](#measurementError).\index{cognitive assessment!modeling}\index{measurement error}
Cognitive modeling seeks to identify the "hidden processes" that underlie behavioral performance in complex tasks.\index{cognitive assessment!modeling}
Cognitive modeling generally requires a lot of data—for example, many trials—but it can be used with small sample sizes.\index{cognitive assessment!modeling}

@Busemeyer2002 provide an example of cognitive modeling of the Iowa Gambling Task to identify the sub-processes that account for behavioral performance deficits in Huntington's Disease.\index{cognitive assessment!modeling}
The Iowa Gambling Task (IGT) attempts to simulate real-life decision-making based on learning from decisions that have different probabilities of rewards and punishments.\index{cognitive assessment!modeling}
In the Iowa Gambling Task, participants are presented with four virtual decks on a computer screen.\index{cognitive assessment!modeling}
They pick a card each turn, and have to select from one of the four decks.\index{cognitive assessment!modeling}
Some decks reward the player more often and some decks penalize the player more often.\index{cognitive assessment!modeling}
The player has to figure out which decks to pick from based on statistical probabilities.\index{cognitive assessment!modeling}

@Busemeyer2002 compared and tested competing models for the task based on different theoretical possibilities.\index{cognitive assessment!modeling}
With cognitive models, the aim is to achieve a combination of accuracy and parsimony (simplicity).\index{parsimony}\index{cognitive assessment!modeling}
@Busemeyer2002 selected a model with three parameters: a cognitive process, a motivational process, and a response mechanism.\index{cognitive assessment!modeling}
A cognitive process was examined with an updating rate parameter, which describes a person's memory for past sequences produced by each deck.\index{cognitive assessment!modeling}
A motivational process was examined with the attention weight parameter, which describes the amount of attention a person allocated to gains versus losses.\index{cognitive assessment!modeling}
A response mechanism, such as recklessness and/or impulsivity, was assessed with threshold parameter, which describes the sensitivity of the choice mechanisms to the expectancies.\index{cognitive assessment!modeling}

@Busemeyer2002 found that people with Huntington's Disease showed deficits in the updating rate and threshold parameters but not in the attention weight parameter.\index{cognitive assessment!modeling}
In particular, the people with Huntington's Disease were more reactive to recent information and forgot old information more rapidly, and they became less sensitive with training and produced more random behavior.\index{cognitive assessment!modeling}
But people with Huntington's Disease did not show deficits in how much attention they allocated to losses.\index{cognitive assessment!modeling}

Other examples of clinically relevant cognitive model include cognitive modeling approaches to better understand cognitive processes in sexual aggression and eating disorders [@Treat2007].\index{cognitive assessment!modeling}

#### Challenges {#challenges-cognition}

Despite the advantages of not relying on participants' recall of their thoughts, there are challenges to using cognitive science approaches to assess cognition.\index{cognitive assessment!challenges}

##### Reliability of Difference Scores {#differenceScores}

One challenge related to the use of performance-based assessment of cognition is that many such tasks involve difference scores.\index{cognitive assessment!challenges}\index{difference score!reliability}
A difference score involves the subtraction of one score from another score.\index{cognitive assessment!challenges}\index{difference score!reliability}
In performance-based assessments of cognition, many tasks subtract scores, especially accuracy or reactive time, in one condition from scores in another condition.\index{cognitive assessment!challenges}\index{difference score!reliability}\index{performance-based assessment}
For example, the most-often used dependent variable in the Flanker task, Stroop task, stop-signal task, and dot-probe task is a difference score.\index{cognitive assessment!challenges}\index{difference score!reliability}\index{performance-based assessment}
For instance, in the Flanker task, the reaction time to incongruent (interference) stimuli is subtracted from the reaction time to congruent stimuli.\index{cognitive assessment!challenges}\index{difference score!reliability}\index{performance-based assessment}

The problem is that difference scores tend to be lower in [reliability](#reliability) than other scores because differences depend on the [reliability](#reliability) of both indices in the subtraction, as described in Section \@ref(differenceScore-reliability).\index{cognitive assessment!challenges}\index{difference score!reliability}
Difference scores tend to be lower in [reliability](#reliability) than each of the indices that compose it, especially when the two indexes are correlated.\index{cognitive assessment!challenges}\index{difference score!reliability}
Therefore, scores on these tasks tend to be less [reliable](#reliability) than scores on other tasks that do not involve difference scores.\index{cognitive assessment!challenges}\index{difference score!reliability}
To be [reliable](#reliability), difference scores require high [reliability](#reliability) of the individual indices compared to the correlation between them.\index{cognitive assessment!challenges}\index{difference score!reliability}
Otherwise said, the more two things are the same thing, the more likely that subtracting one from the other leaves [measurement error](#measurementError) rather than construct variance.\index{cognitive assessment!challenges}\index{difference score!reliability}\index{measurement error}

Because of the reliance of difference scores in the dot-probe task, @Rodebaugh2016 challenged whether attention [bias](#bias) toward threat exists or whether it is [stable](#stability) across time for those with anxiety.\index{cognitive assessment!challenges}\index{difference score!reliability}\index{stability!rank-order}\index{bias!attentional}

##### Reliability Paradox {#reliabilityParadox}

Another challenge with performance-based assessments of cognition is known as the [reliability](#reliability) paradox [@Hedge2018].\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}
Many basic cognitive paradigms, such as the Flanker task, were designed for detecting normative cognitive effects.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}
However, a downside of that is that not all of them are good for assessing individual differences.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}
That is, their scores are not [reliable](#reliability) for a particular person.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}
Low [reliability](#reliability) precludes making strong inferences and decisions about individuals and calls into question the [validity](#validity) of inferences regarding individual differences, or people's change, in scores on those tasks.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}

The *reliability paradox* is that despite some robust cognitive tasks showing well-established experimental effects, many of these tasks do not produce [reliable](#reliability) individual differences, including the Flanker task, Stroop task, and stop-signal task, etc.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}
Experimental effects become well-established, and therefore those tasks become widely used, when between-subject variability is low.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}\index{restricted range}
However, low between-subject variability leads to low [reliability](#reliability) of individual differences.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}\index{restricted range}
The [stability](#stability) coefficient, i.e., [test–retest reliability](#testRetest-reliability), an index of [reliability](#reliability) of individual differences relies on a Pearson correlation.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}\index{restricted range}\index{reliability!test–retest!coefficient of stability}
As described in Section \@ref(correlationConsiderations), correlation requires variability: restricted range leads to weaker associations.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}\index{restricted range}
When a measure has low between-subject variability, it prevents us from detecting consistent rank ordering of people across time.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}\index{restricted range}
That is, the very reason such tasks produce robust and easily replicable experimental effects due to low between-person variability makes their use as correlational tools problematic.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}\index{restricted range}
Moreover, the poor [test–retest reliability](#testRetest-reliability) of many performance-based assessments is exacerbated by the use of difference scores.

The implications of the [reliability](#reliability) paradox are that many well-established approaches in experimental, cognitive, and neuropsychology may not translate well to the study of individual differences, even though the measures can be useful for making group-level inferences.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}\index{restricted range}
It is important to know the [reliability](#reliability) of your measures, especially when dealing with issues of trying to understand where a particular person stands on the construct relative to other people.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}\index{restricted range}

[Unreliability](#reliability) and [measurement error](#measurementError) are threats to science and knowledge.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}\index{restricted range}\index{measurement error}
[Unreliability](#reliability) leads to false inferences and failures to replicate findings.\index{cognitive assessment!challenges}\index{reliability!paradox}\index{performance-based assessment!reliability}\index{restricted range}\index{measurement error}
As described in Section \@ref(effectOfMeasurementErrorOnAssociations), and formalized with the attenuation formula (Equation \@ref(eq:attenuationFormula)), associations with other variables are weakened to the extent that [measurement error](#measurementError) exists.\index{association!attenuation of}\index{measurement error}
If you know the degree of [unreliability](#reliability), you can account for it using the disattenuation formula (Equation \@ref(eq:disattenuationFormula)), to get more accurate estimates of the true associations with other variables.\index{association!disattenuation of}\index{measurement error}

It is important to test the [reliability](#reliability) of your measures and report the [reliability](#reliability) in papers.\index{reliability!reporting}\index{reporting}
Continually work to improve the [reliability](#reliability) of your measures.\index{reliability!ways to increase}
Ways to improve the reliability of measures are described in Section \@ref(increaseReliability).\index{reliability!ways to increase}
Another way to improve [reliability](#reliability) of scores is to aggregate multiple measures using a multimethod approach in [structural equation modeling](#sem).\index{reliability!ways to increase}\index{structural equation modeling}\index{aggregation}\index{methods!multiple}

### Cognitive Neuroscience Approaches {#cognitionNeuroscience}

Another branch of cognitive science approaches includes cognitive neuroscience approaches.\index{cognitive neuroscience}
For example, cognitive neuroscience approaches connect cognitive processes to measures of brain functioning, such as electroencephalography (EEG) or functional magnetic resonance imaging (fMRI).\index{cognitive neuroscience}
An example of linking cognitive processes to measures of brain functioning might be examining people's digit span in relation to fMRI measures.\index{cognitive neuroscience}
Cognitive neuroscience techniques are discussed in Chapter \@ref(psychophysiological).\index{cognitive neuroscience}
Cognitive neuroscience is a high priority of the National Institute of Mental Health with their emphasis on the [Research Domain Criteria](#rdoc) (RDoC).\index{cognitive neuroscience}\index{Research Domain Criteria}
RDoC seeks to assess the underlying substrates of illness across multiple levels of analysis and the neurodevelopmental trajectories rather than just behavioral symptoms, which are heterogeneous.\index{cognitive neuroscience}\index{Research Domain Criteria}

## Conclusion {#conclusion-cognition}

Multiple [aspects of cognition](#cognitionAspects) can be assessed, including cognitive products, processes, and structures or organization.\index{cognition}
The general [approaches to assessing](#cognitionApproaches) cognition include endorsement methods and production methods.\index{cognitive assessment}
[Cognitive assessments](#cognitionApproaches) include [self-report](#cognitionSelfReport), [think-aloud approaches](#cognitionThinkAloud), [random thought sampling](#cognitionRandomThoughtSampling), [performance-based measures](#performanceBased), [cognitive modeling](#cognitionModeling), and [cognitive neuroscience](#cognitionNeuroscience) approaches.\index{cognitive assessment}\index{cognitive assessment!modeling}\index{performance-based assessment}\index{cognitive neuroscience}
One challenge related to the use of [performance-based assessment](#performanceBased) of cognition is that many such tasks involve [difference scores](#differenceScores), which tend to be lower in [reliability](#reliability) than each of the indices that compose it.\index{performance-based assessment!reliability}\index{difference score!reliability}
Another challenge with [performance-based assessments](#performanceBased) of cognition is known as the [reliability paradox](#reliabilityParadox) that despite some robust cognitive tasks showing well-established experimental effects, many of these tasks do not produce [reliable](#reliability) individual differences due to low between-subject variability and the use of difference scores.\index{performance-based assessment}\index{reliability!paradox}

## Suggested Readings {#readings-cognition}

@Dunkley2019; @Treat2007