SCOM 3050 Lesson 13: Quantitative Methods - Content Analysis

Overview

Content Analysis: A research technique for the systematic classification and description of communication content based on predetermined categories. It can be quantitative, qualitative, or a combination of both.

Quantitative content analysis involves counting occurrences (e.g., how often violence is depicted in media).
Qualitative content analysis involves interpreting the meaning behind the content, overlapping with rhetorical and other qualitative methods discussed in previous weeks.

Why Use Content Analysis?

Researchers use content analysis to measure behavior within media, such as how characters in films or television programs act, or how frequently certain themes appear in advertisements or articles.
It is commonly used to investigate themes like violence, bias, identity representation, or other significant patterns in media.

Manifest vs. Latent Content

Manifest Content: What is explicitly presented in the media (e.g., a character saying "I love you").
Latent Content: The underlying, hidden meanings behind what is explicitly shown, though content analysis typically focuses on manifest content.

Coding

Coding is a critical process in content analysis, where researchers classify and categorize the data they've collected based on predefined rules. Coding allows researchers to systematically organize and interpret large sets of data, making it easier to quantify and analyze the information.

Types of Coding

Open Coding: This is the initial phase of coding where researchers start by identifying key themes, concepts, or patterns in the data. This approach is flexible and exploratory, allowing researchers to see what categories naturally emerge.
Axial Coding: Once the initial categories are identified, axial coding refines and links the categories by identifying relationships among them. This step helps in organizing the data around central themes.
Selective Coding: In this phase, researchers focus on the core categories that have emerged and further explore connections. This step helps develop a coherent narrative around the data.

Coding Process

Create Categories: Researchers must establish a clear system of categories based on the research question. For example, if studying violence in TV shows, categories could include "physical violence," "verbal aggression," and "implied violence."
Operational Definitions: Essential for clear, measurable research, operational definitions specify how a concept will be measured in the study. For example, define what counts as "violence" to ensure consistent coding.
- It is crucial to provide detailed operational definitions for each category. For example, if the category is "physical violence," the definition might include "any act where one character physically harms or attempts to harm another character, including hitting, kicking, shooting, etc."
- Coding for abstract concepts like "patriotism" or "beauty" can be difficult because these terms can be interpreted differently depending on context. Researchers must carefully operationalize these concepts to ensure clarity.
Mutually Exclusive Categories: Categories must be distinct, meaning that a single piece of content should only fit into one category. If categories overlap, coding will be inconsistent and unreliable. For example, "verbal aggression" and "insult" should be clearly distinguished to avoid overlap.
Intercoder Reliability: To maintain consistency, especially in larger studies involving multiple coders, researchers should test consistency between different coders, known as intercoder reliability. This process involves having several coders analyze the same content independently and comparing their results.
- A high level of agreement between coders (typically 90% or higher) indicates reliable coding. If discrepancies arise, the operational definitions or coding procedures may need to be revised.

Sampling

Sampling refers to the process of selecting a representative subset of content from a larger pool to study. In content analysis, sampling is crucial because it directly affects the generalizability and reliability of the findings.

Sampling and coding are closely related because the way you select and code your sample affects the validity of your research project. A well-chosen, representative sample ensures that the data collected can be generalized to the broader population, while accurate coding ensures that the data collected is reliable and meaningful.

Sampling Process

Defining the Population: The first step is to define the population of content you are analyzing. This could be all TV shows aired between 2000 and 2020, all magazine advertisements published in a specific year, or all social media posts on a particular topic.
Establishing a Sample Frame: Once the population is defined, create a sample frame—a complete list of all potential items that could be sampled. For example, if studying newspaper articles, the sample frame could be the archives of all editions of a particular newspaper.
Selecting the Sample: Based on your research question, choose the most appropriate sampling method. For instance, if studying how violence in children’s TV shows has evolved, you might use stratified sampling, selecting shows from different time periods and categories (e.g., comedy, action, educational).

Advantages and Challenges of Content Analysis

Advantages

Unobtrusive: Content analysis does not interfere with the subject matter, allowing for objective results.
Inexpensive: Most content, such as magazines or TV shows, is readily available for analysis.
Quantifiable Data: Provides measurable data that can be interpreted and compared across studies.

Challenges

Defining Categories: Developing operational definitions can be difficult, especially for abstract concepts like "violence" or "gender roles."
Coding Reliability: Ensuring coders interpret content consistently is a common challenge.
Sample Selection: It’s important to ensure the sample is representative of the broader media landscape; otherwise, the results might not be reliable.

Example: Analyzing Media Coverage of 2024 U.S. Presidential Election Candidates

Screenshot 2024-10-20 BBC Coverage of 2024 US Election

Research Question

How do major news networks portray the leading candidates in the 2024 U.S. Presidential Election?

Step-by-Step Process

Define the Research Question: What do you want to find out?
- Example: "How frequently do major news networks portray the 2024 U.S. presidential candidates in a positive, neutral, or negative light?"
Develop a Hypothesis: What do you expect to find?
- Hypothesis: "Leading news networks tend to portray candidates from their preferred political party more positively while portraying opposition candidates more negatively."
Operationalize Key Terms: How will you define key concepts?
- Define "positive portrayal" as any instance where a candidate is shown in a favorable light, including praise for policies, personal attributes, or achievements.
- Define "negative portrayal" as instances where the candidate is criticized for policies, actions, or personal issues.
- Define "neutral portrayal" as coverage that neither praises nor criticizes but simply presents factual information without judgment.
Select a Representative Sample: What content will you analyze?
- Select 5 major U.S. news networks (e.g., CNN, Fox News, MSNBC, ABC News, and NBC News) and gather a sample of 10 election-related news segments or articles from each network over a 2-month period leading up to the election.
Create a Coding System: How will you categorize the data?
- Candidate is portrayed positively (Yes/No)
- Candidate is portrayed negatively (Yes/No)
- Candidate is portrayed neutrally (Yes/No)
- Focus of the coverage: Policy, Personal Attributes, or Campaign Strategy
- Mention of candidate's position on key issues (e.g., economy, immigration, healthcare)
Test for Coding Reliability: Ensure consistency among coders. Have two coders independently analyze the same news segment using the coding sheet. Compare results to ensure they categorize positive, negative, and neutral coverage consistently.
Analyze the Sample: Collect and quantify the data. Watch or read the selected news segments and articles, and use the coding sheet to categorize the portrayal of each candidate. Count the number of positive, negative, and neutral portrayals across networks.
Present Results: Quantify the findings.
- Example findings: "In the 50 news segments analyzed, Fox News portrayed the Republican candidate positively 70% of the time and the Democratic candidate negatively 60% of the time, while CNN portrayed the Democratic candidate positively 65% of the time and the Republican candidate negatively 55% of the time. Neutral coverage was more common on ABC News and NBC News."
Interpret the Results: Based on the findings, conclude whether there is evidence of bias in how networks portray candidates from different political parties. The results might suggest that certain networks favor candidates from their preferred party, reflecting partisanship in media coverage of the election.