Methodology

Foreign Influence Election 2020

The Foreign Influence Election 2020 (FIE 2020) project identifies, catalogues and analyzes English-language reporting disseminated by non-U.S. state-sponsored media outlets. State-sponsored content related to the 2020 presidential election, the 2020 candidates and significant campaign issues are aggregated daily. The FIE 2020 analyzes these news stories and then reports on the overt content of three authoritarian regimes broadcasting English-language media to American audiences: Russia, Iran and China. All three countries have robust state-sponsored media outlets, and all three countries have demonstrated a willingness to employ their state-sponsored outlets to advance narratives and themes which directly support the interests of their parent states. 

Sources of state-sponsored media:

At present, the FIE 2020 project analyzes four state-sponsored media outlets from the three countries named above. 

Country	Outlet	Website
Russia	Russia Today (RT)	https://www.rt.com
Russia	Sputnik International	https://www.sputniknews.com
Iran	PressTV	https://www.presstv.com
China	The Global Times	https://www.globaltimes.cn

 

These four outlets were selected based on brand-name recognition, market penetration and observed patterns of overt influence. FIE 2020 examines two Russian outlets because both were observed as significant factors in the 2016 U.S. presidential election, better illustrate the breadth and depth of Russian state-sponsored media and each outlet serves different purposes and mediums. Russia Today serves as both a television service and news website, Sputnik International as a news website and radio broadcast service. Both have a significant presence in the U.S. and both were routinely observed as information sources amongst Kremlin social media campaigns during the 2014 to 2016 period. (For background, see here.) As the project progresses, and should additional project resources become available, the FIE 2020 team would like to expand its analysis to additional countries and their respective state-sponsored media outlets.  

Selection of state-sponsored content:

From these four news sites, the FIE 2020 team selects English-language content pertaining to the 2020 U.S. Presidential election that meet one or more of the following criteria:

• • Explicit mention of the president
  • Explicit mention of a presidential candidate
  • Explicit mention of a relevant political individual pertaining to the election
  • Explicit mention of a contemporary event or political issue discussed in the context of the 2020 presidential election

The FIE 2020 does not analyze all content produced by each of these outlets each day.  However, the selection of elected stories represents a sizeable sample of all state-sponsored content pertaining to the 2020 presidential election. For perspective, the FIE 2020 database, at the time of launch on 19 November 2019, has analyzed more than 3,300 news stories arising from the above mentioned news outlets.  

Duration of state-sponsored content collection:

The FIE 2020 project collects and analyzes state-sponsored news stories from 1 January 2019 to the present day. The FIE 2020 Project will continue this collection and analysis through the 2020 U.S. presidential election on November 3, 2020. 

Distribution for analysis:

The FIE 2020 management team selects state-sponsored news stories from the four designated news outlets and records them in a central database. From that database, FIE 2020 researchers receive a selection of stories to analyze. Researchers receive a mix of news stories from different news outlets and time periods. No single researcher or set of researchers receives all of the selected stories from an outlet or specific time period. The diversity of source content delivered and time periods assessed by any one analyst seeks to thwart any selection bias arising from giving all of a single outlet’s content to one analyst.  

Analysis of state-sponsored content:

The FIE 2020 researchers read stories delivered to their portfolios and analyze the content across a range of research questions. Using a standard form, researchers annotate:

• • Time and date of publication 
  • Author, if applicable
  • External links and social media content and personas mentioned 
  • Individuals and organizations supporting 2020 candidates and their policies
  • Mentions of candidates or presidents
    • Mentions include an assessment of how the news outlet characterizes the candidate or the president. The assessment of the mention is:
      • Favorable – Offers a favorable opinion of the candidate or the president
      • Unfavorable – Offers an unfavorable opinion of the candidate or the president
      • Neutral – Mentions the candidate or the president without discernible bias, usually a statement of fact or a direct quote from the candidate or the president

Analysis of mentions as favorable, unfavorable or neutral can be challenging and subjective. Some outlets—and the content they produce—are distinctly overt in their favorability of a given candidate or president. Others are more subtle, selecting several negative references to a candidate or the president from outside sources without offering a counterpoint. To develop consistency in researcher analysis, the following measures were taken with regard to the favorability assessment: 

• • All researchers receive training from the outset as to how to analyze and transmit their analysis via a standard form. 
  • All researchers train on a set of news stories and receive feedback on their test set before performing analysis on the dataset.
  • All researchers flag any content they are unsure how to assess, pushing tougher judgments up for further review.
  • All researchers are instructed to assess any mention they cannot easily discern as favorable or unfavorable as “neutral”. 
    • Neutral is the default assessment of any mention and upcoming analysis will demonstrate that regardless of the state sponsored news outlet, the majority of candidate/president mentions are neutral.
  • Researcher questions regarding the analysis process and the procedures for assessment are recorded and maintained as a reference guide for all current and future researchers as they assess articles.  
  • The FIE 2020 management team conducts quality control assessments of submissions after they are entered and conducts follow up questions or modifies the submissions as needed. 

Quality control checks:

The FIE 2020 program management team periodically downloads the full database of researcher submissions and conducts quality control checks on the submissions to identify any errors in data entry, evaluate researcher submissions for accuracy and address any concerns raised by the researchers. Once cleaned and formatted, the coded reports are entered into the FIE 2020 research database. 

Analysis and reporting:

The program management uses the FIE 2020 database to develop research projects and reports. Analysis of the complete dataset results in articles, charts and reports is published on the FIE 2020 website at the Foreign Policy Research Institute (FPRI).  

Update (March 6, 2020): After Super Tuesday and the narrowing of the field in election 2020, the FIE 2020 team will deliberately adjust the scope of its coding operations. As of March 9, 2020, the FIE team will code stories only explicitly related to the U.S. presidential election, as opposed to those that simply mention a candidate, allowing the team to shift focus onto other related analysis and deliverables. The process by which FIE 2020 will catalogue stories mentioning 2020 candidates will remain the same.

Update (November 15, 2020): The Foreign Influence 2020 project will continue to analyze overt state-sponsored media coverage of the 2020 election and its results in the post-Election Day period, past Inauguration Day.