Methodology

The Inclusion List was determined through a process of assigning points for films that met inclusion metrics. A total of 20 points could be earned– 10 for on screen inclusion and 10 for behind the camera inclusion.

To determine on screen inclusion scores, two distinct processes were used, one for leads/co leads and the other for all speaking characters. A film could earn up to 10 points for on screen inclusion based on the 5 inclusion indicators (gender, race/ethnicity, LGBTQ+, disability, age) across leads/co leads (up to 5 points) and all speaking characters (up to 5 points).

For films with ensemble leads (n=22), or those with three or more leading characters, a decision to award points was based on whether the constellation of leading characters achieved the benchmarks required for speaking characters across each inclusion indicator. Thus, if a film had 4 leading characters, to earn a point for the gender metric, at least 2 leading characters must be girls/women. For LGBTQ+, at least one character must be LGBTQ+.

In addition to leads/co leads, the percentage of all speaking characters who achieved a given metric was used to determine whether the film earned a point. Thus, if 33% of all speaking characters were underrepresented, the film would earn a point for that indicator.

Behind the camera, it was possible for a film to earn up to 1 point for each behind the camera position evaluated– 10 in total. The positions were: Director, Writer, Producer, Cinematographer, Editor, Composer, Production Designer, Costume Designer, Casting Director, and First Assistant Director.

Up to 1 point was allocated if a woman or person of color was credited in a specific position. So, a film with a woman director earned 1 point. For positions in which multiple individuals were credited, a fraction of a point was allocated for every woman and/or person of color in the role. If two writers were credited and one was a person of color, the film earned 0.5 points for that position. Additionally, a bonus point was awarded to any film that had a woman of color director.

There were two exceptions. For casting directors and costume designers, the historically gendered nature of the role required a different approach. For these positions, if the credited individual was a person of color, the film received a point.

Points were totaled across all available metrics on screen and behind the camera, and films were ranked according to their full point total. The Inclusion List is comprised of films that earned between 5.3 and 15 points across all inclusion metrics.

After selecting the time frame of interest, we selected a set of film distributors based on the number of films released from 2019 to 2023 that were among the 100 top-grossing each year. Distributors with 3 or more films were included. Then, every film released by those distributors in that time frame was included in the sample, with four exceptions: films that grossed less than $1 million (USD) in domestic revenue, re-releases, documentaries, and animated films were excluded (the revenue cutoff in 2023 was $3 million). This resulted in a final sample of 476 films.

Lead/co lead judgments were made in two ways. Films were viewed by members of the Annenberg Inclusion Initiative staff team to identify the protagonist, or the character driving the action in the story. When a film could not be viewed by staff members, judgments made by research assistants were used and online evidence was sought to corroborate information.

 Data regarding on-screen characters and leads/co leads were collected by the Annenberg Inclusion Initiative team. At least two research assistants viewed each film in the sample to identify every speaking or named character that appeared on screen. A total of 19,095 speaking characters were evaluated across all the films in the sample. Once a speaking character was identified, additional information about the character’s gender (male, female, non-binary); race/ethnicity (White/Caucasian, Hispanic/Latino, Black/African American, American Indian/Alaska Native, Native Hawaiian/Pacific Islander, Asian, Middle Eastern/North African, Multiracial/Multiethnic); age (child, teen, young adult, middle aged, elderly); LGBTQ+ identity (lesbian, gay, bisexual, transgender, not lesbian, gay, bisexual or transgender); and disability (present, absent). At the analysis level, race/ethnicity was collapsed to reflect a measure of underrepresented (Hispanic/Latino, Black/African American, American Indian/Alaska Native, Native Hawaiian/Pacific Islander, Asian, Middle Eastern/North African, Multiracial/Multiethnic) vs. not underrepresented for each character. Reliability of coder judgments was assessed and disagreements were adjudicated by a member of the Annenberg Inclusion Initiative staff to arrive at a final answer.

A total of 10 behind-the-scenes roles were analyzed: Director, Writer, Producer, Cinematographer, Editor, Composer, Production Designer, Costume Designer, Casting Director, and First Assistant Director.

The data collection process involved identifying all individuals who worked in these roles for each film in the sample using IMDbPro. Across the sample, there were 2,802 crew roles evaluated in 2023, which were added to the 8,838 evaluated from 2019 to 2022. When the position was not listed on IMDbPro, information was sought from other databases or from film credits.

Across positions, specific credits were included for each role. In general, location-based or unit-based credits were excluded from analysis. Writers included those earning “written by,” “story by,” or “screenplay by” credit and excluded writers responsible only for source material. Only Producers receiving “Produced by” credit were included. When no Costume Designer was listed but a Costume Supervisor was credited, that title was used. Individuals who were credited as “First Assistant Director” were included, and only when no First Assistant Director was credited on a movie was the individual(s) earning “Assistant Director” with no additional modifier included in the sample. Casting Directors were those responsible for overall casting, and location-based casting directors were included in two specific instances. The first was when there was no overall casting director listed and filming took place in only one location. The second was when the overall casting directors both had location based information (e.g., U.S., U.K.) and the film was not shot in those locations. Those casting directors were presumed to be responsible for casting the full production.

Gender and race/ethnicity information was then obtained for each individual using online sources, such as Studio System by Gracenote, IMDbPro, U.S. Census records, interviews, and social media posts. Gender was categorized as male, female, or non-binary, and race/ethnicity categorized as white or underrepresented (including Black/African American, Hispanic/Latino, Asian, American Indian/Alaska Native, Native Hawaiian/Pacific Islander, Middle Eastern/North African, and Multiracial/Multiethnic).

When no information on race/ethnicity could be found, a photograph of the individual was obtained, and senior research team members rendered a judgment on the individual’s race/ethnicity and/or gender. In the past, this process has been used to determine gender and race/ethnicity of series regulars, and there was a .90 correlation between independent judgments made by the research team and documented information on racial/ethnic identity across 2,175 series regulars appearing in episodic content from the 2014-15 season.

There are limitations to this methodology, including the potential for incomplete or inaccurate information available online and the use of limited information to make judgments on race/ethnicity for individuals where no information could be found. However, efforts were made to obtain as much information as possible to ensure the accuracy of the analysis.

Gender List. Films must be at gender balance (44.55%) and be female led. Behind the camera, it was possible for a film to earn up to 1 point for each behind the camera position evaluated– 10 in total. The positions were: Director, Writer, Producer, Cinematographer, Editor, Composer, Production Designer, Costume Designer, Casting Director, and First Assistant Director.

Up to 1 point was allocated if a woman or person of color was credited in a specific position. So, a film with a woman director earned 1 point. For positions in which multiple individuals were credited, a fraction of a point was allocated for every woman and/or person of color in the role. If two writers were credited and one was a person of color, the film earned 0.5 points for that position.

There were two exceptions. For casting directors and costume designers, the historically gendered nature of the role required a different approach. For these positions, if the credited individual was a person of color, the film received a point.

Race/Ethnicity List. Films must be at underrepresented proportional representation and be led by an underrepresented individual. It was possible for a film to earn up to 1 point for having a person of color in any evaluated department. This was the case for all positions except producers who were awarded up to 1 point based on the number of underrepresented producers in the department. If half of the producers were from an underrepresented racial/ethnic group, half a point was awarded etc.

LGBT List. Films must be LGBT Led. The films were ranked by those which had the highest percentage of LGBT characters.

Disability List. Films must be led by a person with a disability. The films were ranked by those which had the highest percentage of characters with a disability.

Age List. Films must be led by a person 65+. The films were ranked by those which had the highest percentage of characters who were age 65 or older.

From each group, the top five films were selected for The Inclusion List, based on the overall percentage of characters from that group. Note that “led” means at least one lead/co lead is a member of that group, or if the film is led by an ensemble cast, then the ensembles must be at proportional representation.

Top Inclusive Filmmakers. The directors who have worked with the most women and/or people of color as cinematographers, editors, production designers, or assistant directors across the last 5 years.

Top Inclusive Editors. Women and/or people of color working as editors whose work appears most often on the Inclusion List of the top-ranking US originating films across the last 5 years.