Multi-Modal Analysis of Body-Worn Camera Recordings: Evaluating Novel Methods for Measuring Police Implementation of Procedural Justice

The project seeks to develop the wider potential of body-worn cameras (BWCs) to improve practices and outcomes in policing.

Project Overview

Although the use of body-worn cameras (BWCs) has expanded rapidly, the capacity to efficiently analyze the enormous amount of data collected by BWCs lags far behind. As a result, the wider potential of BWCs to improve practices and outcomes of policing has gone largely unrealized. The purpose of this project is two-fold: (1) develop programming algorithms to automate analysis of BWC recordings of police-community interactions and evaluate officers’ adherence to principles of procedural justice, and (2) systematically compare the algorithmic evaluations of BWC recordings to ones done manually by human raters under conditions of high and low procedural justice.

This project has significant potential to transform how police agencies use body-worn camera (BWC) data as a training tool that will help to promote more constructive police-citizen interactions. The tools developed will make it possible for agencies to evaluate large amounts of BWC footage in far more accurate, efficient, and cost-effective ways than the status quo of random sampling and unstructured supervisory review. The tools can also be used by agencies to develop department-wide measures of procedural justice to support the rating and evaluation of officers’ encounters with citizens and to track their performance over time across the entire agency, as well as within particular working groups. Once fully implemented, agencies could use the tools to systematically track trends in procedural justice in the same way that they now track crime statistics.

Methodology

The study followed a three-phase methodology to develop and validate the automated software TrustStat, designed to assess body-worn camera (BWC) footage for procedural justice. In the first phase, Polis Solutions developed TrustStat, to analyze verbal and non-verbal cues associated with procedural justice principles. The second phase involved developing a coding instrument based on the four dimensions of procedural justice—neutrality, participation, trust, and respect. Human coders, including community members, police personnel, and academic faculty, were trained to use this instrument to rate the videos. In the third phase, the study compared TrustStat’s automated evaluations with those of human coders to test the tool’s accuracy and reliability in assessing procedural justice in police-community interactions. BWC recordings were obtained from the Dallas Police Department to fully evaluate the system’s performance. The project team is led by Dr. Colby Dolly, Director of Science and Innovation (National Policing Institute), and Jonathan Wender, Ph.D. (Polis Solutions), in collaboration with GE Research and the Caruth Police Institute at the University of North Texas at Dallas.

Results

A key finding of this study shows that despite the diverse demographic backgrounds among coders, they rated procedural justice similarly across all ten videos but demonstrated variability within videos. The results of this study show optimism that procedural justice can be systematically quantified by a diverse group of coders, even with a limited understanding of police practices. This finding has significant implications for law enforcement, indicating that automated tools like TrustStat can effectively assess procedural justice in police-community interactions.