• Jessica Clark, Patricia Aufderheide
• Center for Social Media

Read the full text

07 October 2011This working paper aims to synthesize current efforts to develop comparable evaluation methods for social issue documentary films. Authored by two researchers who have been jointly documenting the field’s transformation over the past five years, this paper offers a framework for planning and evaluating the impact of these films in a networked media environment.

Tracking impact has become increasingly complex as platforms and content streams proliferate, and campaigns evolve over several years. A single piece of media can now spread across a variety of screens—a theater, a university classroom, the Web, home televisions, a mobile phone and more. Each screening carries with it different expectations, different measurement schemes, and different potential publics—i.e., groups of users for whom the film and related campaigns serve as a catalyst for debate—as well as advocates who seize upon the film as a hub for action. Cheaper production and distribution tools, new channels, and increasingly skillful and networked users are challenging previous assumptions about how social documentaries reach users, and offering powerful but vexing opportunities for collaboration and organizing.

The transition from 1.0 to 2.0 opens opportunities for documentarians to fulfill and expand their missions—not only informing individuals and leading public conversation but also building community cohesion and participation. Documentaries travel differently in this new media ecosystem, and they can also play a role in shaping its development.

As a result, evaluating such efforts requires a deep understanding of the mission and intended audiences for each project, and both qualitative and quantitative research methodologies. Given the quick-shifting digital terrain, mobile and documentary producers are operating in a rapid prototyping mode, experimenting with and refining a variety of distribution, outreach, and networking techniques. This makes it difficult to develop comparative assessment frameworks, instead refocusing evaluators’ attention on whether project goals were met, appropriate publics engaged, and unexpected publics pulled into the mix of discussion and action.

Such new discoveries are leading filmmakers in directions they could not have predicted at the start of their projects—creative opportunities that lead to innovations in narrative form and the shift from filmmaking to other modes of communication. Successes and failures alike drive such strategic shifts— finding a fortuitous partner, or an angle or clip that goes viral, or sinking money into a web site that ends up yielding little interaction.

Documentarians are becoming more nimble, adopting research and planning methods that more closely resemble those associated with fields such as product design, agile software development, and community organizing.

Drawing insights from the design thinking field—a user-focused creation process that has emerged from the commercial design field and is now being applied to create and improve social sector projects—this working paper examines state-of-the art methodologies for strategic design and evaluation of documentaries.