Gerald Cuesta stared at the chart in disbelief, as the lines kept going lower and lower.
The television writer was observing the reaction of a test audience to “Babylon Fields,” a show he was trying to get on NBC’s schedule for the 2014-2015 season.
The quirky drama about the dead coming back to life got off to a decent start in the focus group. But a scene featuring cannibalism sent the score “down to grandma’s basement,” Mr. Cuesta said, adding, “I knew right then and there we’re not getting on the air.”
He was right. Despite having a fan of “Babylon Fields” in NBC Entertainment President Jennifer Salke, Mr. Cuesta’s dead wouldn’t rise after the test.
Mr. Cuesta holds no ill will toward his mysterious judges. “It’s simply part of the process. If you’re going to create TV, the chances that you will go through this are great.”
Over the past month, lots of writers and producers have come to know Mr. Cuesta’s pain. This is the time of year when the broadcast TV networks are busy figuring out their prime-time lineups for the next season. They’ll show off their schedules to advertisers in New York this week at glitzy presentations and parties.
More in Media and Advertising
Among the shows that have already made the cut are a television version of the hit movie “Lethal Weapon” on Fox, the comedy “The Good Place” on NBC starring Kristen Bell and Ted Danson—about a mean woman who tries to change her ways—and the drama “Designated Survivor” on ABC starring Keifer Sutherland as a cabinet member who gets thrust into the presidency after an attack on Washington.
The testing of new shows, known as pilots, is a crucial part of the process of deciding which ones to “pick up” and which to toss. The pressure to find new hits is intensifying in an industry racked by fragmented viewership and competition from digital media.
The firm that has been playing judge and jury to much of the television and film industry for more than 50 years is Screen Engine/ASI. The company tests some 250 pilots a year; programs that got their thumbs up after being tested there include NBC’s “ER,” ABC’s “Modern Family,” and FX’s “American Horror Story.”
The most traditional pilot test is to gather roughly 50 people—half men and half women—in a screening room. Viewers are recruited based on age and sex and race, as well as a network’s core demographic. “It’s like putting on a wedding each time you do one,” said Kevin Goetz, chief executive of Screen Engine/ASI.
The test audience watches a show and grades it with a hand-held device that has a knob the viewer moves to the right or left based on whether they like or dislike what they are seeing.
Meanwhile, some new players in media are trying to reinvent audience testing to improve on perceived drawbacks of the traditional model.
Amazon.com AMZN -1.12 % posts pilots of new shows for its streaming service on its website, where a large number of users can rate them before the company makes its decisions. New media specialists BuzzFeed and Vox, which both received investments from Comcast Corp. CMCSA -0.67 % , see the potential to use online videos as a testing ground to see what might develop into TV shows. (Streaming giant Netflix Inc. NFLX 0.16 % says it doesn’t test shows.)
For its part, ASI is now testing whether shows are “bingeable” in an era when watching multiple episodes of serialized shows in a single sitting is becoming more common. In one test for a cable show, sample audiences watched 10 episodes of a show over four days, with intense focus groups along the way to figure out “at what point in the season are you in or out,” Mr. Goetz said.
Many in the industry loathe the idea of random people playing tastemaker and telling them a show that they’ve spent millions of dollars developing stinks.
Last month, the Fox comedy “The Grinder” about a TV lawyer played by Rob Lowe who quits showbiz to join his family’s law practice, poked fun at pilot testing. In an episode titled “A System on Trial,” Mr. Lowe’s character, Mitch Grinder, takes a colleague with him to a shopping mall to find people for a focus group. He says he wants people with “no distractions” and “no thoughts” who are “blank slates” and then “we blindly trust those people and allow them to steer us.”
Former NBC Entertainment President Warren Littlefield who is now an executive producer of the FX cult hit “Fargo,” said testing is “absolutely gut-wrenching.”
Most shows, he added, don’t test well, and that is especially true for anything a little different. Perhaps the most famous show to test poorly, he recalled, was the NBC hit “Seinfeld,” whose characters weren’t seen as likable enough.
Mr. Littlefield believes testing for mass-market appeal is less relevant because there are so many shows on cable TV that offer edgy plotlines and draw small but loyal audiences.
- J. Walter Thompson CEO Forms Diversity Committee May 13, 2016
- Playboy Unveils ‘Safe For Work’ Web Video Slate May 13, 2016
- Vine Stars Are Leaving for Facebook and Other Platforms May 13, 2016
- Forbes Tests New Tactics to Combat Ad Blocking May 13, 2016
- Mark Penn Is Out to Reinvent the Ad Holding Company May 12, 2016
- Facebook Tweaks Rules on Video Ads After Mic Tests New Units May 12, 2016
“The research was to sand down the edges that possibly offended or pushed away a larger audience. Today we seek those edges and embrace them,” he said.
HBO eschews testing altogether. “Our bet is on the creative vision, and you can never measure that by just looking at a pilot without any context,” said Mike Lombardo, president of programming for HBO.
Still, testing has plenty of backers. “I don’t believe we’re that smart and that we know what the rest of the country wants to see,” said Preston Beckman, a former top programming executive at Fox and NBC.
Showtime Chief Executive David Nevins said he uses testing to see how specific stories and characters are resonating with viewers. Testing can tell you “if they’re confused, into it, or bored,” which helps with the editing process, he said.
For writers watching audiences dissect their shows, emotions can get out of control.
“I’m part therapist in there,” said Twentieth Century Fox Television President Jonnie Davis, describing what it is like to watch a writer observe a test. “Have I ever had to jump in front of the glass to stop a [show] creator from ‘going Frankenstein’? Yes.”
Write to Joe Flint at [email protected]