Levine warns against early focus testing

2K Boston creative boss Ken Levine has warned developers of the dangers of focus-testing games too early - saying that nobody would have wanted to play Bioshock if the idea had been focus-tested years before release.

Speaking at the Develop conference this morning, Levine said that if Irrational games (as the studio was then called) had gone to a focus group and said, "how would you like to play a first-person shooter set in an underwater objectivist utopia", "nobody would have wanted to play that".

He said there had been a similar experience with the development of Thief, a game which deliberately flaunted the conventions of the top FPS games of the time like Quake and Unreal. "Here was a game where you had a sword and a bow and arrow, and you had to hide under a dresser for half an hour... It's not exactly saying, rev up the focus group, they're going to love this.

"We had people going to focus test groups and asking, 'Would you like more weapons or less weapons?'" explained Levine - the answer, of course, being obvious. "Marketing people have really improved their methods, and it's much more sophisticated now, but you can still get some really bad data out of these things."

Later on in development, when the game is coming together and the scope of potential changes are reduced, Levine does believe that focus testing has a place.

"You have to be the author," he said. "You can't move with the wind. But if everyone tells you they hate something... They can't tell you how to fix your game, but you have to be receptive to them saying there's a problem."

(This is the last story we'll do from that session, we promise - although we'll be chatting to Levine in person later on, and will let you know if he says anything interesting.)