Keith Stuart on AI, acting and the weird future of open-world games
Why, AI?
Julian Togelius has an idea about where open-world games are going. Let's call it the infinite world theory. In his version of the future, titles like Skyrim and Grand Theft Auto will have no set missions or narrative arcs, and no pre-defined landscapes. Instead, the game engine will use procedural generation, artificial intelligence and creative computing techniques to dynamically build environments and experiences to suit every individual player. In the Togelius infinite world, you'll be able to drive a car in one direction for several miles and find that the game has built a city at the end of the journey, just for you. What's more, that city will be populated with characters who act like real humans rather than bizarre automatons. It will be the player's interactions with these characters that creates the stories.
Although this sounds like science fiction, Julian Togelius is a university professor who has spent the last five years researching the concept of procedurally generated content in games. His infinite world theory is not a fantastical dream, it's a distinct possibility. "[Procedurally] creating new environments, including things such as mountains, vegetation, houses, etc, is pretty straightforward," he says. "There are out-of-the-box solutions available to all of these problems individually. It's just a matter of integrating them and putting in the adaptivity. This would provide a big increase in perceived freedom, but it would also enable the game to adapt to the player, to create environments that it thinks the player wants..."
It's a good time to be thinking about this sort of stuff. This week, EA announced that it is working on a big open-world action project, hiring Assassin's Creed producer Jade Raymond to oversee the studio. "We've never really operated in the largest genre of gaming," said the company's chief financial officer Blake Jorgensen. "That's the Assassin's Creed-style games; more open-world, more single-player versus multiplayer."
But you have to ask, where can EA go with this? Big names like Grand Theft Auto, Assassin's Creed and Witcher have taken the current formula just about as far as they can. All offer a central narrative told through a sequence of story missions, all supplement this with an explorable landscape and discoverable side-quests. This is how the open-world genre has worked for a decade.
But the cracks are showing, and the key problem is asset generation. As games get bigger and more ambitious, developers need exponentially larger teams to write the stories, script every encounter and design every building, object and vehicle in the universe. Eventually, this approach will drive development costs so high, the genre will collapse in on itself.
Michael Cook, a computational creativity researcher at Goldsmiths University has been thinking about this problem for a number of years. In 2011, he began development of ANGELINA, a computer program capable of designing its own games using assets drawn from image search engines. He thinks the future of this genre is about handing creative power to the machine itself. "The most crucial thing I think open worlds need to do is back off the brink of total scripting and hand-design," he says. "There has always been a tension between handcrafted missions and wild systems that let you invent magic or manipulate local politics. In my opinion, the systems are what bring open world games to life. Games where everything is scripted and static are only possible because the teams are so large and so efficient at making them - but we can't do this forever, and I don't think it'll lead us towards the next generation of open worlds. We need to build more systems in our games, things that players can interact with and break."
We're already seeing moves toward automated and reactive design in games. With Left 4 Dead, for example, Valve introduced the concept of an AI director which altered the intensity and style of zombie attacks based on player activities; for Skyrim, Bethesda created the Radiant quest system, which generated simple tasks based on player location and enemy types. Meanwhile, games like Minecraft, Dwarf Fortress and the forthcoming No Man's Sky have all used procedural generation to construct unique environments - and middleware packages like SpeedTree allow studios to generate areas of terrain without having to draw every last piece of foliage.
What these systems don't do, however, is give you the impression of a creative intelligence working behind the scenes to join all the dots and create complex new experiences - they're just limbs operating without a brain. How can we make an open-world game that looks like it is really responding to, and learning from, the player?
Ten years ago Jeff Orkin was a coder at Monolith Productions, masterminding the revolutionary AI systems behind games like No One Lives Forever and F.E.A.R. Later, he completed a PhD at MIT, looking into the concept of crowd-sourcing data for AI systems; in other words, he wanted to get computers to watch people play a game and then learn from what they did, using that knowledge to make interesting reactive non-player characters.
"Open world games are all about player stories; they're about having players walk away with memorable moments, caused by their own decisions and actions," he says. "For example, last year's Shadow of Mordor did a great job of producing memorable moments through their Nemesis system, which created randomised enemies with a memory of past encounters with the player that they could vocalise in their taunts. This is a step in the right direction. Contextual dialogue ties events together, and explains to the player what the NPCs are thinking, and how decisions made in previous encounters have led to the current situation. However, giving every NPC the ability to dynamically voice contextual dialogue is an intractable problem in an open-world game."
But Orkin thinks he has a solution. In his PhD project, The Restaurant Game, he asked 16,000 people to download and play a simple game in which a customer goes for a meal and is served by a waitress; at first he asked participants to play as either the diner or the server and just improvise a meal scenario - all the while, the computer was analysing and recording their actions and behaviours. Several thousand games later, Orkin was able to replace the waitress with an AI, equipped with a database of reactions drawn from those thousands of playthroughs. The result was a computer controlled character seemingly able to learn, adapt and behave like a real person.
In 2013 Orkin founded a new company, GiantOtter, where he and several veteran AI coders gradually refined the system into a new crowd-sourcing AI platform called GroupPlay, which the company is set to make available to development studios working on narrative games. During development, the team recruited testers to play a series of short RPG mini-games while improvising their own dialogue and interactions with the game world - all of this was recorded and added to a database of conversations and actions which could then be made available to NPCs. As Orkin explains, "by recording several gameplay sessions of each scenario, cross referencing them with other similar scenarios, and doing some tagging and clustering to associate semantic meta-data, we can generate a vast dialogue database to draw from depending on the context."
So let's say a game like Witcher implements GiantOtter's technology. The developer could spend a few weeks getting groups of designers and actors to play the game, mess about in the world, chat to each other, and then record all of that talk and action. These realistic experiences would then be stored in a database of discreet behaviours and dialogue segments, which would inform NPCs in the game. So when you, say, visit a blacksmith, whatever you say or do, even if you stand on his anvil and twirl your sword around your head, the AI will have a suitable response because a human tester, at some point in the past, will have already done that.
"The cool thing is that this approach does not only capture individual reactions, but entire conversations that can play out between multiple NPCs, breathing the illusion of intelligent life into the world," says Orkin. "These conversations can be intertwined with other similar conversations to produce a nearly infinite variety of sequences for the player to overhear".
GiantOtter has already performed a pilot test of the tech with one of the major open world game developers (he can't way which), and the results were convincing. As Orkin explains, "We got validation from designers that this kind of authoring via demonstration is actually a more natural way to express behavior and dialogue, as compared to trying to turn designers into programmers who have to encode branching gameplay logic, which often leads to brittle behavior when these scripted events are placed anywhere in an open world". He also foresees a future of community contributed dialogue, with players able to record their own voice performances, which the GroupPlay system could add to the game. Want a minor character in GTA VI to sound and act like PewDiePie? That's unfortunately possible.
So, crowd-sourced dialogue and actions can be used to inform AI characters and make them more adaptable and naturalistic; can the same kind of idea be applied to whole stories? Mark Riedl heads up the Entertainment Intelligence Lab at the Georgia Tech School of Interactive Computing. For the past three years he's been working on a project named Scheherazade, a crowd-powered automated story generation system, which creates its own interactive fiction by studying groups of plot lines developed by humans. In one experiment his team asked dozens of volunteers to write their own very simple bank robbery story. Scheherazade then analysed every narrative, worked out common elements, and produced its own interactive bank robbery text adventure. Later, test subjects were shown two versions of the bank robbery interactive fiction: one written by Scheherazade and one by a human. Most subjects were unable to tell which was produced by the AI.
Riedl's research is being funded by the US defence agency Darpa and the plan is to produce military simulations, based on the written experiences of veterans, which teach new soldiers how to, say, interact with civilians in unfamiliar countries. It takes a narrative designer many months to research and write dozens of different possible scenarios, but Scheherazade is able to keep generating new variations, simply by analysing reports filed by experienced military personnel. Riedl also sees the video game potential of the technology - one example he's given is a Star Wars RPG that creates endless narrative quests and subplots by reading online Star Wars fan fiction and working out common elements. Okay, you may end up with a lot of ewok/droid love affairs, but as Mass Effect has shown with its inter-species romances, people are totally down with that sort of thing.
What both Orkin and Riedl are working toward then is a concept known as drama management, a program of interlocking AI systems that serve quests and narrative adventures to players - often via intelligent NPCs - within a dynamic open world. "This is a growing and exciting area of research," says Cook. "A lot of work is being done on making game worlds both rich and unpredictable, as well as making them more controllable by designers. At the Experimental AI for Games workshop this year we had a paper called 'Toward Characters Who Observe, Tell, Misremember, and Lie' by folks at UC Santa Cruz.
"The paper's about a new AI framework that the researchers are using to make open-world narrative games about information and social relationships. Their game is called Talk Of The Town, and in it are hundreds of NPCs who know things about the town - where people live, where they work, when they were born, and so on. But just like the real world, these NPCs aren't perfect - they might forget what they know, or remember things wrongly, or they might have been lied to when they learned a piece of information. So when you're talking to people in the world, trying to discover what the locals know about someone, you have to think, is this person reliable? Do they have a reason to lie to me? Do they look like they know what they're talking about?
"For me this is a powerful new concept in AI systems - and it's a system, not a script. We're familiar with characters in games lying to us, but only when a writer or designer decides they should. This is a changing, malleable system that demands a new kind of thinking from players. Open world games are falling prey to over-scripting and over-design - they're more like really big closed worlds than truly open ones. We need these kinds of unpredictable and organic AI systems to add life back to our open world games."
Riedl agrees that the future of open world games should not just about bigger environments and more detailed visuals, the future should also be about social dynamics. Right now, when you're in a location like Los Santos or Tamriel, you know you're the only inhabitant with agency - everyone else is either a mindless pedestrian cycling through limited set behaviours or a quest giver who has one pre-written speech to deliver. But a deeper system, combining the Nemesis concept with the Talk of the Town technology would provide a world filled with dynamic Machiavellian plotters. And that means a very different set of games.
"If NPCs possessed social reasoning, then open-ended action games can be about power dynamics," says Riedl. "They may support strategies in which the player forges and betrays lasting relationships with NPCs instead of just shooting. Think of an interactive version of The Wire, House of Cards, or The Sopranos. These TV shows are about the weaving and unweaving of complex interpersonal relationships as a means to an end. This provides new, complex strategies that may complement the action-oriented aspects of the game. NPCs with innocuous beginnings could become regular companions, or grow into mortal enemies. An indication of success would be the player having a strong emotional response to the procedurally generated actions of an NPC, or to the betrayal or demise of a favourite character."
For Riedl one possible future for the open world game may be an experience that combines The Sims, which features lifelong ambition arcs, with something like Prom Week, the fascinating University of Santa Cruz project that simulates social interactions between students in the run up to their prom. The goal would be to produce characters that exhibit both short term and long term goals. Right now, he is using stories to teach computer characters how to behave believably - and also morally. By studying stories of simple human interactions such as going to a chemist to pick up a prescription, his program is discovering for itself certain acceptable and unacceptable behaviours.
What all of these AI researchers agree on is that the future of open world game design is going to be less about lots of scripted stuff being fed to the player, and more about the player collaborating with the computer to create fresh, personal stories together. While at the University of Copenhagen, Julian Togelius and his students created a platformer named Super Mario Evolution, which would watch how each player explored the world, whether they preferred jumping around, running, or collecting objects, and then built new levels, on the fly, based on their preferences. "The game is curious about you," he says."The game plays the player".
Michael Cook, meanwhile, has discussed developing two games: Murder and Mystery. In the first you play a guest at a Cluedo-style dinner party trying to assassinate the host. In Mystery you play a detective sent to investigate a murder. The idea is that an AI system would watch games of Murder to invent case files for Mystery.
"There's a lot of work being done to get AI to understand what the player wants and to have the AI either help or hinder those goals. Martin Cerny's Sarah And Sally is looking into how to solve the problem of AI companions in games, for example.
"But I think really good open world games should be like improv theatre - you do something dramatic, and the game thinks about what it can do to add the most drama, tragedy or comedy to the game. Grand Theft Auto shouldn't be about static missions that come one after another; it should be an improvised story that you and the game tell each other. We think about AI mostly as something that tries to stop us from winning. Really, we should be thinking about it as a friend who wants us to have as much fun as possible."
So, EA, if you're looking for ideas on how to advance the open-world genre rather than just apply the same old features to a new setting, the future is about crowd-sourced AI, about capturing player activity, and about stories that evolve around the individual participants. The future is collaboration between program and player, the former learning from the latter. But that won't be easy.
"I think co-creation research can be really helpful, but it's still early days," says Cook. "Understanding what a human wants is as hard for AI as it is for other people".