Boffins build AI agents that respond like real people • The Register
Computer scientists have devised a technique for making AI models behave like specific people.
The researchers from Stanford University, Northwestern University, the University of Washington, and Google DeepMind describe their work in a pre-print paper titled “Generative Agent Simulations of 1,000 People.”
The US-based authors claim that by using their generative agent architecture, they were able to train generative AI models simulating at least 1,000 different people, such that the resulting models responded to a set of common social science survey questions in a way that closely matched the responses given by the people being simulated.
“We present a generative agent architecture that simulates more than 1,000 real individuals using two-hour qualitative interviews,” the paper explains. “The architecture combines these interviews with a large language model to replicate individuals’ attitudes and behaviors. By anchoring on individuals, we can measure accuracy by comparing simulated attitudes and behaviors to the actual attitudes and behaviors.”
The two-hour interviews consisted of a series of interview questions developed by sociologists as part of the American Voices Project. They involved questions like “Tell me the story of your life – from your childhood, to education, to family and relationships, and to any major life events you may have had,” and “How have you responded to the increased focus on race and/or racism and policing?”
Study participants then answered questions from a set of common tests, including General Social Survey, Big Five Personality Inventory, economic games (e.g. Prisoner’s Dilemma), and various behavioral experiments.
The responses of study participants were then fed into an AI agent architecture that injects the entirety of participant responses into the AI model’s prompt when the LLM agent is queried – an approach made possible by recent advances in long-context understanding. While AI models last year could process only a few thousand tokens (1 token = ~4 characters), recent large commercial models can handle millions of tokens.
This allows the model to better imitate the person who provided those answers. The model’s capabilities are further improved with the addition of memory, which allows the model to handle multi-step decision making by retaining sequential input (prompts) and output (responses).
At the end of this process, the researchers say their agents were able to provide responses that match human-supplied answers 85 percent of the time when participants were asked two weeks later to retake the survey.
Why bother? Well, there’s some interest in AI models that behave like specific people, whether that involves repeating things the person said or a more generalized simulacrum that provides responses which seem to be consistent with a person’s expected behavior.
Paper co-author Meredith Ringel Morris of DeepMind, in prior work [PDF] published in September with co-author Jed Brubaker from University of Colorado Boulder, wrote: “We anticipate that within our lifetimes it may become common practice for people to create a custom AI agent to interact with loved ones and/or the broader world after death; indeed, the past year has seen a boom in startups purporting to offer such services.”
Coincidentally, Netflix’s Black Mirror was renewed for a seventh season. ®
