Study explores which pairings of people and AI agents perform best - hub.jhu.edu

In a first of its kind study, researchers from Johns Hopkins University and the Massachusetts Institute of Technology found that strategically pairing artificial intelligence personalities with human traits has the potential to transform workplace collaboration and performance in human-AI teams.

"In the literature, we found several research studies that showed different types of people or personality types actually get different results using AI," says Harang Ju, an assistant professor at the Johns Hopkins Carey Business School. "The question that we asked was: Can we vary the type of AI that people are using by changing and randomizing the personalities of the AI?"

For the study, published in the August edition of Proceedings of the National Academy of Sciences, Ju and Sinan Aral of MIT conducted a controlled, randomized trial to determine the best personality pairings for people and AI agents. They randomly paired 1,258 participants with AI agents programmed to exhibit varying levels of the so-called "Big Five" personality traits, a widely accepted psychology framework for categorizing human personalities. The human participants were categorized using the same framework, which includes the traits of openness, conscientiousness, extraversion, agreeableness, and neuroticism. The human-AI teams were then asked to produce more than 7,000 display ads, which the researchers evaluated for quality and performance using 1,995 independent human raters and a field experiment on X that generated nearly five million impressions.

The results showed that ad quality significantly influenced ad performance, measured by click-through rates and cost-per-click. Human and AI personalities individually shaped ad quality and teamwork, and when examined together, human-AI personality pairings directly affected ad quality outcomes. The experiment demonstrated that personality pairing affected ad quality, teamwork, and real-world ad performance. For example, extraverted humans paired with conscientious AI produced the lowest-quality ads.

The researchers believe their large-scale experimental evaluation is an important first step toward robust AI personalization, with the results potentially pave the way for better hiring and team-building strategies.

"You can't manipulate the personalities of humans, but we know we can change the personalities of AI agents and get results," Ju says. "This means that you can actually hire for skills and traits that actually matter for the role and less on whether you can use an AI better or not."