AI teams significantly outperform human teams 

at creative problem-solving 

Researcher spotlight: Luning Sun, The Psychometrics Centre, Cambridge Judge Business School 

C2D3 Early Career Researcher Seed fund awardee, 2024  

Creativity has traditionally been considered a uniquely human skill - but can AI be equally creative? The answer to this question carries significant implications for how AI will shape the world of work, particularly in industries such as content creation.

Several attempts have been made to compare the creativity of individual large language models (LLMs) and humans, including recent work by Cambridge researcher Luning Sun. The results are mixed but suggest that individual LLMs can perform as well as, or better than, humans at certain types of creative tasks. However, human teams are consistently better at creative problem-solving than individuals. Meanwhile teams of AI agents have been shown to outperform individual agents on other types of task - prompting Luning to ask: how do AI teams compare to human teams when it comes to creativity?

To answer this question, Luning and his collaborators collected nearly 5000 ideas generated by teams of humans or teams of LLM agents in response to six diverse problem-solving tasks. The ideas were anonymised and scored by human assessors for novelty and usefulness following the standard definition of creativity as the capacity to generate ideas that are both novel and useful. Overall, the AI teams performed substantially better than human teams. This advantage was driven by higher novelty scores, while maintaining comparable scores for usefulness. Notably, this advantage extends to the most creative ideas which tend to matter most in real-life scenarios.  

What affects the creativity of AI teams? Unlike human teams, AI teams need guidance on how to collaborate, so each team was instructed to use one of five different methods to structure their discussion. Both the discussion method and the model of LLM used were found to affect the creativity of the ideas generated, suggesting a way forward for optimising AI teams for creative problem-solving. Luning and his team also investigated differences in how conversations are shaped in human and AI teams - find out more in their recently released pre-print.

Funding from the C2D3 Early Career Researcher Seed Fund, along with a grant from Cambridge Judge Business School, enabled Luning to recruit and pay the human participants needed for this project. This is an important step in building his track record of independent research, which he plans to use as a springboard for future grant applications.

What’s next?

This project has been selected for Microsoft’s Accelerating Foundation Models Research Initiative. Looking ahead, the team plan to explore the impact of agent persona on AI team creativity.