1 · AI beats humans on some tasks, but not on all
AI has passed human baselines on image classification, visual reasoning and English understanding — and still trails on competition mathematics, visual commonsense reasoning and planning.
2023: money left the room, and generative AI took it anyway
The 2024 AI Index covers calendar year 2023 across nine chapters — research, technical performance, responsible AI, the economy, science and medicine, education, policy, diversity, and public opinion. Overall private AI investment fell for a second straight year, yet generative AI funding nearly octupled, models got dramatically more expensive to train, and the public grew markedly more nervous. A few numbers that set the scene:
149foundation models released in 2023, over double 2022
25.2generative AI private investment (US$ bn), ~8× 2022
61notable AI models from US institutions (EU 21, China 15)
191US$ millions of compute to train Gemini Ultra
123AI incidents reported in 2023, up 32.3%
25AI-related US regulations in 2023 (one in 2016)
The 10 Top Takeaways
The report's own summary of the year, in its own order. Click any card for the full text.
2 · Industry continues to dominate frontier AI research
In 2023 industry produced 51 notable machine learning models to academia's 15, plus 21 from industry–academia collaborations — a new high.
3 · Frontier models get way more expensive
By AI Index estimates, OpenAI's GPT-4 used about $78 million of compute to train; Google's Gemini Ultra cost roughly $191 million.
4 · The US leads China, the EU and the UK on top models
61 notable AI models originated from US-based institutions in 2023, far outpacing the European Union's 21 and China's 15.
5 · Standardized evaluations for LLM responsibility are seriously lacking
OpenAI, Google and Anthropic each test their models against largely different responsible-AI benchmarks, which makes their risks impossible to compare systematically.
6 · Generative AI investment skyrockets
Despite a decline in overall AI private investment, generative AI funding nearly octupled from 2022 to reach $25.2 billion.
7 · The data is in: AI makes workers more productive
Several 2023 studies found AI helps workers finish tasks faster and produce better output, and can narrow the gap between low- and high-skilled workers.
8 · Scientific progress accelerates even further, thanks to AI
2023 brought AlphaDev, which makes algorithmic sorting more efficient, and GNoME, which accelerates materials discovery — a step up from 2022's first applications.
9 · The number of AI regulations in the United States sharply increases
There were 25 AI-related US regulations in 2023, up from just one in 2016 — a 56.3% increase in a single year.
10 · People are more cognizant of AI's impact — and more nervous
Ipsos finds those expecting AI to dramatically affect their lives within three to five years rose from 60% to 66%, while 52% now express nervousness — up 13 points from 2022.
Nine chapters, in brief
Each chapter's headline highlights. Expand to read the key numbers.
1 · Research & Development
Industry produced 51 notable models against academia's 15, with a record 21 from collaborations. 149 foundation models were released, 65.7% of them open-source, up from 44.4% in 2022. Training costs reached an estimated $78M for GPT-4 and $191M for Gemini Ultra. The US led on notable models (61 vs the EU's 21 and China's 15), while China led AI patent origins at 61.1%. GitHub AI projects reached roughly 1.8 million.Full deep dive: Research & Development →
2 · Technical Performance
AI passed human baselines on image classification, visual reasoning and English understanding but not on competition mathematics, visual commonsense reasoning or planning. Saturation on ImageNet, SQuAD and SuperGLUE prompted harder benchmarks — SWE-bench, HEIM, MMMU, MoCa, AgentBench and HaluEval all launched in 2023. Closed models beat open ones by a median 24.2% across ten benchmarks.Full deep dive: Technical Performance →
3 · Responsible AI
The headline finding is a measurement gap: leading developers each test against different responsible-AI benchmarks, so their risks cannot be compared. AI incidents rose to 123 in 2023, up 32.3 percentage points and more than twentyfold since 2013. Political deepfakes proved easy to make and hard to detect, LLMs were shown to output copyrighted material, and the new Foundation Model Transparency Index found disclosure especially thin on training data.Full deep dive: Responsible AI →
4 · Economy
Global corporate AI investment — the broadest measure, including mergers and acquisitions — fell about 20% to $189.2 billion, and private investment fell for a second year to $96.0 billion. Yet generative AI funding nearly octupled to $25.2 billion. US investment reached $67.2 billion, about 8.7× China's. AI job postings in the US fell from 2.0% to 1.6% of all listings. 55% of organizations now use AI, up from 50%. China installed 52.4% of the world's industrial robots. AI was mentioned on 394 Fortune 500 earnings calls.Full deep dive: Economy →
5 · Science & Medicine
AlphaDev improved algorithmic sorting and GNoME accelerated materials discovery; EVEscape improved pandemic prediction and AlphaMissense assisted mutation classification. GPT-4 Medprompt scored 90.2% on MedQA, 22.6 points above the best 2022 result, roughly tripling performance since the benchmark launched in 2019. The FDA approved 139 AI-related medical devices in 2022, up 12.1% and more than 45-fold since 2012.Full deep dive: Science & Medicine →
6 · Education
US and Canadian CS bachelor's graduates kept rising while master's and PhD numbers flattened. The migration of new AI PhDs into industry accelerated: 70.7% went to industry in 2022 against 20.0% to academia, from a near-even split in 2011. Movement the other way thinned — new AI faculty coming from industry fell from 13% in 2019 to 7% in 2022. 201,000 AP CS exams were taken in 2022, more than ten times the 2007 figure.Full deep dive: Education →
7 · Policy & Governance
US AI-related regulations reached 25 in 2023, up from one in 2016 and growing 56.3% in a year, issued by 21 agencies rather than 17. Federal AI bills proposed more than doubled, from 88 to 181. Globally, mentions of AI in legislative proceedings nearly doubled from 1,247 to 2,175, across 49 countries and every continent. The EU agreed the terms of the AI Act and President Biden signed an Executive Order on AI.Full deep dive: Policy & Governance →
8 · Diversity
US and Canadian CS students grew more ethnically diverse at all three degree levels — the Asian share of CS bachelor's graduates rose 19.8 percentage points since 2011 and the Hispanic share 5.2 points. Every surveyed European country still produced more male than female informatics and CS graduates at every level, narrowing only slowly. In US K–12, the female share of AP CS exams rose from 16.8% in 2007 to 30.5% in 2022.Full deep dive: Diversity →
9 · Public Opinion
Ipsos found the share expecting AI to dramatically affect their lives within three to five years rose from 60% to 66%, and 52% now express nervousness, up 13 points. In the US, Pew found 52% more concerned than excited, up from 38%. The public is pessimistic about the economics: only 37% think AI will improve their job, 34% the economy and 32% the job market. 63% are aware of ChatGPT, and about half of those use it weekly.Full deep dive: Public Opinion →
In the report's own words
Lines that capture the year, from the Top 10 Takeaways and the chapter highlights.
AI has surpassed human performance on several benchmarks, including some in image classification, visual reasoning and English understanding. Yet it trails behind on more complex tasks.
Despite a decline in overall AI private investment last year, funding for generative AI surged, nearly octupling from 2022 to reach $25.2 billion.
Leading developers primarily test their models against different responsible AI benchmarks. This practice complicates efforts to systematically compare the risks and limitations of top AI models.
In 2011, roughly equal percentages of new AI PhDs took jobs in industry and academia. By 2022, 70.7% joined industry compared with 20.0% entering academia.
52% express nervousness toward AI products and services, marking a 13 percentage point rise from 2022.
Read the full report
502 pages, nine chapters, hundreds of charts — free from Stanford HAI, with the underlying public data.
Open the AI Index 2024 →