GPT-2 — the 50,000-dollar baseline
Released in February 2019 with 1.5 billion parameters, at an estimated training cost of about 0.05 million dollars.
In 2022, the frontier of AI research finished moving inside companies
Chapter 1 of the AI Index 2023 follows where AI knowledge is made: publications, significant machine learning systems, large language and multimodal models, conferences and open-source code. Output keeps rising — the world published almost 500,000 AI papers in 2021, more than twice the 2010 figure, and universities still write three-quarters of them. But of the systems that actually moved the frontier in 2022, industry released 32 and academia three. The numbers:
496.01thousand AI publications worldwide in 2021 (about 200 thousand in 2010)
32significant ML systems released by industry in 2022 (academia released 3)
540billion parameters in PaLM, about 360 times GPT-2’s 1.5 billion
8.01million USD estimated to train PaLM in 2022 (GPT-2 in 2019: about 0.05)
54.2% of large language and multimodal model authors based in the US in 2022
347,934GitHub AI projects in 2022, up from 1,536 in 2011
1.1 — Twice as many AI papers as in 2010, and a different mix underneath
Publication data in this edition stops at 2021, because a year’s papers are only fully captured around the middle of the next one. On that basis: the world produced 496,010 AI publications in 2021, up from roughly 200,000 in 2010. Over 12 years journal publications grew about threefold and preprint repositories 26.6-fold — while conference papers have been falling since 2019.
In 2021, 60% of published AI documents were journal articles, 17% conference papers and 13% repository submissions; books, book chapters, theses and unknown types made up the remaining 10%. The shift toward journals and preprints is not cosmetic. Conference publications peaked in 2019, and the 85,094 papers of 2021 sat 20.4% below that peak — only marginally above the 75,592 recorded back in 2010.
What the papers are about, and who writes them
- Pattern recognition and machine learning grew fastest over the past half-decade: since 2015 pattern recognition papers roughly doubled and machine learning papers roughly quadrupled. The next most published fields in 2021 were computer vision (30,075 papers), algorithms (21,527) and data mining (19,181).
- Universities still do most of the writing. In 2021 the education sector accounted for 75.2% of AI publications, nonprofits 13.6%, industry 7.2% and government 3.7%. Industry participation is highest in the United States, then the European Union.
- But education’s share has been dropping in every region since 2010 — the research base is broadening outward rather than concentrating in universities.
- Cross-sector collaboration is where the growth sits: education and nonprofit institutions produced 32,551 joint publications in 2021, industry and education 12,856, and education and government 8,913.
- Industry–education collaborations have been among the fastest-growing categories of all, increasing 4.2 times since 2010.
The US–China axis is still the biggest — and it just stopped growing
Between 2010 and 2021, no country pair collaborated on AI research more than the United States and China. Joint publications rose roughly fourfold over that period, reaching 10,470 in 2021, and were 2.5 times the total of the next nearest pair, the United Kingdom and China. But the curve has flattened: US–China collaborations grew only 2.1% from 2020 to 2021, the smallest year-over-year increase since 2010.
China leads on volume, the United States still leads on attention
China produced the largest share of the world’s AI journal publications (39.8%) and conference publications (26.2%) in 2021. The United States leads on repository publications (23.5%), conference citations (23.9%) and repository citations (29.2%) — but the report notes those leads are slowly eroding.
Journals, conferences, repositories: three different maps
- AI journal publications in 2021: China 39.8%, the European Union plus the United Kingdom 15.1%, the United States 10.0%, India 5.6% — India’s share has climbed steadily from 1.3% in 2010.
- AI journal citations in 2021: China 29.1%, the EU plus the UK 21.5%, the United States 15.1%. Those three together account for 65.7% of all AI journal citations in the world.
- AI conference publications in 2021: China 26.2% — it overtook the EU and the UK in 2017 — followed by the EU plus the UK at 20.3% and the United States at 17.2%. On conference citations the United States still leads with 23.9% against China’s 22.0%, and the gap is narrowing.
- AI repository publications in 2021: the United States 23.5%, the EU plus the UK 20.5%, China 11.9%. The United States also takes 29.2% of repository citations, ahead of the EU plus the UK (21.5%) and China (21.0%).
- By region, East Asia and the Pacific produced 47.1% of AI journal publications and 36.7% of conference publications in 2021. South Asia’s conference share more than doubled over 12 years, from 3.6% in 2010 to 8.5% in 2021.
The ten institutions that publish the most
Since 2010 the single largest producer of AI papers has been the Chinese Academy of Sciences, and the next four are all Chinese universities. In 2021 the Chinese Academy of Sciences published 5,099 AI papers, Tsinghua University 3,373, the University of Chinese Academy of Sciences 2,904, Shanghai Jiao Tong University 2,703 and Zhejiang University 2,590. MIT, with 1,745, was the only non-Chinese institution in the world’s top ten. The report attaches its own caveat: many Chinese research institutions are large, centralized organizations with thousands of researchers, so a raw publication count flatters them.
1.2 — Thirty-two significant systems from industry, three from academia
Epoch AI counted 38 significant machine learning systems released in 2022. Until 2014, most such systems came from academia. In 2022 industry released 32, academia three, research collectives two, and one came out of an industry–academia collaboration. Nonprofits released none.
The report’s explanation is blunt: producing state-of-the-art AI systems increasingly requires large amounts of data, computing power and money — resources that industry actors possess in greater amounts than nonprofits and academia. The compute used by significant systems has grown exponentially over the past half-decade, and since 2010 language models have consistently demanded the most of it. More compute-intensive models also carry greater environmental impact, and industrial players have easier access to those resources than universities do.
What got built, and by whom
- By domain, language was by far the most common class of system released in 2022, with 23 — roughly six times the next most common type, multimodal systems (4). Drawing accounted for three, and vision and speech for two each.
- By country, the United States produced the most significant systems in 2022 with 16, followed by the United Kingdom (8) and China (3); Canada and Germany contributed two each.
- Counting people rather than systems widens the gap. In 2022, 285 authors of significant systems were affiliated with US institutions, against 139 in the United Kingdom and 49 in China — more than double the UK and nearly six times China.
- The pattern is not new: since 2002 the United States has out-produced both the European Union plus the United Kingdom and China in cumulative significant machine learning systems.
What it cost to train the flagship models of 2022
New in this edition, the AI Index estimated training costs for large language and multimodal models from the hardware and training time their authors disclosed, tagging each figure as a mid, high or low estimate. The results confirm what the field had only speculated: these models now cost millions of dollars to train, and cost tracks size and compute closely.
PaLM (540B) — 8.01 million dollars
Google’s 2022 flagship: 540 billion parameters, nearly 360 times GPT-2, at an estimated 8.01 million dollars to train.
Megatron-Turing NLG 530B — 11.35 million
The most expensive training run in the AI Index’s estimates, at about 11.35 million dollars.
Chinchilla — 2.11 million dollars
DeepMind’s May 2022 model, estimated at 2.11 million dollars — a sign that the million-dollar training run is now ordinary, not extreme.
BLOOM — 2.29 million, and no single country
Estimated at 2.29 million dollars and built by more than 1,000 international researchers, BLOOM was listed as indeterminate in national affiliation.
GLM-130B — China’s only entry in 2022
The only Chinese large language or multimodal model released in 2022: a bilingual model from Tsinghua University, estimated at 0.16 million dollars to train.
Five questions the chapter answers about the wider ecosystem
Conferences, open-source code, and the arguments sitting underneath the headline numbers.
Are AI conferences shrinking?
Attendance fell in 2021 and again in 2022, to 59,450 across the conferences the AI Index tracks — but the report puts the dip down to format rather than interest: many conferences returned to hybrid or in-person after being fully virtual in 2020 and 2021, and IJCAI and KR went strictly in-person. NeurIPS remained one of the largest at around 15,530 attendees, followed by CVPR at about 10,170 and ICML at about 7,730. The biggest single-year jump was ICRA, from 1,000 attendees in 2021 to 8,008 in 2022. The report also warns that attendance at virtual conferences is genuinely hard to measure, so the numbers deserve caution.
How big is open-source AI now?
GitHub AI projects grew from 1,536 in 2011 to 347,934 in 2022. The contributor map looks nothing like the publication map: as of 2022, developers in India accounted for 24.2% of GitHub AI projects, the European Union plus the United Kingdom 17.3%, the United States 14.0% and China 2.4%. The American share has been declining steadily since 2016. On stars — GitHub’s equivalent of a like — US projects still hold the largest cumulative total at 3.44 million, ahead of the EU plus the UK (2.34 million), China (1.53 million) and India (0.46 million), though new stars have levelled off in many areas.
Why are conference papers falling?
Conference publications peaked in 2019, and 2021’s 85,094 papers were 20.4% below that peak — barely above the 75,592 of 2010. Over the same 12 years journal publications grew about threefold and repository submissions 26.6-fold. The chapter does not name a single cause, but it does describe the mechanism: researchers increasingly post pre-peer-reviewed papers to repositories such as arXiv and SSRN to share findings before submitting them to journals and conferences, which accelerates the cycle of information discovery.
Has academia actually been pushed out?
Out of model building, not out of research. Academia released three significant machine learning systems in 2022 against industry’s 32, and the report is explicit that state-of-the-art systems now require data, computing power and money on a scale industry holds and universities do not. But the education sector still produced 75.2% of all AI publications in 2021, and nine of the world’s ten most prolific AI publishing institutions are universities or academies. The bridge between the two worlds is growing fastest of all: industry–education collaborations reached 12,856 joint publications in 2021, 4.2 times the 2010 figure.
Is US–China research decoupling?
The data shows a plateau, not a break. The United States and China remain by far the largest collaborating pair in AI publications, with 10,470 joint papers in 2021 — roughly four times the 2010 figure and 2.5 times the next pair, the United Kingdom and China. But growth from 2020 to 2021 was 2.1%, the smallest year-over-year increase since 2010. The chapter’s own framing is that although the United States and China continue to dominate AI R&D, research efforts are becoming increasingly geographically dispersed: India’s share of AI journal publications rose from 1.3% in 2010 to 5.6% in 2021, and South Asia’s share of conference papers from 3.6% to 8.5%.
The chapter in its own words
Five lines from Chapter 1 that carry the year.
In 2022, there were 32 significant industry-produced machine learning models compared to just three produced by academia.
Building state-of-the-art AI systems increasingly requires large amounts of data, computer power, and money — resources that industry actors inherently possess in greater amounts compared to nonprofits and academia.
PaLM was around 360 times larger than GPT-2 and cost 160 times more. It’s not just PaLM: Across the board, large language and multimodal models are becoming larger and pricier.
The total number of U.S.-China collaborations only increased by 2.1% from 2020 to 2021, the smallest year-over-year growth rate since 2010.
Although the United States and China continue to dominate AI R&D, research efforts are becoming increasingly geographically dispersed.
Read Chapter 1 in full
Chapter 1 (sections 1.1–1.4) — publications, significant machine learning systems, large language and multimodal models, conferences and open-source software — with every figure and citation is free from Stanford HAI.
Open the AI Index 2023 →