You Are What You Eat

The concept of AI being trained on datasets that were also created by AI, and then polluting their output, has been referred to as 'model collapse.' [1] New data will be required for these models to continue to develop and improve.

Model Collapse

• Efficiency Gains Are Decreasing – GPT multitask performance has increased only marginally from 87.2% vs 86.4%, two previous versions ago. [2]
• Running Out of Training Data – Over the past 12 months, 25 percent of data from the highest quality sources has become restricted. If such constraints weren’t enough, the supply of public data to train AI models is expected to become exhausted soon. [3]
• Outputs Are Reverting to the Mean  – All dogs become golden retrievers, and models gravitate toward the most common output. [5]
• Training Costs Continue to Increase – While AI companies are paying millions to acquire training data, [3] compute to train frontier models continues to become more and more expensive. [4]

AI Hype Cycle – What is Next?

• Peak of Expectations  – Many of the current AI research and applications are in the 'Peak of Inflated Expectations' stage as per Gartner, with the 'Trough of Disillusionment' approaching.
• Long-term Changes Take Time  – Plateaus of productivity for most AI technologies are still 2-10 years away, with General AI expected to be greater than 10 years.
• On the Rise  – Multiagent systems and Decision Intelligence technologies are in the innovation stages and will likely lead to real efficiency gains in specialist verticals like finance and law.
• Obstacles – Copyright infringement issues and IP ownership issues will continue to generate headwinds, as well as pending government legislation. [6]

Where Do We Go From Here

In 2023, several studies assessed AI’s impact on labor, suggesting that AI enables workers to complete tasks more quickly and improve the quality of their output. These studies also demonstrated AI’s potential to bridge the skill gap between low- and high-skilled workers.

Still, other studies caution that using AI without proper oversight can lead to diminished performance.[4] As businesses begin to deploy and test in live environments using proprietary and original data, understanding current AI technology limitations as well as opportunities requires a pragmatic approach to assess both the risks and rewards of utilizing these new tools.