AI’s Next Feat: The Descent from the Cloud

It’s been two years since ChatGPT made its public debut, igniting a wave of investment in generative artificial intelligence (AI). This frenzy has driven up valuations for startups like OpenAI, the inventor of the chatbot, and for major technology companies whose cloud computing platforms train and host the AI models that power these applications. However, the current boom is starting to show signs of strain. The next phase of AI growth may be in the palm of users’ hands, as innovations in edge computing come to the forefront.

Current AI Landscape: The Role of the Cloud

Generative AI, which revolves around models that create new content based on their training data, is largely cloud-dependent at present. For instance, OpenAI utilizes Microsoft Azure to train and operate its large language models (LLMs). Users from across the globe can access ChatGPT through Azure’s extensive network of data centers. However, as these models grow in size and complexity, so too does the underlying infrastructure required to train them and respond to user inquiries.

The result? A frenzied race to develop larger and more powerful data centers. OpenAI and Microsoft are currently in discussions regarding a massive data center project scheduled for a 2028 launch, with projected costs hitting an astonishing $100 billion, according to reports from The Information. Overall, tech giants such as Google (owner of Alphabet), Microsoft, and Meta Platforms (the company behind Instagram and Facebook) are expected to collectively spend around $160 billion on capital expenditures next year, a staggering 75% increase compared to 2022. Most of these expenses will go toward securing Nvidia’s highly sought-after $25,000 graphic processor units (GPU) and the necessary infrastructure for model training.

Technological Hurdles: Challenges on the Horizon

The largest hurdle the industry faces is technological. Today’s smartphones and devices lack the computing power, energy, and memory bandwidth necessary to run an expansive model like OpenAI’s GPT-4, which contains approximately 1.8 trillion parameters. Even smaller models like Facebook’s LLAMA, containing 7 billion parameters, would demand an additional 14 GB of temporary storage—an impractical feat for current smartphones. For example, Apple’s iPhone 16 only offers 8 GB of RAM.

Optimism on the Horizon: A Shift Toward Smaller Models

Despite these challenges, there’s room for optimism. Companies and developers are increasingly turning to streamlined models tailored for specific tasks. These smaller models require less data and effort to train, and they are often open-source and freely accessible. Google’s newly introduced “lightweight” model, Gemma, exemplifies this trend with only 2 billion parameters. Their specialized nature frequently allows them to outperform larger, more generalized models while exhibiting fewer errors.

Moreover, many everyday uses of AI, including photo-editing tools and personal assistants, likely won’t necessitate the expense of extensive models. Several smartphones already incorporate live translation and real-time transcription capabilities. Thus, it’s logical for cloud providers to transition basic AI functionalities to edge devices, reserving dense data centers for more complex tasks.

The Rise of Advanced Semiconductors

Additionally, advancements in semiconductor technology are propelling the capabilities of devices. Research firm Yole Group estimates that the proportion of smartphones that can support an LLM with 7 billion parameters is projected to increase to 11% this year, up from 8% last year. Leading chip manufacturers, including Taiwan’s TSMC and South Korea’s Samsung Electronics and SK Hynix, are developing cutting-edge techniques such as advanced chip packaging, which involves stacking multiple chips into a single “chiplet.” This innovation enables them to create more powerful processors by consolidating more transistors without reducing chip circuitry size.

Investment Opportunities: The Edge AI Market

For investors, the burgeoning field of edge AI holds the promise of generating new winners. Up until now, market assumptions have centered on large tech firms, deep-pocketed giants, Nvidia, and a select few startups capturing the majority of AI’s economic boosts. However, the introduction of AI-enhanced tools has the potential to drive consumers toward upgrading to sophisticated smartphones and personal computers. UBS analysts predict that sales in these markets will exceed $700 billion by 2027, reflecting a 14% increase from this year.

Brands ranging from Apple to Lenovo, along with their respective suppliers, stand to benefit from this trend. While Nvidia’s sophisticated GPUs are likely to continue leading the market, other chip manufacturers, such as Qualcomm and MediaTek, are also poised to gain. MediaTek plans to unveil its latest chipset capable of supporting large models next month, predicting a 50% growth in revenue from its flagship mobile products this year.

Conclusion: The Next Big Thing in AI

The forthcoming success of edge AI will hinge on developers creating compelling applications that users find valuable. Should this reality transpire, the next significant evolution in AI may lie within smaller models and devices, reshaping the landscape and impacting how consumers interact with technology.