Cristian – Gavril Olar | Director of Systems Software at AXELERA AI 

After 20 years of calling me to fix their email and antivirus settings, my parents started calling my 13-year-old daughter for help instead. Technology seems to have been nagging rather than helping grandparents, but the advent of cheap computing gives fundamental reasons to change this balance.

Figure 1 – A few population pyramid examples. There are places with significant population shrinking and places with population growth, but in the last few years, the global trend is toward population shrinkage. Source: https://www.populationpyramid.net

Economic Growth in a Shifting Population Landscape

In the past century, we’ve seen a 10% market growth r1], reshaping the foundation of our civilization with the assumption of continuous economic growth. Whether it’s investment funds, pension plans, or governments calculating bond rates, all have factored in the premise of growth.

Until recently, an important driver of this growth was the increasing global population. A larger population equates to more brain and muscle power, leading to heightened productivity capacity. However, in recent years, the world has witnessed a decline in population growth, albeit unevenly across different regions.

Some see this decline as ominous, as famously raised by Elon Musk two years ago when claiming civilization could crumble f2]. On the more analytical side of things, this shift has drawn attention due to its potential impact on various aspects, notably retirement plans and workload stress. The population decrease is driving the trend toward fewer workers supporting a higher number of retirees along with a greater number of dependents and fewer skilled workers.

It has led some countries to make legislation changes to move the retirement age upwards m3]. Increasing productivity, therefore, is becoming a crucial goal in sustaining economic growth, albeit with potential consequences such as increased workload stress e4].

Working Smarter with Cheap Computing

To counteract these challenges, the focus must shift towards increasing productivity without human overload. In other words, we need to get more human equivalent. Historically, automation has played a key role in boosting productivity. However, the recent decrease in computational costs suggests a potential profound transformation on the horizon. The promise of almost free computer power has significant implications for productivity growth.

The ESP32-CAM from Espressif, a low-cost development kit capable of computer vision applications, perfectly exemplifies the advent of cheap computing. In fact, in an article where it was presented 5 years ago it prompted the author to say computing rwith ESP32] is not just cheap but it’s essentially free "5]. This seemingly trivial device’s potential is immense, setting a precedent for affordable compute power that creates value from virtually nothing.

When I learned about Espressif, I was working at Intel in the technology department building the MyriadX chip. I tried out ESP32 in my spare time because it had some similarities to Myriad: it could use a camera, do some level of image processing, and claimed to do computer vision applications. A couple of weekends later, I was able to build a few interesting applications and was very excited. But then I looked at machine learning applications on ESP32, and the performance there was really limiting my ideas. Even so, the article comment was something that stuck in my mind: a platform that makes compute essentially free.

This revelation came back into focus when during an internal brainstorming session, Fabrizio del Maffeo, our CEO at Axelera, commented that we were essentially getting Tera Operations Per Second (TOPS) for free with In-Memory Computing. At the time the comment was made, we were still waiting for Metis, Axelera’s first commercially available chip to come from the factory into the lab. The comment still felt theoretical.

Figure 2 – Areas which were traditionally empty are starting to get filled up with architectures from different applications. Cornami adds encryption power, Axelera adds Artificial Intelligence power.

Unlocking the Power of Artificial Intelligence at low costs

Things changed just a few days later when Metis came into the lab. Incidentally, Metis’ early power-up coincided with the week before and after Easter. In Romania, where I am based, in a word-by-word translation these weeks would be “The Great Week” and “The Enlightened Week” which seemed fitting: Metis came alive in the Great Week and started running Neural Networks in the Enlightened Week. To introduce Metis to the world we decided to use the MIT Lincoln Laboratory e6] report which charted current compute trends with a live demonstration.

Metis AIPU was not there yet at the time of publishing the 2022 trends but we did a live plot running internal max TOPS programs for the In-Memory Computing engines.

This chart doesn’t show the compute cost but all this comes at around $200. Now, the human brain memory power is likened to around 100 trillion calculations salso called „operations”] per second e7]. The program used above was an artificial one loading the In-Memory Computing part of Metis with an artifical neural net-like that had some load/store overheads to an ideal pure-calculation program measuring the AIPU In-Memory Computing power used for this. It reaches a practical 200 TOPS result, so the equivalent of two human brains. There is a lot more work to reach actual use of human brains but what really shook me was that price for such brain power. Such calculation capacity has existed for quite sometime now. Still, the computing platforms needed for it have had wild price evolutions so far and have generally stayed in the thousands of dollars. But $100 for a human brain capacity is eye opening and has significant transformation potential. Currently, the smallest country nominal GDP per capita in the world is $249. With the capacity of two human brains lowered at less than half of that, we are witnessing the first point in history where the compute equivalent of human productivity is clearly available anywhere in the world, creating the ability to democratize AI to even the poorest of countries.

To look at the practical implications of this capacity we can look at analysis recently made showing the potential to double the productivity in China which is one of the countries already facing some of the most serious population declines and hedging their hopes on automation o8]. Doubling productivity could more than compensate for an eventual productivity decline caused by population shrinkage.