At the beginning of November 2023, IBM announced the creation of a new AI processor called North Pole, claiming it to be the most powerful AI processor to date.
IBM North Pole
IBM North Pole marks a significant advancement in AI technology. It is 22 times more efficient than the best currently available processors, even though the prototype was manufactured using a 12nm lithographic process, which is considered outdated. However, thanks to its new architecture, it outperforms 4nm GPU processors.
This information was very popular in my information bubble, but I initially viewed it as something specialized for data centers. However, few people realize that TPU and NPU processors are also present in our daily lives. Even though we don’t see them, the device you’re using to read this text—your smartphone—has such a unit integrated into its System on a Chip (SoC).
AI Processors (NPU/TPU)
Illustrative SoC Diagram
NPU/TPU processors in smartphones are integrated circuits specifically designed for AI-related tasks. TPU stands for Tensor Processing Unit, while NPU stands for Neural Processing Unit. They can be considered as another piece of the puzzle alongside CPU (general-purpose), GPU (specialized), and TPU (highly specialized). These processors are used in various applications such as speech recognition, language translation, and image recognition, allowing AI-related computations to be performed more efficiently and with lower power consumption than CPUs or GPUs.
In smartphones, TPU/NPU processors handle AI tasks locally, avoiding cloud processing, which can be impractical due to high latency. Additionally, shifting tasks from the cloud to the device is more cost-effective for manufacturers.
Some of the tasks handled by TPUs and NPUs in smartphones include:
- Speech recognition – Assisting in recognizing user speech, enabling features like voice assistants and voice commands.
- Language translation – Used in applications like Google Translate for real-time language conversion.
- Image and video recognition – Identifying objects in photos and videos, enabling smart cameras and automatic photo tagging.
- Face ID/Face Unlock – Enhancing the accuracy and performance of facial recognition systems, such as those in iPhones or Android devices.
- Process automation – Used for tasks like facial recognition for device login or object recognition for online shopping.
Google Tensor G
Tensor G3
Google is so committed to on-device AI processing that the name of its Tensor processor serves as a billboard advertisement for this technology. “Tensor” refers to tensors, which are mathematical objects used in machine learning.
Company claims that its Tensor G3 SoC is the “most advanced AI processor in a smartphone.” Besides traditional ARM CPU and GPU cores, it features a TPU codenamed “Rio”. This processor is likely the reason why the new Pixel 8 allows secure Face Unlock authentication for banking apps without compromising security.
Google has prioritized AI processing in its SoC at the expense of traditional computing performance (CPU, GPU), which has sparked controversy. However, Google explicitly states on its website:
“Our work with Tensor has never been about speeds and feeds, or traditional performance metrics. It’s about pushing the mobile computing experience forward. And in our new Tensor G3 chip, every major subsystem has been upgraded, paving the way for on-device generative AI.”
Not Just Google
Apple Bionic
Of course, Google is not the only company using TPUs in smartphones. Apple has been integrating Neural Engine TPUs in iPhones since the iPhone X.
Other smartphone manufacturers using TPUs include:
- Huawei – Kirin NPU
- Qualcomm – Hexagon DSP
- MediaTek – NeuroPilot
- Samsung – Exynos NPU
Whether IBM will succeed in bringing its TPU vision to the business market remains to be seen. You might want to think about this the next time you ask Android Auto or CarPlay to take you home.