What does it take to make an AI initiative successful?
- software? yes.
- partner? yes.
- What about education and technological literacy? Yes.
As these resources become widely available and the number of ready-to-use AI products proliferates, it is easy to forget another fundamental element of AI applications: hardware.
Most of the hardware currently available for running AI workloads is designed with other capabilities in mind. So while existing options are powerful and capable, we need new solutions developed natively for AI use cases. Recently, IBM announced a new prototype chip architecture that makes AI development and processing faster, cheaper, and more energy efficient.
Chip design revolution
Dr. Dharmendra S. Moda, IBM Fellow and Principal Scientist for IBM’s Brain Inspired Computing, and his team at IBM Research’s lab in Almaden, Calif., are working to help organizations efficiently deploy AI hardware. We’ve been working for years on chip architectures that have the potential to revolutionize the way we scale. system.
NorthPole is a digital AI chip for neural inference based on the human brain’s computational pathways. It takes a dramatically different approach than standard chip design, where data is shuffled between memory, processing, and other elements.
The work Modha performs on the TrueNorth chip is significantly more energy and space efficient, has the lowest latency of any existing chip, and is approximately 4,000 times faster than previous generation chips.
All of NorthPole’s memory is stored on-chip, rather than individually, making it easy to integrate and perform ultra-fast AI inference. “This is an entire network on a chip,” Moda says. “Architecturally, NorthPole blurs the line between compute and memory.
“At the level of the individual core, NorthPole appears as memory close to compute, and outside the chip, at the input/output level, it appears as active memory.”
This unique architecture limits NorthPole to retrieving data from internal memory. External sources are not available. Multiple NorthPole chips can be connected to support large neural networks through partitioning. However, IBM researchers believe this limitation is an advantage for consumers.
The chip is explicitly designed for AI inference, which means it doesn’t require a complex cooling system and is more adaptable. When it comes to AI use cases, this chip has a wide range of roles.
It has been extensively tested for computer vision applications such as image segmentation and video classification. However, it also has a proven track record in other fields such as NLP and speech recognition.
When it comes to real-world use cases, NorthPole’s portability makes it an ideal candidate for AI tasks at the edge. IBM cites examples such as self-driving cars, satellite monitoring applications, wildlife management, road safety, robotics, and cybersecurity. However, this may be the tip of the iceberg.
Is hardware the new AI battleground?
There’s no getting around the fact that it took IBM, the world’s leading computer hardware maker, decades to prototype the NorthPole chip. But this news could spark a race to develop alternative AI-only chips that cover the core requirements of efficiency, scalability, and speed.
IBM is defining the rules of the game by focusing on NorthPole’s AI inference capabilities, a particular feature that means the chip was not designed with flexibility in mind. The most important software and data companies, whether in the private sector or large enterprise organizations, are offering AI solutions to their passionate customers, including chatbots designed for specific platforms and trained on contextual data. As we have seen providing use cases, the chip industry could follow suit.
As AI rapidly penetrates the technological fabric of organizations, it seems logical that future developments will rely on specialization as a key differentiator. Just as there is no one-size-fits-all CRM, in the future companies will be able to rationally consider hardware nuances when developing new AI use cases.