MemryX Is A New AI Company We Actually Need

MemryX

Artificial Intelligence (AI) is all around us, in the cars we drive to the smartphones we use, powering our most favored or despised apps. Consequently, one of the major growth drivers in the semiconductor industry is AI. It’s everywhere and revolutionizing the way humans interact with the modern world, from the datacenter to the device edge and everything in-between. Running AI on the edge has become commonplace as these interactions need to happen quickly with low latency while minimizing the data transport expense.

MemryX, a new startup with a focus on AI processing for edge devices, is creating new technologies to address this emerging market. But do we really need another AI chip company? I think we’re still in the second or third inning of this game and MemryX has the potential to disrupt the device edge semiconductor market with a combination of its unique, memory-centric AI architecture and its easy-to-use and scalable software/hardware solution. The company’s leadership is both experienced and practical, knowing what it takes to ramp high volume semiconductors. If you’re not first to market, you must be very different and that’s where I see MemryX fitting in.

Let’s take a closer look at MemryX, its direction, and how it might be different from the pack.

Company basics

MemryX was co-founded in 2019 by Dr Wei Lu, MemryX CTO, to develop a leading AI accelerator for Edge devices. Dr. Lu has been an Electrical Engineering and Computer Science professor at the University of Michigan since 2005. His expertise in memory systems and neuromorphic computing is internationally recognized. Dr Lu and his team spent nearly three years developing and proving MemryX’s unique approach.

In December 2021, MemryX hired Keith Kressin as President and CEO to commercialize MemryX’s technology. Keith has over 25 years of experience in semiconductor leadership and was previously a Senior Vice President and General Manager at Qualcomm. He helped guide technology and products at Qualcomm during the key growth phases of 3G/4G/5G smartphones, AR/VR, and AI initiatives. He also spent nearly a decade at Intel. I know Keith well and I think he’s the right kind of guy to scale a company. Keith’s reputation enabled him to hire a strong leadership team in Engineering and Sales/Marketing bringing decades of executive experience from AMD, Intel, Micron, and Qualcomm. Experience matters.

The company has been frugal, having raised a very modest $11M in Seed and Series A, while successfully taping out multiple generations of chips to prove the technology works. It just recently announced customer sampling of their pre-production MX3 AI Accelerator hardware and software. I’m sure series B round isn’t far behind given the required financial levels it takes to be in the increasingly strategic semiconductor market.

MemryX is backed by several well-known investors including HarbourVest, eLabs, M Ventures, and Arm MEMRYX

The Target and Flag Plant

Unlike most AI startups that pretend to solve all things for all people, MemryX is focused on delivering an Edge AI inference (not training) solution for vision and sensing solutions. Its solution is an AI accelerator that complements, rather than replaces, a central processor for either new or legacy systems. So, it is trying to do one thing really well. Overall, MemryX is planting the flag on delivering a high efficiency, low power solution, which benefits the end user. But oftentimes even more importantly, they are delivering an Edge AI solution that is the easiest to implement, benefiting the designer.

The challenge

The problem that MemryX is trying to address is a multi-variate challenge.

The growing diversity of smart devices makes it difficult to employ traditional computing architecture to run AI models at the Edge. Furthermore, AI systems are data centric, while classic processors are designed to execute instructions. In traditional computing, AI models are stored in DRAM resulting in bottlenecks, higher latencies, and higher power. So many systems can run some AI, but not very efficiently.

AI accelerators for edge devices are expensive to develop because of the difficulty and inconsistency of software implementation. It typically takes a tremendous engineering effort to create and update AI algorithms across a widening array of hardware solutions including ARM, x86, and now RISC-V ISAs. And just as investments get made, ongoing changes to algorithms and operating systems continuously increases the need for further investment. 

As AI makes its way into nearly every chip and device, it becomes apparent why dedicated AI accelerators have limited market traction. A new approach to edge AI which addresses these issues could add tremendous value.

A solution with ‘One Click Optimization’

By moving toward a unique dataflow architecture with at-memory computing, much like how AI networks are naturally organized, MemryX should overcome the bottlenecks and complexity of software implementation and meet the needs of Edge AI at broader scale.

MemryX’s Neural Processing Unit (NPU) has lower latency compared to other AI accelerators because of its inherent ability to store AI models on-chip rather than in DRAM. The proprietary dataflow architecture is also designed to minimize any movement of data within the chip, since moving data requires more energy than AI computations. This enables their solution to maximize throughput while minimizing power usage.

The architecture enables “one click performance optimization” delivering 50-70% utilization of the chip without the need to hand-tune any software. I’ll admit, I was a bit skeptical when MemryX briefed me on this. It was almost too good to be true given hand-tuning AI models for weeks or months is common practice. I became a lot more convinced when the company shared with me a growing list of well over 100 models across 13 categories that run efficiently on its solution using all the same software. It also supports models across every popular AI framework. Meanwhile, MemryX has just 30 employees. Impressed yet?

The company’s latest chip, the MX3 is also scalable, so one can connect anywhere from 1 to 16 small chips, scaling the production version of the chip from 5 TFLOPs to 80 TFLOPs. Even more impressive is performance per watt given each chip averages only 1 Watt of power. This means that MemryX with no software tuning, says it is over 5X more efficient than an NVIDIA AGX Xavier system that uses NVIDIA’s own proprietary software. I did not personally run the tests, but the test methodology passes my smell test.

MemryX AI Accelerator(s) can be directly plugged into an existing USB or PCIe port. MEMRYX

Moving with momentum

MemryX is only a little over three years old and has made great progress by my yardstick, completing its seed financing round in 2019 and just seven months later proving its proprietary architecture worked in silicon. This past year, MemryX produced its pre-production chip, which it announced this week was sampling to customers. The company expects volume production to start in 2023.

Since MemryX is producing such a focused product with the potential to scale in ways that have not been seen much of in the low power, AI edge market, I believe that if all goes according to plan, it could have a tremendous impact on the intelligent edge. MemryX says it’s working with alpha customers in several target markets including automotive, edge servers, metaverse, and security cameras. I think this gives MemryX a diverse portfolio of markets and multiple opportunities to experience significant growth.

Wrapping up

While some might say the AI accelerator space is crowded, it’s apparent to me MemryX is different. It’s uniquely solving well known problems in edge AI. Its focus is on unique technology to meet real market needs is half the battle. MemryX looks to be off to great start.

I look forward to further disclosures and will share as appropriate.