An Update On Data Center FPGAs

By Karl Freund - July 20, 2018

FPGAs from Intel and Xilinx have been steadily carving out niches in datacenter applications where low power, high performance, and configurability may trump programming challenges. Xilinx and AWS have been working with solution providers to create shrink-wrapped applications and tools which use AWS F1 FPGA instances, and Microsoft has recently announced some pretty stellar results in its Project BrainWave AI program using Intel (Altera) FPGAs. Almost a year ago I covered the initial AWS offerings. At the time, I felt that AWS needed to go from 3 solutions to 30 to convince me and the market that there is real demand, and then to 100 to have a material impact on the market. I recently noticed that AWS is now at 20 Amazon Marketplace Instances (AMIs), so it seemed like a good time to check back in.

New AWS F1 Solutions (AMIs) First, let’s review the basic FPGA building blocks in the cloud. Xilinx provides FPGAs to Amazon, which can be configured in Amazon F1 elastic cloud instances to accelerate a wide variety of workloads. However, using this technology traditionally takes significant work by the few engineers in the industry who are skilled in both hardware and software design. To address this barrier to adoption, Amazon and Xilinx in 2017 initially worked with solution providers Edico Genome, NGCodec, and RYFT to deliver genomics, image processing, and complex analytic solutions, respectively, as a shrink-wrapped solution using F1 AMIs. Now the challenge is to find and nurture more solutions in Deep Learning and other fields that can take advantage of this elegant architectural and go-to-market approach to FPGA application acceleration.

Figure 1: Xilinx FPGAs are being offered by Amazon Web Services to accelerate compute intensive applications. XILINX

Looking at the 20 AMIs now available on AWS F1 hardware, roughly half of these AMIs are FPGA development tools, and half are quasi-applications (mostly APIs which can be called from applications). While the list is not exactly earth-shattering, the growth from 4 AMIs to 20 in the last 9 months is a milestone and indicates interest at least from the application supply-side of the equation—essential if we are going to see a knee in the demand curve. There has been good progress in support for Machine Learning inference tools, with five AMIs providing tools ranging from DeePhi’s speech recognition app to optimized tools for Apache Spark (Side note: Xilinx announced this week that it has acquired the Chinese company DeePhi for an undisclosed amount). I believe that Deep Learning Inference represents one of the largest growth opportunities for FPGAs in the future.

Conclusions GPUs like NVIDIA ’s Volta TensorCore V100 have quickly become the gold standard for training deep neural networks (DNNs) in every cloud datacenter, including Microsoft, but the job of using the resulting DNNs is up for grabs for GPUS, FPGAs, CPUs, and custom chips like Google’s TPU. Microsoft has demonstrated its intent to use FPGAs here, and Amazon has continued to ramp up its portfolio of Xilinx-powered FPGA solutions for a variety of apps. What remains unclear is whether and when we might see FPGAs cross the chasm and reach meaningful volume in the data center, with more cloud providers jumping on board for internal apps and making them externally available for accelerated solutions. In my opinion, this is only a matter of time—it can take years to produce an industrial-grade solution that is built on FPGAs. Given Microsoft’s impressive results I believe FPGAs' day is on the horizon.