Last week, I had the pleasure of attending Amazon.com AWS’s re:Invent conference in Las Vegas. Re:Invent is AWS’s once a year mega-event where it announces new services and holds 2,500 educational sessions for builders, CIOs, channel and ecosystem partners, customers, and of course, industry analysts like me. It’s a large event at 65,000 attendees but could be much larger as it sells out after a few days. The attraction is simple. It’s the most important cloud show you can attend and attendees want to get a head-start and hands-on with the latest and greatest of what AWS has to offer. AWS made hundreds of announcements and disclosures and while the Moor Insights & Strategy analyst team will be going deeper on the most impactful announcements, I wanted to make a top 5 list and why you should care.
1/ Graviton2 for EC2 M, R, and C 6th Gen instances
Based on an Arm N1 core, AWS says these new instances deliver up to 40% improved price/performance over comparable x86-based Skylake instances. In preview, AWS will make these available for Mainstream (M), memory-intensive (R) and compute intensive (C) instances.
Why this matters
You may expect that I gave the #1 spot to new chips because I can be a chip nerd. I can be, but when you think about a 40% improvement over IaaS, PaaS and SaaS services that can’t easily be copied, I’d say that’s important. That’s not saying that advantage will last forever, but it’s very disruptive right now. First off, I’d say that now no one can say Arm isn’t ready for general purpose datacenter compute. It is, as AWS IaaS is larger than the #2-10 IaaS provider combined. I can see VMware and Oracle accelerating its offerings and maybe SAP doing anything with Arm, which they aren’t publicly. Finally, don’t overthink this related to AMD and Intel. The market is massive, growing and I don’t believe this is anti-Intel or AMD. But if a small AWS team can outperform AMD and Intel on some cloud workloads, you do have to do a pause. I wrote in-depth on all of this here.
2/ Many new hybrid offerings
While AWS doesn’t want to use the term “hybrid” a lot, I think enterprises understand that it means they can extend their AWS experience to on-prem or close to on-prem compute and storage. AWS announced three capabilities here that are important, including going GA on Outposts and announcing Local Zones and Wavelength.
AWS describes it as, “AWS Outposts are fully-managed and configurable racks of AWS-designed hardware that bring native AWS capabilities to on-premises locations using the familiar AWS or VMware control plane and tools. AWS Local Zones place select AWS services close to large population, industry, and IT centers in order to deliver applications with single-digit millisecond latencies, without requiring customers to build and operate datacenters or co-location facilities. AWS Wavelength enables developers to deploy AWS compute and storage at the edge of the 5G network, in order to support emerging applications like machine learning at the edge, industrial IoT, and virtual and augmented reality on mobile and edge devices.”
Why this matters
AWS took the hybrid idea and doubled down on it. If you’re a customer who wants a low latency experience on-prem with Outposts, lowest-latency in the public cloud with Local Zones, or in the core carrier network with Wavelength, AWS has you covered. When you add this to what AWS is doing with Snowball and where I think it’s going, it’s hard not for me to say AWS won’t have the broadest and most diverse hybrid play. After our analyst fireside chat and Q&A with AWS’s Matt Garman, I’m convinced we will see tremendous compute and storage variability with all of AWS’s offerings. It doesn’t have all the blanks filled in, but I believe it will. This isn’t for show; it’s for world domination.
What I’m most interested to see is how the economics and agility stack up compared to on-prem giants Dell Technologies, Hewlett Packard Enterprise, Cisco Systems, Lenovo and IBM.
3/ SageMaker Studio
AWS says the “Amazon SageMaker Studio is the first comprehensive IDE (integrated developer environment) for machine learning, allowing developers to build, train, explain, inspect, monitor, debug, and run their machine learning models from a single interface. Developers now have a simple way to manage the end-to-end machine learning development workflows so they can build, train, and deploy high-quality machine learning models faster and easier.”
Why this matters
Machine learning is really hard without an army of data scientists and DL/ML-savvy developers. The problem is that these skills are very expensive, hard to attract and retain, not to mention the need to have very unique infrastructure like GPUs, FPGAs and ASICs. AWS did a lot with its base ML services to help solve the infrastructure and SageMaker to connect the building, training, and deploying ML at scale. But how do you connect the developer on an end to end workflow basis? Enter SageMaker Studio. Studio replaces many other components and toolsets that exist today for building, training, explaining, inspecting, monitoring, debugging, and running that may make those ISVs unhappy, but developers could be a lot happier.
I’m very interested in lining this up against what both Google Cloud and Azure are doing and getting customer feedback. With SageMaker Studio, AWS isdelivering what enterprises want; the only question is if it’s better than or a lot less expensive in what devs can put together themselves or run on another cloud.
4/ Inf1 EC2 instances with Inferentia
Last year, AWS pre-announced Inferentia, its custom silicon for machine learning inference. This year, it announced the availability of instances based on that chip, called EC2 Inf1. AWS explains that “With Amazon EC2 Inf1 instances, customers receive the highest performance and lowest cost for machine learning inference in the cloud. Amazon EC2 Inf1 instances deliver 2x higher inference throughput, and up to 66% lower cost-per-inference than the Amazon EC2 G4 instance family, which was already the fastest and lowest cost instance for machine learning inference available in the cloud.”
Why this matters
Machine learning workloads in the cloud are split into training and inference. Enterprises train the workload with big data and monster GPUs and then run the model, or infer on smaller silicon close to the edge. Currently, the highest performance training and inference currently occurs on NVIDIA GPUs, namely the V100 and G4. Most inference is done on a CPU for lower cost and latency purposes as described by Amazon retail gurus during the last two Xeon launches. While I am sure NVIDIA is hard at work on its next generation silicon, this is fascinating as nothing has served as a challenge even to NVIDIA’s highest-performance instances. While I haven’t done a deep dive yet like Graviton 2 above, when I do, I will report back as will ML lead Karl Freund. Whatever the outcome, it’s good to see the level of competition rising in this space.
5/ No ML experience required AI services
AWS came out strong touting new services that don’t require ML experience. Think of these as SaaS or high-order PaaS capabilities where you don’t need a framework expert or even a data scientist. Amazon said
- “Amazon Kendra reinvents enterprise search by using natural language processing and other machine learning techniques to unite multiple data silos inside an enterprise and consistently provide high-quality results to common queries..”
- “Amazon CodeGuru helps software developers automate code reviews and identify an application’s most expensive lines of code.”
- “Amazon Fraud Detector helps businesses identify online identity and payment fraud in real time, based on the same technology developed for Amazon.com”
- “Amazon Transcribe Medical offers healthcare providers highly accurate, real-time speech-to-text transcription so they can focus on patient care.”
- “Amazon Augmented Artificial Intelligence (A2I) helps machine learning developers validate machine learning predictions through human confirmation”
Why this matters
I will posit that there’s more market opportunity for AWS in ML PaaS and SaaS if for nothing else the lack of data scientists and framework-savvy developers. If you’re not a Fortune 100 company, you’re at a distinct disadvantage to attract and retain those resources and I doubt they can be at the scale that you need them. Also, as AWS does most of its business in IaaS, there’s just more opportunity in PaaS and SaaS.
Kendra sound incredible and it will have an immense amount of competition from Azure and Google Cloud. Azure likely already has a lot of the enterprise data through Office 365, Teams and Skype and Google is good at search. CodeGuru sounds too good to be true but isn’t, based on a few developer conversations I had at the show. The only thing limiting this service will be the cost, which I think is dense, given what it can save, but it’s human nature to not see the big picture. Fraud detector, like Kendra, will have a lot of competition, especially from IBM who have been doing this for decades. I love that the service is bringing its knowledge from its Amazon.com dealings and I’d be surprised if the website has the highest fraud attacks given it does 40% of online etail transactions. Transcribe Medical is a dream come true for surgeons like my brother in-law and I hope AWS runs a truck through the aged transcription industry. AWS will have a lot of competition from both Azure and Google Cloud. A2I has been needed in the industry for a while as no state or federally regulated industry can deal with a black box.
There were so many good announcements to choose from I had to do an honorable mention list with my quick take.
- AWS Quantum- This is AWS’s quantum axe thrown in the sea including the Braket managed service, research lab, and application office. This is exactly the right approach especially with the three leading compute quantum approaches. I would have liked to have seen Honeywell in the trapped ion camp, but we’ll see how IonQ goes.
- AWS Nitro Enclaves- This provides further isolation above and beyond that from processor enclaves. I’m super interested to see how this works across AMD, Intel, NVIDIA, Xilinx, and home-grown AWS compute like Graviton1, Graviton2, and Inferentia.
- AWS Fargate for EKS- Not to be outdone by standard EKS or ECS, serverless fans now get a way to more easily deploy, manage, and scale K8S on AWS
- AQUA- AWS is using a hardware accelerated cache bringing ASICs and FPGAs to Redshift resulting in what it says is 10x better query performance than any other cloud vendor. This is another great Nitro love story.
- Amazon Federated Query- Enterprises don’t want to do separate queries on different systems and this feature lets customers analyze data across their Amazon Redshift data warehouse, Amazon S3 data lake, and Amazon RDS and Aurora (PostgreSQL) databases. It’s what it always should have been.
While it’s impossible to do justice to a huge event like AWS re:Invent in a single point, I also think it’s as important to point out the highlights with some honorable mentions. All in all, AWS answered the hybrid critics and raised the ante, introduced some homegrown silicon that de-commoditizes IaaS, and gave more reasons to use its databases and machine learning services from newbie to Ph.D.