It has been a bit over a week since AWS re:Invent 2018 ended in Las Vegas. I had the chance to attend Amazon’s premier conference for all things AWS with 50,000 other on-site participants (100,000 online) and it was a great opportunity to see what’s new in its #1 market share cloud services portfolio. The conversations with AWS executives and customers was quite enlightening and helpful, too.
AWS re:Invent has become an enterprise bellwether industry conference as you will likely see the competition emulate or blatantly copy the announcements months or years later. There were too many announcements to cover in its totality here, but today I wanted to provide a highlights reel of what I believe to be the top announcements from the event and its implications. You can also catch re:Invent analysis from Matt Kimball (Arm compute), Karl Freund (ML) and Steve McDowell (storage) here.
AWS Lake Formation- expand the SAM
A data lake is basically a location that stores all customer’s data—both structured and unstructured data required for analysis. Data lakes are so important now because to provide analytics and ML on large data sets most effectively, the data needs to be in the same place.
At re:Invent 2018, AWS launched its new AWS Lake Formation service, which is designed to enable users to easily set up a secure data lake in a matter of days versus months. Data lakes make it easier to combine different kinds of analytics and break down data silos, theoretically resulting in better business insights. Data could be in a thousand different places in the enterprise and provides zero cumulative value if unconnected.
While creating these data lakes was historically complicated and time-consuming (months), AWS Lake Formation seeks to automate and streamline the process. With this service, all customers need to do is specify where their data resides, and which access and security policies they want to put in place. AWS Lake Formation then gathers and catalogs data from databases and object storage using ML, transfers it into an Amazon S3 data lake, applies machine learning to clean and classify data, and ensures secure access. I am sure it is more complex than this, but infinitely less complex than setting up your own data lake.
This is one of the first features that pays off the AWS goal to expand its services to a broader audience, a less technical audience, and increase its SAM (Serviceable Available Market). Some enterprises just want more prescriptive solutions and Lake Formation is just that.
AWS Control Tower- expand the SAM
Next up is the newly announced AWS Control Tower, which the company touts as “the easiest way to set up and govern a secure, compliant multi-account AWS environment.” It does so by automating and configuring a landing zone to manage AWS workloads, with parameters in place for security, operations, and compliance, based on established best practices. Users have access to “blueprints,” which are best practices for configuring AWS security and management services. “Guardrails” do just what you would expect, and that is to give warnings when your internal customers are about to veer off the policy road. The offering provides users with ongoing policy enforcement, as well as an integrated dashboard view of their summarized AWS environment.
In short, AWS Control Tower ensures all new AWS accounts are aligned with company-wide compliance policies, without slowing down the momentum of the development teams who provision the new accounts. Previously, enterprises could build their own landing zones- Control Tower is a turnkey solution.
I think AWS Control Tower will be warmly received by enterprises who want a more turnkey and locked-down public cloud experience for their developers. Like AWS Lake Formation, Control Tower could broaden the market opportunity for AWS to those enterprises who want more control over their internal developer customers. I can see this being quite popular in financial, healthcare, and government verticals.
AWS Security Hub- expand the SAM
AWS also announced its new AWS Security Hub. AWS Security Hub is a new service available in preview that provides users with a comprehensive summary of their high-priority security alerts and compliance statuses across their various AWS services. It compiles users’ security learnings from a variety of AWS services, including Amazon GuardDuty, Amazon Inspector, Amazon Macie, and additional solutions from AWS partners. Security Hub enables users to perform continuous, automated configuration and compliance checks, which can identify specific accounts within environments that need further attention. The high-level view provided by Security Hub promises to make it easier to spot trends, identify issues, and remediate when necessary.
Again, enterprises could build these on their own, but that is incredibly difficult given the sheer number of security services and vendors who are making constant changes. Instead of enterprises playing whack a mole chasing security vendors, security vendors write to an AWS API and customers use the Security Hub.
Amazon Elastic Inference- reducing customer cost “up to 75%”
AWS customers are used to “buying” their GPUs by the month, week or day for only what they use, but for customers who aren’t using a full GPU instance, this may not be optimal and could be expensive. Amazon Elastic Inference is a service designed to allow users to add GPU acceleration (1-32 TFLOPS per accelerator) to any Amazon EC2 and Amazon SageMaker instance and literally only pay for its exact use for a fraction of the cost of traditional deep learning inference.
According to Amazon, this service allows users to choose the best-suited instance type for a specific application and attach “just the right amount” of acceleration, no code change required. By matching capacity to demand, Amazon says this flexibility can lower the costs of inference by as much as 75%--which is significant since inference often accounts for the bulk of the costs associated with a deep learning application. I found it eye opening and informative for Amazon to say that 90% of their ML costs are inference versus 10% training. Amazon would know as it has Alexa, the premiere in-home assistant.
Unlike other competing ML services from Google Cloud, Elastic Inference isn’t limited to Tensorflow, as Apache MXNet and Pytorch have planned support.
Amazon Elastic Inference hits on another major theme- cost cutting for customers. While AWS makes a hefty quarterly profit, it is also aggressive about saving money for its customers and Elastic Inference is a great example.
AWS Inferentia custom ML chip- reducing customer cost
A related ML announcement to Elastic Inference was the unveiling of Inferentia, a custom chip designed specifically to deliver machine learning inference at a lower cost. While Elastic Inference could save costs by attaching acceleration to EC2 and SageMaker instances that don’t use a full GPU, some workloads do require a full chip and could use dedicated inference chip to get the job done more efficiently. AWS says those customers requiring a full GPU can save on an order of magnitude with Inferentia.To this end, AWS says AWS Inferentia delivers high throughput (hundreds of TOPS and banded together for thousands of TOPS) and low latency inference performance. The chip will be available for use with Amazon SageMaker, Amazon EC2, and Amazon Elastic Inference, and will support TensorFlow, Apache MXNet, and PyTorch frameworks, along with ONNX-formatted models and mixed precision workloads. Supporting many frameworks is important as different ones are better for different ML workloads. Generally, the community embraces that Apache MXNet is best for video analytics, recommendations, and NLP; Caffe is best for vision and Pytorch 2 is showing great research value. Amazon reiterated many times during the show that most of its customers are using many different frameworks.
There is a lot of analysis to be done for me to say specifically how this compares to GPU, CPU and FPGA inference capabilities and costs, but as more information becomes available, you can bet that ML analyst Karl Freund and I will be all over it. What I can categorically say right now is that AWS Inferentia looks infinitely more flexible than Google GCP’s TPU with support for so many different frameworks.
AWS Outposts- expanding the SAM
One of the biggest announcements of the week was that AWS is going on-prem with its new Outposts offering—AWS custom-designed hardware in the enterprise datacenter. AWS Outposts will bring the same native AWS services, software, infrastructure, management tools, and deployment models customers already use in the AWS or VMware cloud to basically any datacenter or on-prem environment. If customers are starting with the public cloud, this could reduce the complexity of hybrid cloud, since customers will no longer have to navigate different, disparate, multi-vendor IT environments. It’s also a big single vendor commitment, too.
AWS Outposts will be available in two different offerings at the end of 2019: VMware Cloud on AWS that runs on Outposts, and AWS Outposts that allows customers to use the same native APIs used in AWS. Yes, the same native APIs. The Outposts infrastructure will be fully managed, maintained and supported by AWS, with regular hardware and software updates to the latest AWS offerings. I found it quite interesting that Amazon Outposts will only require 1-2 servers—not a full rack or fleet of racks.
I told everyone years ago AWS would eventually go even more hybrid, and now it has. AWS is now headed on-prem—this is huge. While AWS took its time getting into the hybrid cloud, enterprises I talk with want it, want it done right, and it’s safe to say it is all-in now.
The single Outposts API is a big deal for hybrid.
Outposts isn’t AWS’s first hybrid offering, it is the deepest yet. AWS already offers Snowball Edge, Vmware Cloud on AWS and many ways to integrate on-prem resources with AWS including Amazon Storage Gateway, VPC, Direct Connect, Systems Manager, Identity and Access Management, Directory Service, OpsWorks, and CodeDeploy. I see the AWS Outposts hybrid up-level as a way to pull over those apps requiring the lowest latency and those who just want the data close by for other reasons like security and control.
There are many questions to be answered about Outposts and we will be intently watching out for answers like which exact compute, storage and networking options are available and when, and of course, pricing. AWS said Outposts would have “the same breadth and depth of features,” which, if taken literally, could number in the thousands, which I think would be very hard to do. Also, interesting too is that server form factors dimensions as rack size, shapes and power is slightly different across enterprise data centers. For instance, in China carriers, racks are smaller sized to fit in the carrier’s elevators. Oh, and they are painted white. No, I’m not kidding.
Glacier Deep Archive- lowering costs
Amazon also announced a new Amazon S3 storage class, called Amazon Glacier Deep Archive. Essentially, this storage class is geared towards long-term data retention, well-suited for archival data that is infrequently accessed, which tape cannot do.
It’s the lowest price storage offering in AWS, at less than $.001 per gigabyte, per month. According to Amazon’s Andy Jassy, with Amazon Glacier Deep Archive now an option,“ You’d have to be out of your mind to manage your data on tape.” I don’t know if I fully agree with that yet, but it certainly makes it more difficult to justify new tape installments when one looks at the price and accessibility.
Arm EC2 A1 Instances- lowering customer costs for specific workloads
The last bit of big news from re:Invent I wanted to hit on is the immediate availability of Amazon’s new Arm Neoverse-based EC2 instances, known as EC2 A1, powered by “Graviton,” AWS’s custom Arm server chip. There are five different instances that fit under this A1 umbrella, which range from 1 to 16 virtual CPUs, and from 2 to 32 GB of RAM. AWS says the A1 instances are ideal for scale-out workloads and applications like container-based microservices, websites, and scripting language-based applications. AWS quoted an eye-popping 45% cost reduction and I will need to dig into the claim. More narrowing the targeted use case makes a lot of sense as Graviton version one uses the Arm A72 core today, but I expect much higher performant A76-based instances with higher IPC and cache sizes in the future.
Moor Insights & Strategy has been covering the Arm-based enterprise for nearly a decade and A1 is significant because it represents the first time a major cloud provider has deployed Arm general purpose compute at scale. Some have read this to think that this means AWS is moving off Intel, which is ridiculous. AWS is embracing a more aggressive multivendor CPU (and for that matter GPU) strategy with AMD, Arm and Intel, designed to either lower costs or add unique capabilities for its customers. One of Amazon’s secret weapons here is “Nitro”, it home-grown compute virtualization architecture that more easily enable mix and match compute.
AWS demonstrated several important themes at the event this year. The two that made the big difference to me were its march to expand its SAM through simplifying the offerings to a less technical crowd and the continued efforts to lower costs through down-shifting, fractional services and building custom chips.
It appears that AWS is finally serious about the HPC market, with the right compute instances, storage, file systems, and networking. It has a rock solid 3-tier machine learning strategy: 1) offer the nerds (no disrespect intended) everything they want with IaaS and frameworks; 2) offer PaaS SageMaker to the data scientists who aren’t gearheads; and 3) for everyone else, go vertical and horizontal with no ML experience required. AWS Outposts is huge, and Amazon’s entry into the hybrid cloud space is going to have huge industry reverberations. AWS continues its march towards vertical integration with its custom silicon—I’ll continue to watch with interest.
As you can see, there was plenty to wrap one’s head around at AWS re:Invent 2018, and these were just my highlights. Be sure to check out the complete Moor Insights & Strategy re:Invent analysis from Matt Kimball (Arm compute), Karl Freund (ML), Steve McDowell (storage), Chris Wilder (IoT) and Rhett Dillingham (cloud services).
Note: Moor Insights & Strategy writers and editors may have contributed to this article.
Patrick founded the firm based on his real-world world technology experiences with the understanding of what he wasn’t getting from analysts and consultants. Ten years later, Patrick is ranked #1 among technology industry analysts in terms of “power” (ARInsights) in “press citations” (Apollo Research). Moorhead is a contributor at Forbes and frequently appears on CNBC. He is a broad-based analyst covering a wide variety of topics including the cloud, enterprise SaaS, collaboration, client computing, and semiconductors. He has 30 years of experience including 15 years of executive experience at high tech companies (NCR, AT&T, Compaq, now HP, and AMD) leading strategy, product management, product marketing, and corporate marketing, including three industry board appointments.