On this episode of The Six Five On The Road, hosts Daniel Newman and Patrick Moorhead welcome Jinman Han, President of Samsung Semiconductor, US for an inside look at Samsung Memory Tech Day happening in San Jose, California. They cover the driving forces changing the Memory business, including AI, as well as key innovations Samsung is showcasing at the event, and more.
Their discussion covers:
- An overview of the event and what announcements and key innovations are taking center stage
- How the Memory business is changing and what new applications and verticals are driving Samsung’s approach to innovation
- The impact of AI on the Memory industry
- Insight into Samsung’s strategy in leading the Memory industry
Be sure to subscribe to The Six Five Webcast, so you never miss an episode.
You can watch the full video here:
Or Listen to the full audio here:
Disclaimer: The Six Five Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded, and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors, and we ask that you do not treat us as such.
Patrick Moorhead: The Six Five is live. It’s Samsung Memory Tech day, and we are here chatting up, quite frankly, one of the elements of semiconductors that just needs to get more credit as we’re moving into this AI world, as we’re cranking out more devices out on the edge. Daniel, how are you my friend? Here, we are talking semiconductors and maybe a little bit of AI, maybe a little bit.
Daniel Newman: You think we’re going to talk about AI?
Patrick Moorhead: I don’t know, we talk about AI.
Daniel Newman: That seems unlikely, but yeah, no, it’s good to be here. I’m very excited for another great conversation here on the Six Five, Pat. We are visiting with our friends from…
Patrick Moorhead: Samsung Semiconductor. Jinman, let me reintroduce you. You are not a first-timer to Six Five. You are a Six Five veteran. You are part of the Six Five summit. It’s great to have you and thank you for letting us be part of this big announcement day.
Jinman Han: Oh, thank you for having me. It’s a beautiful day here in San Jose and it’s also a beautiful day to discuss technology. I’m very excited to have this Samsung Tech Day today.
Daniel Newman: It’s so good to have you here and I’m really excited to hop into this conversation. Samsung Tech Day is memorable to me. If you remember, I was on stage just six years ago giving some of the digital transformation trends that I saw, right before we had Steve Wozniak come on and do a presentation.
Patrick Moorhead: Yeah, it was fun.
Daniel Newman: It was fun to make a little trip down memory lane. I cannot believe that it has been six years, but here we are again. We’re still having these conversations and there’s still a great reason that people need to be joining us here at Samsung Memory Tech Day.
So let’s tell them about that. Let’s start there. Give us a little bit of the background about this event and what it’s all about.
Jinman Han: Yeah, this is our sixth annual event. Like I said, six years ago we met on the stage and actually this is one of our biggest events of the year. Right now, we have more than 600 customers and partners as well as industry experts. They’re here to collaborate, network and learn what’s new in the memory business.
We also have this unique feature of 24 unique breakout sessions, because we really want to give our partners, customers a deep dive into the technology details so that not just about keynote speeches, we also want them to experience lively discussion with our engineers and talk about a lot of questions. So, we really want to go into this technology detail. This is different from previous year,
Right now we are covering everything from automotive and, of course, AI to even sustainability and computer memory solutions, SSC, net solutions, almost everything. So there is always something for everyone and also, as a bonus, we have invited Chris Miller. He’s the author of Chip War, a very popular book in the semiconductor industry, and he’s going to participate in the panel with our head of our memory business. I’m really excited to have him on the stage. I’m looking forward to the discussion.
Patrick Moorhead: That’s great. When I started in this industry over 30 years ago, it’s not that memory was straightforward, but there were just a lot less different devices and applications. You had mainframes, mini computers. We were entering the client server era. But as memory and the applications and the use cases have diversified so much, how has this changed your approach to overall innovation? Because on one hand, these different types of applications require different fundamentals of memory, but then on the other hand you have to have scale. So, how do you manage your innovation pipeline?
Jinman Han: You’re right. So when people hear about memory business, what comes to their mind right away is a commodity, right? It’s been their way for a long, long time, but right now we are seeing a little bit of a different trend in the memory industry. So all the computer requirements need different memory solutions. So, we are talking about AI accelerators, all these compute solutions, new architecture discussions, but behind all this, you really need a very optimized memory solution.
What they’re trying to do here is, we really have to expand our ecosystem so that we can meet all the requirements from different compute requirements. So, we really have to look at this industry in a holistic way. We have to move from just commodity product concepts to our solutions focused product solutions, which will make our product much more conducive for all different kinds of requirements from cloud service providers as well as all the high performance computing industry. That’s the real change from our perspective. Everybody has to think about what’s going to be the future architectural changes to meet all these AI requirements. I think that memory is going to play a central role in that endeavor and so we really have to think about memory, not just from the commodity perspective, more about optimized, fine-tuned, and customized solutions. That’s what I’m seeing right now.
Daniel Newman: You said a lot of things there and you probably heard Pat and I in the preamble, Jinman, that are we going to talk about AI? I think when you start to talk about cloud scale, you start to talk about high performance, you start to talk about all of these things. What you can’t look away from in 2023 has been this massive trajectory around AI and Pat and I have talked a lot about this symbiotic relationship that coexists between memory. If you want to be able to run inference or you want to be able to run these workloads that are AI intensive, memory at all levels, in the data center on the device, and the PC is going to be really, really important. But I think the market really wants to fully understand and appreciate this. How are you sort of explaining the relationship between demand for memory and this growth of AI? And is this a huge opportunity in your mind? Does Samsung see, it might’ve been obvious, does Samsung see the potential here and can you talk about how big that is?
Jinman Han: Yeah, as we all know that AI has arrived. So the discussion these days is all about AI and the change was so broad, but I think underneath this AI revolution has been going on for a long, long time. But after the introduction of ChatGPT, suddenly people realized the real potential of this AI application. So from memory perspective, we used to think about this industry changing from the PC era to the smartphone era and server era. But right now this AI, I believe, is becoming more ubiquitous. It just runs across all the user applications from mobile phone to PC servers and hyper-clouding. So I think this is only the tip of the iceberg and it’s getting ubiquitous every day.
As an example, last quarter, second quarter of this year, more than 3000 companies mentioned AI in their earnings call. It’s so popular, right? Samsung also of course mentioned AI quite a lot during the earnings call. The first half of this year, I think AI startups. That’s the real trend, everybody is focusing on how to solve this AI issue. So I think this is just the beginning, beginning of the big thing to come.
It should come as no surprise that Samsung’s working on bringing the best AI memory solution to the market because, like I said before, memory is going to be a fundamental component that can feed all the data that AI accelerators need to process the training and even inference. So, we are very excited about this potential.
Patrick Moorhead: Yeah, I like the way that Daniel, you set it up and Jinman, you carried that forward. About every five or six years there’s something big that comes along and, as analysts, we always try to determine, hey, is this a fad or is this a trend?
I can do the tick boxes of which ones were trends and fads. This one is definitely a trend. This is real and what makes it even more believable is the fact that we’ve been taking a whack at AI for a while now. The original algorithms were created in the sixties and even five, seven years ago, we went through this AI boom that I think left a little bit short, but with generative AI, it really does look like the changes that we had envisioned in science fiction are going to come true and memory is front and center.
So I did the setup. It’s an exciting day here at Samsung Memory Tech Day, and you made a lot of announcements. So let’s hear what you announced today.
Jinman Han: Oh yes, thanks for asking. Today we are announcing Samsung’s next generation high bandwidth memory, which is called HBM3E. We trademark as Shinebolt, and this new HBM3E really meets the capacity demands because it can support more than 24 gigabytes per stack, and then it will allow our customers to innovate faster. I cannot wait to see what our customers can do with these new technologies.
We already have begun shipping to sell the customers and it is commercially available in early 2024. Like I said, the die itself is 24 gigabits and it is about 50% bigger density than the previous generation. And It has an impressive speed of 9.8 gigabits per second per pin, which means you can achieve transfer rates exceeding optimal than 1.2 terabytes per second, which is an amazing number.
Daniel Newman: Let me double check you there. 9.8. Okay. I believe Pat, is that the fastest we’ve seen?
Patrick Moorhead: That’s the fastest I’ve heard of so far.
Jinman Han: Yeah, looks that way. I think the highest I’ve seen it report so far is about 9.2 Gb/s, but what really we are excited about is the trajectory that this puts our business on as AI continues to explode.
Daniel Newman: Okay. So significant increase generation to generation and over the competition. How do you feel this helps you stack up to drive your next wave of growth at Samsung for the memory business?
Jinman Han: Yeah. Now you may have more about the other guys, but really Samsung has a long leadership in HBM. For example, in 2016, we commercialized the industry’s first HBM for high performance computing. At the time, no one was really talking about AI. But anyway, we were the first company to introduce this great technology to the market. The next year in 2017, we introduced a eight layer stack HBM2, so what we are announcing today is HBM3E, and this is going to be a huge improvement over the last products. We are really looking forward to seeing what our customers can do with this new advance. This is a great product. I’m very proud of this.
Patrick Moorhead: By the way. Higher number usually means higher performance, higher performance per watt, better throughput.
Daniel Newman: Pat.
Patrick Moorhead: Yes.
Daniel Newman: We should validate that.
Patrick Moorhead: Oh yeah. I’m thinking we should. Okay. I know a certain vendor that’s going strong in Q1, but we won’t talk about that right now.
Daniel Newman: So what can I do, just for folks who might not be as tied into the memory market, understand high bandwidth memory. We do understand that 3E is better than three and it’s better than two, but what are some of the applications? What are maybe some new black and white demonstrable things that 3E can do that its predecessors or the competition can’t?
Jinman Han: The main application use case for HBM3E is for training of AI accelerators. It can feed a lot of data in a very short latency with this huge transfer rate. So, the main customers will be data centers and cloud computing. And also, a lot of supercomputers need vast amounts of HBM3E technologies to meet their demands.
We also are providing this technology to network companies because the network devices, to support 5G, really have to have faster data transmission as well as lower delay, low latency. So, HBM3E is very critical for 5G or 6G or beyond. Yes.
Daniel Newman: So Samsung Tech Day is never just one thing. I totally get it. Big moment, big breakthrough. Probably the headline item, besides bringing in the chip wars. That’s a very good book, by the way. Pat, I think you and I probably have written-
Patrick Moorhead: I lived in the chip wars for over 20 years, so I might have to read the book.
Daniel Newman: You’re right. Still we are still in there.
Patrick Moorhead: Yes.
Daniel Newman: You were probably on TV today talking about chips.
Patrick Moorhead: I think I was.
Daniel Newman: I don’t think a day goes by that we don’t have to opine, so this sounds like a really just exciting conversation.
But, Samsung Tech Day has many innovations. Can you talk a little bit about maybe some of the other key innovations that you’re planning to showcase today?
Jinman Han: Oh, of course. So in addition to HBM3E, we are also announcing GDDR7. Actually, we already announced this in May this year, but we are going to talk a lot about this technology because it can support industry first, 32 Gb/s. 32 gigabit per second and this will expand capabilities in graphics performance, such as workstation, PCs and gaming. So for gaming, game hardcore fans will love this technology to see how smooth their gameplay can be with this great technology.
So, this GDDR7 sets a new standard in data transfer speed by adopting the PAM3, which means it has three different levels of output, which will allow for more than 50% higher data transmission compared to NRZ signaling within the same signaling cycle. This is the first time we introduced PAM3 technology, GDDR7, and it achieves an impressive bandwidth, 1.5 terabytes per second with 12 chips in parallel, which is about 1.4 times the data with the predecessor, which was 1.1 Tb/s. This features boosted speed per pin of up to 32 Vb/s.
This is the fastest memory per pin, even faster than HBM in terms of pin speed. So, this is great technology for the gaming industry and workstation industry. Yes.
We also have announced DDR5. We just announced this a few weeks ago. A unique thing about this is that this is the industry first and highest capacity DDR5 DRAM, which has 32 gigabit density per chip using our 12 nanometer class process technology. For your reference, Samsung has been increasing the DRAM capacity by a factor of 0.5 million over the last four years. This is amazing.
We are really fulfilling Moore’s law. We are the only one, I think the memory industry is a real industry that really can have been truthful to Moore’s Law. So our DDR5 has the highest capacity on a single chip, like 32 gigabit. This is double the density of the previous generation. This is a great technology. So we can have a very big, high density module for servers, but right now, you know that the number of cores in the CPU is increasing every year. Each core needs a certain minimum density of DRAM to support the core, so this technology will allow our customers to adopt many core CPUs in their system. Yes.
Patrick Moorhead: This is great, so not only does the AI market rejoice with HBM3E, you’ve got the next generation of GDDR and you have gamers rejoicing. I believe that we are going to see some massive growth in the gaming industry when we get some of the new AAA titles that were a little bit delayed over the past couple of years. But that’s the driver for all of high performance gaming. It’s great to see.
And by the way, just when we thought the amount of cores or threads per SOC in the data center would go down, we actually saw a new part that got announced with 576 threads on a single SOC. If you can’t feed those cores, you’re going to have latency. Pour in the memory in there to make everything speed up. So this is great. This is coming together here. I love this.
I want to step back a moment and maybe take a wide view of this. My relationship with Samsung, it was storage and memory, goes back to the 1990s, and Samsung is the market share leader in memory, no matter how you cut this. The question I have for you, it’s hard to get there. It’s hard to be the leader. How are you going to maintain your leadership posture here?
Jinman Han: Well, Pat, thank you for raising the topic. To be in leadership really takes quite a lot of things. As technology gets difficult to improve, because right now process scaling is getting slower and the demand from customers is very wide and varied and at the same time, maintaining the leadership for all these different market segments is really tough.
But, I think Samsung has these three philosophies. First one is to really have to innovate. So our first top priority is innovation and not just increase the density of DRAM and M-flash, we also have to offer differentiated and customized solutions. Right now, there are a lot of discussions in the industry about new computing architecture in the future to address all these requirements, especially from the AI industry.
So when you think about a human brain, there are a lot of neuron cells. And when you look at the brain, it’s really hard to differentiate where data is stored, where compute is happening. This industry has been having this compute and memory as a separate entity, but in the future, our memory might have to be integrated more into compute area so that we can emulate what our brain is functioning. I think that may be the eventual architectural evolution that we really have to address in the future.
To do that, we really have to innovate. Not just increasing the density and speed and loading power, but we have to work with compute industry to address these new requirements and needs from the industry in a totally different and new way. I think memory will play a critical role in that endeavor. So, innovation is our top philosophy. We’ll not continue to innovate.
Another big thing that people don’t really talk about very much is investment.
Daniel Newman: Exactly.
Jinman Han: Semiconductor industry is all about scale, scale. And Samsung is one of very few companies in the world right now who doesn’t significantly reduce R&D and CapEx. We are actually investing quite a lot of money into Fab and equipment tools in the USA and in Korea. This investment is really the real secret sauce for us to provide our innovative solutions to our customers in a very timely manner without disrupting this whole industry.
Patrick Moorhead: Well, Dan and I are broadcasting from Austin, Texas. We are very well familiar with your investments here in the city, and we appreciate the stewardship and the partnership between Samsung. Not that I represent Austin, I don’t. I’ve lived here for maybe 25 years, but watch the economies that you’ve supported and all the families who work there who pay taxes. I mean, it’s incredible and I cannot wait to see when you prop up your next latest facility.
Jinman Han: Have you visited our fab in Texas?
Patrick Moorhead: The first one, but not the new one. Okay. Yeah. So I visited Logic One and then Logic Two and you expanded it, but not the new one yet, but everybody’s talking about it here.
Daniel Newman: Invite incoming?
Jinman Han: Okay. Yeah.
Patrick Moorhead: I think I heard an invite, Dan. I don’t know.
Jinman Han: I’ll invite you guys to the Taylor Fab. It’s going to be a great site and you can feel the scale of this semiconductor industry.
Daniel Newman: We got to wrap up here. But what I will say is there’s a lot of really exciting things you’re sharing today and the importance of being able to tie the macro trends to all the parts and pieces that are required. So we all like to hop on and do the, I joke I say the Charlie Brown of AI, just AI, AI, AI, AI and you just say it a lot in your marketing and things are supposed to happen, but it is companies like Samsung that are serving as a critical enabler. All the GPU horsepower on the planet without memory, doesn’t enable very much. You need these things to work together to create these exciting applications, the future of vehicles, the future of computing, of personal, of mobile devices.
So, that’s a lot of what I see as what this Samsung Memory Tech Day really is all about. It’s about key innovations that are going to enable these very exciting trend lines. I think on behalf of both Patrick and myself, I’d like to congratulate you on the breakthroughs of today and just say, we do look forward to that Fab tour. I’m going to send you some open dates and we’re going to make that happen.
Patrick Moorhead: Yeah, let’s make it happen.
Jinman Han: Oh, the last thing I really want to focus on before we wrap up this nice talk is, something’s really serious about collaboration. We just mean it, because like I said, with all these different and wide and various requirements from our customers, this collaboration is not just a marketing term. We really have to collaborate with our partners and customers to make sure that we can move from here to there and we are not sure what there will be like in the future, but as long as we talk together and think about how to solve these problems, I think Samsung can be really playing a very good role in this industry to move this whole industry for better world. This collaboration is a real philosophy that Samsung is very, very serious about. So today, I think a lot of attendees will feel that collaborative spirit. I really hope you can also enjoy that too.
Patrick Moorhead: For sure. And by the way, the industry has changed. It used to be one day that JEDEC committees would get together and it was about a certain device, but right now in this is day and age of AI, memory is a discussion that needs to be part of the data center infrastructure, the architecture of the entire data center, particularly the way that we’re headed here. I like to think of this as there’s the CPU, GPU, and there’s memory. I mean, those three things have to come together on the smartphone, on the pc, on the edge of the data center and in the center of the data center for us to move this innovation at a pace that I know everybody is expecting us to do that, and co-investment is paramount to make this happen.
Jinman Han: Right. I cannot agree with you more.
Daniel Newman: Well, thank you so much, Jinman, and it was a lot of fun to be here with you on Samsung Memory Tech Day. We’ll have to do this again more often. It’s great to hear from Samsung and your team on the memory and semiconductor side, but for this episode, you know the drill, we’ve got to let our people get back to their day. Hopefully you follow all of the announcements at the Samsung Memory Tech Day event. We’ll put some links in the show notes that you can check out more.
Jinman Han: Oh, thank you.
Daniel Newman: Hit that subscribe button. Join us for all of our episodes here on the Six Five, but we got to go for now. So we’ll see you all really soon.