The Six Five Team discusses Groq’s decision to do its manufacturing and to partner with Samsung over TSMC and IFS.
If you are interested in watching the full episode you can check it out here.
Disclaimer: The Six Five Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors and we ask that you do not treat us as such.
Transcript:
Patrick Moorhead: Groq chooses Samsung and a Samsung Foundry over TSMC and IFS. We talked about this last week when we were talking about this cool performance on LLaMA, but Dan, what’s up here? Two in a row, two weeks in a row talking Groq.
Daniel Newman: Well look, I mean the originator of the LPU.
Patrick Moorhead: The LP?
Daniel Newman: LPU.
Patrick Moorhead: Is that a record? Oh, yeah.
Daniel Newman: Layer processing units. Remember how much credit we gave Jensen when he came up with the DPU, even though I’m pretty sure that Marvell pioneered that more than Nvidia did. Hold on. Anyway, I’m sorry I had my alerts on, but even though that happened.
And then of course there are some other companies that are building chips for processing large language models, but Groq has been uniquely focused on being able to do that. And last week we talked about some of the advancements the company had made in terms of the tokens per second per user, this week we’re talking about the company making a decision to do its manufacturing and to partner with Samsung. Now Pat, that’s not often what you hear from an established semi designer, but Pat, I think this is an example of interdependence. I think Samsung sees Groq, and I believe it also, was it Tenstorrent that it also announced something with recently, Pat?
Patrick Moorhead: Yeah.
Daniel Newman: I think so. I think they’re looking at AI HPC data center solutions and trying to increase their relationships, and Groq is showing some really strong performance and they’ve decided to lean in with Samsung Foundry. Pat, we’ve talked about this a little bit with the Nvidia and what’s going on with AI and how companies like AMD could end up winning and how Intel could end up winning, the same with Groq. I think Groq here is making a strategic decision to partner with a company that sees value in its business, that’s willing to be probably a bit more flexible on its terms, invest into the relationship and help a company like Groq that has a strong capability to innovate and disrupt and at least help fulfill this surging demand for AI using Samsung’s capacity. And so to me it’s really straightforward, and if I’m Jonathan Ross, CEO and that team, I’m looking for a company that’s willing to invest with me in the future and that’s willing to give me the capacity I need.
And Pat, I can’t speak as much to what Intel maybe could or couldn’t have done here, but I think TSMC probably doesn’t have the same need or demand and probably giving up the kind of term, and obviously the volume commitments to a company like Rock could have been very challenging. So I think with all that in mind, Groq found a partner that was willing to play within its constraints, within its expected revenue and capacity and volume. And I think Groq as we’ve said, and we’re both investors, so I think it’s fair that we disclose that, I think for Groq it’s very important that they focus on the partnerships that are going to enable them to hit their growth marks as cost effectively as possible, with the global capacity and scale of someone like Samsung, it’s a good win-win.
Patrick Moorhead: Yeah, good analysis there, Dan. I’m going to probably take it from maybe a little bit of a tech perspective and maybe talk a little bit about the relationship there. The reality is that picking a foundry partner is more than just who has the best process, the bleeding edge, a lot of it comes down to relationships. Samsung made an investment in that, and quite frankly, TSMC is very hard to work with if you’re not a top five semiconductor player. And even when you’re a top five, they’re difficult, because what you’re competing with are companies like Apple who are paying TSMC billions of dollars upfront for reserved capacity.
But architecturally, I don’t think that Groq has to have a bleeding edge from TSMC. In fact, their architecture, it’s more of an ASIC than it is a GPU. And as we talked time and time again on this show, ASICs are a lot more efficient than a GPU at doing training and inference. Now a GPU is more efficient than a CPU, and I like to look at that at the overall wave of programmability, and FPGAs fit in there as well. So it makes sense to me where Groq might only need 6 nanometers, which is what Nvidia was on in their last, I think it’s the 8100 was on 6 nanometer, to be successful, because they have the efficiency of an ASIC to overcome that, like we saw with LLaMA and what they’re doing on that. I think it was a 70 billion parameter model, a lot less expensive in what you need to get done, which makes perfect sense.
Good stuff, congratulations to Samsung Semiconductor on that one.
