0:00
Apple’s M5 Pro, M5 Max and M5 Ultra have more in common than just their architecture, they actually share the same silicon.
Apple’s M5 Pro, M5 Max and M5 Ultra have more in common than just their architecture, they actually share the same silicon.
It’s Apple first true chiplet design.
And before you start typing angry comments into your keyboard, yes, I know about the previous “Ultra” chips, and no, they aren’t chiplets.
Chiplets are nothing new in the industry, so in true Apple fashion, being a bit late to the party, Apple had to one-up everyone else and introduce their first chiplet architecture with an actual brand-new die-to-die interconnect that can rightfully be called next-gen.
SemiAnalysis bought the chips, STEEL tore them down, and I’m gonna tell you all about Apple new Fusion Architecture, and what makes it so special.
M5 Pro, Max and Ultra aren’t the first Apple chips with more than one silicon die, but they are the first true chiplet design.
And the difference is in the approach.
Previous version, like the M1, M2 and M3 Ultra took two of the largest chips of an Apple Silicon generation, for example two M3 Max dies, and fused them together. Classic MCM. Multi-Chip-Module.
On its own, a single M3 Max already is a fully functioning SoC, it doesn’t need a second one to work.
And Apple also didn’t optimize silicon area with this approach.
A true chiplet architecture has two design targets.
First, optimize the area of the individual dies, the “chiplets”, in order to increase yield and second, outsource different functions of the SoC onto different chiplets.
The final product only works once you combine at least two different chiplets.
Of course, everyone is entitled to their own chiplet based opinion.
But that’s my point of view.
Apple taped out three different dies: one CPU die and two GPUs dies, a small and a large one.
The M5 Pro is built by combining the CPU die with the small GPU die, the M5 Max is created by combining the CPU die with the large GPU die and the M5 Ultra is the combination of two CPU dies with two large GPU dies. Kinda cool.
Before we talk about the new die-to-die interconnect, let’s take a quick look the individual chiplets first.
The CPU die is the common denominator between all three SKUs and like the other chiplets, it’s fabricated in TSMCs N3P process node.
With a die size of about 168 square millimeters, it’s actually larger than a base M5.
In total, the CPU chiplet contains 18 CPU cores, arranged into three clusters.
The S-core cluster looks a bit asymmetrical, with two S-cores on top and four S-cores below a 16-megabyte shared cache.
Each S-core comes with a 1-megabye slice of L2 cache and just like always, the S-core cluster has its own AMX unit.
The twelve new P-cores, internally called M-cores, are divided into two mirrored clusters, each containing six cores and eight megabytes of shared cache, plus dedicated AMX unit.
Take a second to check the size difference between the S- and the new P-cores.
I know I’m repeating myself, but M-core is actually a much more fitting name, at least area wise.
Over all, it’s a pretty interesting layout, especially that the twelve M-cores are split into two six-core clusters.
This might be due to power management, like being able to shut off an entire cluster if you don’t need it.
And before you ask, no, there are no E-cores on the CPU chiplet.
It’s just S- and P-cores.
On the left side of the die, we can find the classic 16-core Apple Neural Engine, which has since been doubled on the M6.
But here we still get 16-cores, or rather eight dual-cores.
Only the M5 Ultra gets double the NPU, because it uses two CPU chiplets.
To the right of the M-cores are four pretty massive Display Engines and the bottom shoreline of the die is used for a large array of IO.
From left to right we can find an Apple proprietary high-speed PHY, used for cameras and other sensors, a 4-lane Display PHY, and 16-lanes of PCI-Express, most likely gen 5, arranged in four 2 by 2 clusters.
The CPU chiplet also comes with four massive Thunderbolt 5 PHYs.
They are so big; they actually outclass the M-cores in size and aren’t that far behind an S-core.
It’s crazy to see how big analogue SerDes are becoming and how much space the command-and-control area is taking up.
And last, but not least, the CPU die also comes with an on-die NAND controller and AES engine in the secure enclave.
But there’s one thing missing.
And arguably, it’s the most important aspect about Apple’s new chiplet architecture.
We can find it on the upper shoreline of the die.
When we zoom in, we see a long row of what appears to be some kind of lanes or PHY, but there isn’t much to it, besides a faint striped structure.
What you are looking at is the physical layer of the new die-to-die interconnect.
And it’s unlike any other interconnect we have seen so far.
Or rather, it was never used in that specific way.
But before we check out how exactly it works, let’s take a quick look at the two GPU dies first.
The GPU chiplets are both interesting and a bit boring at the same time.
Produced in the same N3P process node, the large GPU die clocks in at around 300 square millimeters and the small one at about 157 square millimeters.
Depending on how you look at it, the large GPU chiplet is almost twice the size of the small one, or the small one is almost half the size of the large one.
And this relation continues once we look inside the silicon.
The large GPU chiplet comes with 40 GPU cores, 48 megabytes of System Level Cache and a wide 512-bit memory interface, while the small GPU chiplet contains exactly half of that: 20 GPU cores, 24 megabytes of SLC and a 256-bit interface.
Only some of the media acceleration logic close to the die-to-die interface stays the same.
So, the small GPU die is basically a cut in half large GPU die.
I guess that helps with development time.
But how is Apple connecting CPU and GPU dies?
And how do they manage the massive M5 Ultra? I’m so glad you asked!
Let’s go back to our CPU chiplet and the weird PHY we found on the upper shoreline of the die.
When we place the CPU die and any of the two GPU dies next to each other, like they are placed on the package, we can see that the GPU dies contain the same kind of PHY on the adjacent shoreline. Two matching structures.
But Apple has fused two dies together before, so what’s new with the M5 series?
What’s new is that Apple is using TSMCs latest SoIC-mH, also called SoIC-X F2F for Face-2-Face.
It’s TSMCs term for hybrid bonding, the same technology used for AMD’s 3D V-Cache for example.
But wait a second, 3D V-Cache is 3D stacking a cache die on top, or below, a compute die, and so far, hybrid bonding describes bonding two wafers, or chips, directly to each other, creating a chip sandwich.
But when we look at the package, CPU and GPU chiplets are placed next to each other, not on top of each other.
That’s where the “mH” part of “SoIC-mH” comes in.
It’s short for “molding horizontally” and it does exactly what it says.
The way it works is that the chiplets are bonded onto a special hybrid bonding silicon interposer.
At first glance not that different to other silicon interposers and packaging methods, but hybrid bonding has one major advantage.
Standard interconnects, even advanced packaging, is usually microbump based.
That means if you want to connect two chips via an interposer, you manufacture both chips, build out the metal layers, and then attach the finished die to the interposer via microbumps.
Tiny little drops of solder that complete the connection between chiplet and interposer.
And even though they are called “microbumps”, you can’t make them as small as you want, there’s a size limit.
And every bit of data that’s flowing between the chiplets and the interposer has to pass through these microbumps.
Hybrid bonding gets rid of microbumps altogether, no solder is used.
Instead, the chiplets and the silicon interposer have a so-called “hybrid bonding layer”, which is basically matching exposed copper interconnects surrounded by a dielectric material.
These interconnects are then perfectly lined up, which is way harder than it sounds, and then both the chiplets and interposer are heated.
The heat expands the copper, directly fusing the copper from the chiplets to the copper of the interposer.
The result is the highest quality interconnect you can think of, it’s almost like fusing the metal layers, creating one large coherent chip.
Because you can control copper better than solder, you can create smaller interconnects, which means you can put more interconnects in the same area.
Microbumps are currently in the 25-micrometer range, maybe going down to 18 micrometers, while hybrid bonding allows for a pitch of less than 10 micrometers and even approaching 3 micrometers for the most advanced process.
The result is an extremely high-bandwidth, low-latency and energy efficient interconnect.
And because SoIC-mH allows Apple to place CPU and GPU chiplets next to each other, instead on top of each other, there are no thermal problems.
BTW: SoIC-mH is a supply chain term, while TSMC calls it SoIC-X F2F, or Face-2-Face.
Both terms describe the same process.
So, the M5 Pro is CPU + small GPU chiplet, the M5 Max is CPU + large GPU chiplet.
But how does the M5 Ultra work?
And here we are running into a problem, because the lab didn’t get the M5 Ultra yet.
But we can figure that out by ourselves, all we have to do is to take a closer look at the large GPU die of the M5 Max.
On the opposite side of the CPU to GPU interconnect, basically taking up the entire upper shoreline of the large GPU die, we can find another SoIC-mH interface.
Only this time, it’s even larger than the one that connects CPU to GPU.
The structures are identical, which is already a very good indication that Apple is alos using hybrid bonding for the M5 Ultra.
But the final clue is Apple’s own marketing.
According to Apple, the M5 Ultra uses “UltraFusion” to connect two dual-die M5 Max chips.
That much we already know.
But Apple also claims that this version of UltraFusion increases the connection density by over 6x.
And that would be impossible without hybrid bonding.
The last UltraFusion chip, the M3 Ultra, used InFO-LSI packaging with a local silicon interconnect and traditional solder-based microbumps.
Apple never publicly confirmed the bump pitch of their Ultra chips, but measurements show a 25-micrometer pitch.
And a 6x density increase means the new pitch has to be about 10 micrometers or less, a feature size that’s well into hybrid bonding territory.
SoIC-mH follows a recent trend for Apple, showing an increased concern for thermals, like the A20 Pro switched to WMCM packaging, moving the DRAM to the side of the SoC.
Like other TSMC customers, Apple had access to hybrid bonding for a while now, but choose to wait for the side-by-side implementation.
It’s not unlikely that Apple actively worked on this solution with TSMC.
Hybrid bonding further elevates the M5 series.
A higher interconnect density and direct copper-to-copper contacts reduce the amount of energy required to transfer data and increases the bandwidth potential.
Not only did we get the new S-cores with a private slice of L2 cache and new M-cores, but also a brand-new packaging technology that offers real benefits over older, solder-based solutions.
Can’t innovate anymore my ass.