Apple's New Chiplet Design

0:00

Apple’s M5 Pro, M5 Max and M5 Ultra have  more in common than just their architecture, they actually share the same silicon.

0:06

It’s  Apple first true chiplet design.

0:06

And before you start typing angry comments into your  keyboard, yes, I know about the previous “Ultra” chips, and no, they aren’t chiplets.

0:17

Chiplets are nothing new in the industry, so in true Apple fashion, being a bit late to  the party, Apple had to one-up everyone else and introduce their first chiplet architecture  with an actual brand-new die-to-die interconnect that can rightfully be called next-gen.

0:35

SemiAnalysis bought the chips, STEEL tore them down, and I’m gonna tell you all about Apple new  Fusion Architecture, and what makes it so special.

0:49

M5 Pro, Max and Ultra aren’t the first  Apple chips with more than one silicon die, but they are the first true chiplet design.

0:54

And the difference is in the approach.

1:00

Previous version, like the M1, M2 and M3  Ultra took two of the largest chips of an Apple Silicon generation, for example two M3  Max dies, and fused them together. Classic MCM. Multi-Chip-Module.

1:12

On its own, a single  M3 Max already is a fully functioning SoC, it doesn’t need a second one to work.

1:20

And Apple also  didn’t optimize silicon area with this approach.

1:26

A true chiplet architecture has two design  targets.

1:26

First, optimize the area of the individual dies, the “chiplets”, in order  to increase yield and second, outsource different functions of the SoC onto different  chiplets.

1:38

The final product only works once you combine at least two different chiplets.

1:44

Of course, everyone is entitled to their own chiplet based opinion.

1:48

But that’s my point of view.

1:52

Apple taped out three different dies: one CPU  die and two GPUs dies, a small and a large one.

1:59

The M5 Pro is built by combining the CPU die  with the small GPU die, the M5 Max is created by combining the CPU die with the large GPU die  and the M5 Ultra is the combination of two CPU dies with two large GPU dies. Kinda cool.

2:14

Before we talk about the new die-to-die interconnect, let’s take a quick  look the individual chiplets first.

2:24

The CPU die is the common denominator between  all three SKUs and like the other chiplets, it’s fabricated in TSMCs N3P process node.

2:29

With  a die size of about 168 square millimeters, it’s actually larger than a base M5.

2:36

In total, the CPU chiplet contains 18 CPU cores, arranged into three clusters.

2:42

The S-core cluster looks a bit asymmetrical, with two S-cores on top and four S-cores below  a 16-megabyte shared cache.

2:48

Each S-core comes with a 1-megabye slice of L2 cache and just like  always, the S-core cluster has its own AMX unit.

3:01

The twelve new P-cores, internally called  M-cores, are divided into two mirrored clusters, each containing six cores and eight megabytes  of shared cache, plus dedicated AMX unit.

3:13

Take a second to check the size difference  between the S- and the new P-cores.

3:13

I know I’m repeating myself, but M-core is actually  a much more fitting name, at least area wise.

3:24

Over all, it’s a pretty interesting layout,  especially that the twelve M-cores are split into two six-core clusters.

3:28

This  might be due to power management, like being able to shut off an  entire cluster if you don’t need it.

3:36

And before you ask, no, there are no E-cores  on the CPU chiplet.

3:36

It’s just S- and P-cores.

3:43

On the left side of the die, we can find  the classic 16-core Apple Neural Engine, which has since been doubled on the  M6.

3:47

But here we still get 16-cores, or rather eight dual-cores.

3:52

Only the M5 Ultra gets  double the NPU, because it uses two CPU chiplets.

4:00

To the right of the M-cores are four pretty  massive Display Engines and the bottom shoreline of the die is used for a large array  of IO.

4:04

From left to right we can find an Apple proprietary high-speed PHY, used for cameras  and other sensors, a 4-lane Display PHY, and 16-lanes of PCI-Express, most likely  gen 5, arranged in four 2 by 2 clusters.

4:22

The CPU chiplet also comes with four massive  Thunderbolt 5 PHYs.

4:22

They are so big; they actually outclass the M-cores in size and aren’t  that far behind an S-core.

4:29

It’s crazy to see how big analogue SerDes are becoming and how much  space the command-and-control area is taking up.

4:40

And last, but not least, the CPU  die also comes with an on-die NAND controller and AES engine in the secure enclave.

4:44

But there’s one thing missing.

4:44

And arguably, it’s the most important aspect about Apple’s  new chiplet architecture.

4:50

We can find it on the upper shoreline of the die.

4:56

When we zoom in,  we see a long row of what appears to be some kind of lanes or PHY, but there isn’t much  to it, besides a faint striped structure.

5:07

What you are looking at is the physical layer of  the new die-to-die interconnect.

5:07

And it’s unlike any other interconnect we have seen so far.

5:12

Or  rather, it was never used in that specific way.

5:18

But before we check out how exactly it works,  let’s take a quick look at the two GPU dies first.

5:23

The GPU chiplets are both interesting and a bit  boring at the same time.

5:23

Produced in the same N3P process node, the large GPU die clocks in at  around 300 square millimeters and the small one at about 157 square millimeters.

5:35

Depending on how  you look at it, the large GPU chiplet is almost twice the size of the small one, or the small  one is almost half the size of the large one.

5:46

And this relation continues once we look inside  the silicon.

5:46

The large GPU chiplet comes with 40 GPU cores, 48 megabytes of System Level  Cache and a wide 512-bit memory interface, while the small GPU chiplet contains  exactly half of that: 20 GPU cores, 24 megabytes of SLC and a 256-bit interface.

6:04

Only  some of the media acceleration logic close to the die-to-die interface stays the same.

6:11

So, the small GPU die is basically a cut in half large GPU die.

6:16

I guess  that helps with development time.

6:21

But how is Apple connecting CPU and  GPU dies?

6:21

And how do they manage the massive M5 Ultra? I’m so glad you asked!

6:26

Let’s go back to our CPU chiplet and the weird PHY we found on the upper shoreline of  the die.

6:32

When we place the CPU die and any of the two GPU dies next to each other, like they  are placed on the package, we can see that the GPU dies contain the same kind of PHY on the  adjacent shoreline. Two matching structures.

6:49

But Apple has fused two dies together  before, so what’s new with the M5 series?

6:55

What’s new is that Apple is using TSMCs latest  SoIC-mH, also called SoIC-X F2F for Face-2-Face.

7:03

It’s TSMCs term for hybrid bonding, the same  technology used for AMD’s 3D V-Cache for example.

7:09

But wait a second, 3D V-Cache is 3D stacking  a cache die on top, or below, a compute die, and so far, hybrid bonding describes bonding  two wafers, or chips, directly to each other, creating a chip sandwich.

7:22

But when we look at  the package, CPU and GPU chiplets are placed next to each other, not on top of each other.

7:28

That’s where the “mH” part of “SoIC-mH” comes in.

7:34

It’s short for “molding horizontally”  and it does exactly what it says.

7:46

The way it works is that the chiplets are bonded  onto a special hybrid bonding silicon interposer.

7:52

At first glance not that different to other  silicon interposers and packaging methods, but hybrid bonding has one major advantage.

7:57

Standard interconnects, even advanced packaging, is usually microbump based.

8:02

That means if you  want to connect two chips via an interposer, you manufacture both chips, build out the  metal layers, and then attach the finished die to the interposer via microbumps.

8:13

Tiny  little drops of solder that complete the connection between chiplet and interposer.

8:19

And even though they are called “microbumps”, you can’t make them as small as you want,  there’s a size limit.

8:24

And every bit of data that’s flowing between the chiplets and the  interposer has to pass through these microbumps.

8:34

Hybrid bonding gets rid of microbumps  altogether, no solder is used.

8:34

Instead, the chiplets and the silicon interposer  have a so-called “hybrid bonding layer”, which is basically matching exposed copper  interconnects surrounded by a dielectric material.

8:49

These interconnects are then perfectly lined up,  which is way harder than it sounds, and then both the chiplets and interposer are heated.

8:55

The heat  expands the copper, directly fusing the copper from the chiplets to the copper of the interposer.

9:03

The result is the highest quality interconnect you can think of, it’s almost like fusing the  metal layers, creating one large coherent chip.

9:13

Because you can control copper better than solder,  you can create smaller interconnects, which means you can put more interconnects in the same area.

9:18

Microbumps are currently in the 25-micrometer range, maybe going down to 18 micrometers,  while hybrid bonding allows for a pitch of less than 10 micrometers and even approaching  3 micrometers for the most advanced process.

9:35

The result is an extremely high-bandwidth,  low-latency and energy efficient interconnect.

9:41

And because SoIC-mH allows Apple to place CPU  and GPU chiplets next to each other, instead on top of each other, there are no thermal problems.

9:48

BTW: SoIC-mH is a supply chain term, while TSMC calls it SoIC-X F2F, or Face-2-Face.

9:55

Both terms describe the same process.

10:01

So, the M5 Pro is CPU + small GPU  chiplet, the M5 Max is CPU + large GPU chiplet.

10:07

But how does the M5 Ultra work?

10:07

And here we are running into a problem, because the lab didn’t get the M5 Ultra yet.

10:13

But we can  figure that out by ourselves, all we have to do is to take a closer look at the large GPU die of  the M5 Max.

10:18

On the opposite side of the CPU to GPU interconnect, basically taking up the entire  upper shoreline of the large GPU die, we can find another SoIC-mH interface.

10:31

Only this time, it’s  even larger than the one that connects CPU to GPU.

10:38

The structures are identical, which is  already a very good indication that Apple is alos using hybrid bonding for the M5 Ultra.

10:42

But the final clue is Apple’s own marketing.

10:48

According to Apple, the M5 Ultra uses  “UltraFusion” to connect two dual-die M5 Max chips.

10:54

That much we already know.

10:54

But Apple  also claims that this version of UltraFusion increases the connection density by over 6x.

11:00

And  that would be impossible without hybrid bonding.

11:07

The last UltraFusion chip, the M3 Ultra,  used InFO-LSI packaging with a local silicon interconnect and traditional solder-based  microbumps.

11:13

Apple never publicly confirmed the bump pitch of their Ultra chips, but  measurements show a 25-micrometer pitch.

11:23

And a 6x density increase means the new pitch has  to be about 10 micrometers or less, a feature size that’s well into hybrid bonding territory.

11:30

SoIC-mH follows a recent trend for Apple, showing an increased concern for thermals,  like the A20 Pro switched to WMCM packaging, moving the DRAM to the side of the SoC.

11:42

Like other TSMC customers, Apple had access to hybrid bonding for a while now,  but choose to wait for the side-by-side implementation.

11:52

It’s not unlikely that Apple  actively worked on this solution with TSMC.

11:57

Hybrid bonding further elevates the M5  series.

11:57

A higher interconnect density and direct copper-to-copper contacts reduce the  amount of energy required to transfer data and increases the bandwidth potential.

12:08

Not only did  we get the new S-cores with a private slice of L2 cache and new M-cores, but also a brand-new  packaging technology that offers real benefits over older, solder-based solutions.

12:19

Can’t innovate anymore my ass.