A different direction on chips

Chiplets, a perennial favorite of our very own Shahin Farshchi, are in the limelight. As he puts it: “Chiplets are here, and they will become ubiquitous.”

Today, a single chip is often a combination of multiple processing systems manufactured together (what’s dubbed “System on a Chip” or SoC). If you’ve seen the Apple M1, then this is familiar: you have separate cores for processing, ML and GPUs all packed together onto one piece of silicon.

That model has great economics, but there is a clear downside to this all-in-one approach. Fabs are overwhelmed with demand right now, putting an extremely high premium on nodes with the smallest sizes such as TSMC’s 5nm and coming 3nm process. Machine learning cores need to run at the highest performance, but the same is not true for, say, memory or wireless modem systems. Right now, whichever system requires the smallest node constrains the selection of a fab for manufacturing.

Chiplets unbundle the systems on a chip. Each system can be manufactured on its own silicon (perhaps at a larger node with cheaper economics) and packaged together later. What gets more interesting is that engineers are stacking these chips in the Z-axis for much greater performance while using significantly less power. By working in three dimensions rather than two, the distance — and thus latency — between systems can be significantly reduced.

Once experimental, these technologies are hitting early scale, but challenges linger. AMD and Intel (via its Foveros technology) have incompatible standards and approaches, which is slowing progress, plus the fabs aren’t entirely ready for this new paradigm.

Nonetheless, there’s an opportunity here for startups to come in and build something just as this evolution in semiconductors crescendoes. One key area that Shahin has been interested in is technology that makes the multi-chip packages that connect chiplets together into a single unit more efficient and less expensive. Another opportunity is to build a “chiplet-native” startup that uses the enhanced performance of this chip design to outcompete on certain workloads.

Chiplets and 3D stacking are getting underway, but they will be definitive in the coming years and represent one of the largest macro trends coming up in semiconductors.