MemCon: Memory for the Next Five Years

This year's MemCon keynotes were given by Hugh Durdan, VP of the IP Group at Cadence, on Diverging Products for Diverging Needs: Where Will the Memory Industry Be in Five Years, and by Steve Pwalowski, VP of Advanced Computing Solutions at Micron, on How Market Forces Are Driving Us to New Memory-Centric Architectures.

Diverging Products for Diverging Needs: Where Will the Memory Industry Be in Five Years?

Hugh Durdan started with a snapshot of where the industry is today: of the $347B semiconductor industry, $70B is memory, made up of $42B of DRAM and $28B of flash. On top of that, there is memory embedded in SoCs that is counted as logic.

The three big meta drivers of the industry are business, technology, and end markets. The semiconductor market is maturing. It is still growing, but the growth rate is slowing. Also, the cost of designing an SoC are increasing. There is roughly a 4X increase in design costs from 28nm to 16/14nm.

The result of this has been a lot of consolidation in the semiconductor industry. The number of acquisitions was not especially high in 2015, but the size of them was. The above graph doesn’t even include the rumored acquisition of NXP by Qualcomm, just the NXP/Freescale merger.

There are two other big industry trends. The first is that increasingly NAND-flash-based solid-state drives (SSD) are displacing magnetic media. The second is that memory needs are increasingly driven by mobile, not the PC market. Since mobile has different requirements from PC, this is a first level of fragmentation of the DRAM market, with a second level driven by the special requirements of datacenters.

The key market segments for semiconductor in general, but also for memory, are mobile (still), enterprise (especially cloud-based datacenters), automotive (especially ADAS and autonomy), and IoT.

Mobile is focused on increasing performance but also lowering power. It is the market that drives the LPDDRx, today LPDDR4, with LPDDR5 coming, and LPDDR6 presumably coming at the end of the decade.

Enterprise has two separate vectors driving memory. First, it is the market driving the DDRx interface roadamap, with DDR4 today, DDR5 coming, and presumably DDR6 early in the next decade. But DDR is not enough for the most demanding applications. Two other technologies, high-bandwidth memory (HBM) and hybrid memory cube (HMC) are also important. They are expensive but for some applications that extra cost is justified.

Another major trend is the adoption of flash in the memory hierarchy, not just SSDs replacing hard-disk drives, but adding a non-volatile component to the memory hierarchy. I covered this topic in more detail last week in MemCon 2016: Storage Class Memory.

Automotive is going to be an increasingly important market for memory as ADAS and autonomous driving continue to grow in importance, along with the growing semiconductor content of vehicles. The big difference between automotive and other markets is the need for extended temperature range (125C), longer lifetime (10-20 years), and high reliabiliy (<1ppm/year and the need for ECC). Oh, and it still costs only $3.

IoT remains a market that is hard to predict. From a memory point of view, most of the market is expected to be embedded, either in microcontrollers or in specialized SoCs for particular market segments. But how many devices will ship and at what price?

In summary:

• Slowing growth and increased costs are driving consolidation
• Mobile growth is slowing but still huge
• Automotive, enterprise and IoT are growing faster than the overall market
• The one-size-fits-all market (driven by the PC) is evolving into many types of specialty memory

How Market Forces Are Driving Us to New Memory-Centric Architectures

Steve Pawlowski's key opening point was that traditional architectures (CPU, GPU, ASIC) are limited in performance and power by data movement, by the ability to feed memory.

Steve used the highest performance supercomputers in the world as examples. The top ones are in China. The #1 is Sunway TaihuLight, which has over 10M cores and consumes over 15GW of power. It uses a local Chinese architecture. Peak performance is 125,000 TFlop/s. The #2 machine, also in China, uses Intel architecture and consumes nearly 18GW. Its peak performance is 55 TFLop/s. So the #2 machines consumes more power than the #1 machine but delivers less than half the performance.

But when you run the HPCG benchmark, which is much more representative of a real-world workload, its performance drops.and it can only deliver 0.3% of that headline peak performance. It turns out that local memory performance is much more important in the real world unless you are just going for a "muscle" machine. Even the machine that performs the best is only delivering 5.3% of its headline number. The reason that there is a memory bottleneck is that the HPCG benchmark focuses on workloads where memory bandwidth per Flop is greater than 1.

To improve this, we need better memory/processor architecture. Energy use needs to be at least 10X better than today's systems at very high bandwidth (>1TB/s). Even if we get the average cost of a data access/movement to 1 pJ/bit, memory power is about 8W at 1TB/s (and 8MW for 1EB/s). We need to move compute to where the data is, instead of today's architecture with a separate processor and memories, with power-hungry DDR type interfaces. Micron's hybrid memory cube (HMC) is one move in this direction, but there are new innovative interfaces being developed, and new consortia working on them.

The one-line summary: Energy of data movement dominates system power, so heterogeneous architectures with innovative packaging is where the real innovation will occur over the next few years.

Next: Video Cameras: No Service for You

Previous: How Virtualization Is Changing Networking