In high-performance computing systems, particularly AI servers, the capacity and bandwidth of memory (DRAM) are becoming increasingly important as processors need to handle massive amounts of data, and traditional DRAM can no longer meet the demand. Currently, High Bandwidth Memory (HBM) is the hot topic.
Compared to traditional DRAM, the manufacturing of HBM is much more complex, as it requires stacking multiple DRAM chips, which necessitates the use of more advanced packaging technologies.
With technological advancements and changes in market demand, the density of HBM stacks is also increasing. Some organizations have estimated that, based on the current trend, the capacity will increase from 16GB in 2022 to 48GB in 2027. Micron, a major DRAM manufacturer, is even more optimistic, predicting that 64GB HBMNext (HBM4) will emerge in 2026, with a stack of 16 layers. This would allow the construction of a 64GB HBM module using 16 32Gb DRAM chips, requiring memory manufacturers to further reduce the spacing between DRAM chips and to adopt new production technologies, especially better packaging techniques.
Typically, HBM stacks use Through-Silicon Vias (TSVs) to vertically connect multiple DRAM chips. This TSV-equipped stacked architecture allows for a very wide memory interface (1024 bits), memory capacities of up to 36GB or 64GB, and bandwidth exceeding 1TB/s.
Producing HBM stacked chips is much more complex than producing traditional DRAM. First, the DRAM chips used for HBM are completely different from typical DRAM (such as DDR4, DDR5). Memory manufacturers must produce a sufficient number of DRAM chips, test them, and then encapsulate them on top of pre-tested high-speed logic chip layers, followed by testing the entire package. This process is both expensive and time-consuming.
Taking the latest mass-produced HBM3E as an example, its chip size is about twice that of equivalent-capacity DDR5. In addition to the logic and DRAM layers, an interface layer is also required. Such a complex packaging stack can affect the yield rate. Therefore, as HBM develops and the number of stacked layers increases, the packaging complexity also increases, making the manufacturing more difficult and the yield rate hard to improve.
01
3D DRAM Takes the Baton
HBM is not the ultimate form of memory for high-performance computing systems. Looking at the research and development direction of major memory manufacturers, before the complete resolution of the "memory wall" problem through the integration of storage and computing, and the maturity and mass production of related chip technologies, 3D DRAM will be the successor to HBM.Traditional DRAM requires a complex process for reading and writing data, while 3D DRAM can directly access and write data through vertically stacked memory cells, significantly improving access speed. The advantages of 3D DRAM include not only high capacity and fast data access but also low power consumption and high reliability, which can meet various application needs.
Here is a brief introduction to the basic structure of DRAM.
The DRAM cell circuit consists of a transistor and a capacitor. The transistor is responsible for transmitting current, allowing information (bits) to be written or read, while the capacitor is used to store bits.
DRAM is composed of conductive materials called "bit lines (BL)", which provide carriers (current) injected into the transistor. The transistor acts like a gate that can be opened (connected) or closed (disconnected) to maintain or stop the flow of current within the device. This gate state is defined by the voltage bias applied to the contact conductive structure called "word line (WL)". If the transistor is conductive, the current will flow through the transistor to the capacitor and be stored in the capacitor.
The capacitor needs to have a high aspect ratio, which means its height is much greater than its width. In some early DRAMs, the active area of the capacitor was embedded in the silicon substrate, while in the latest generations of DRAMs, the capacitor is processed on top of the transistor.
3D DRAM is a new type of memory chip with a completely new structure, stacking DRAM cells vertically, breaking the original pattern. It is somewhat similar to the mature 3D NAND unit vertical stacking, but the manufacturing difficulty is greater than that of 3D NAND. 3D DRAM is not simply stacking 2D DRAM components together, nor is it the same as HBM. It requires redesigning the DRAM architecture and using some advanced transistor manufacturing technologies and advanced packaging technologies.
The focus of 3D DRAM design is to solve the challenges of process node scaling and multi-layer stacking, as well as the scaling of capacitors and transistors, as well as the interconnection between cells and via arrays, and to establish corresponding process specifications. Through vertical stacking, 3D DRAM chips increase the capacity per unit area by three times. 3D DRAM is different from HBM in terms of design and manufacturing.
According to The Elec, Samsung and SK Hynix have both identified hybrid bonding as the key packaging technology for manufacturing 3D DRAM in the future. It is reported that Samsung plans to launch 3D DRAM chips in 2025, while SK Hynix has not yet determined a specific time. Currently, Samsung and SK Hynix use micro-bumps to connect DRAM modules. Hybrid bonding technology can eliminate the need for micro-bumps by using through-silicon vias to vertically stack chips, thereby significantly reducing chip thickness.Manufacturing 3D DRAM involves addressing several challenges. To advance the miniaturization of DRAM processes, it is necessary to lay 2D DRAM components on their sides and stack them, but this presents a few difficulties: lateral etching is required in the horizontal direction, but due to the significant differences in trench sizes, lateral etching is very challenging; different materials must be used in the stack etching and filling processes, which complicates manufacturing; and there are integration issues when connecting different 3D components.
In the production of 3D DRAM, the length of the capacitor (Cap) must be shortened (the length of the capacitor cannot be the same as its height) and stacked to increase the number of storage cells per unit area.
The structure represented in the above diagram remains unchanged, and when rotated 90 degrees clockwise, the structure will be viewed from the top down. In this orientation, nano-sheets can be stacked. However, in this case, the area shown in the original design is very dense, so the bit lines and capacitors need to be processed from top to bottom, and they are very close together. To achieve 3D stacking in this direction, the architecture must be redesigned.
In addition to designing a new architecture, the metallization and connectivity of 3D DRAM must also be changed. New methods need to be designed to facilitate the flow of current through the central bit line stack, including connecting horizontal MIM (Metal-Insulator-Metal) capacitor arrays on each layer, and wrapping the gate around the transistor (gate full wrap). The principle is that when the current flows, only the target bit line (layer) is activated, and in the activated layer, the current can connect to the correct transistor.
There is also the issue of the through-silicon via array. To avoid the limitations of the step structure used in 3D NAND, a via array structure that penetrates the silicon stack layer and can stop at a specific layer (one via per layer) needs to be introduced, placing contact points inside the storage cells. After the trench is made, an isolation layer that only exists on the side walls can be introduced.
High trenches are used to introduce the etching medium to remove silicon, and then conductive metal is introduced into the empty trenches. As a result, each square at the top (the light green and purple boxes in the last three images below) is only connected to the layer below.
In terms of process technology, unique and innovative processes are required. 3D DRAM is a cutting-edge design, and to achieve mass production, the processes and designs used are unprecedented or untested.3D Packaging Support
The above introduction covers the challenges and solutions in chip design and manufacturing processes for 3D DRAM. After the corresponding wafers are manufactured, more suitable and advanced packaging technologies are required to organically combine these DRAM wafers with logic and other functional parts, in order to maximize application performance.
The more advanced packaging is needed, the smaller the packaged wafers are, and the higher the overall complexity when they are packaged together. 3D DRAM fully covers these two points. Advanced packaging includes 2.5D and 3D packaging. 2.5D is difficult to meet the packaging requirements of 3D DRAM, and 3D packaging with vertical stacking of ultra-small building blocks (DRAM wafers) and interconnection through through-silicon vias (TSVs) must be adopted.
In 2.5D packaging, logic units, memory, or other types of chips are horizontally stacked on a silicon interposer using a flip-chip method, with microbumps connecting the electronic signals of different chips. Through-silicon vias in the interposer connect to the metal bumps below, and then the chips are packaged onto the IC substrate, establishing tighter interconnections between the chips and the substrate. From the side, although the chips are stacked, they are essentially still horizontally packaged (traditional chip packaging is horizontal). However, compared with traditional packaging, the size and spacing of the wafers in 2.5D packaging are much smaller, approaching 3D packaging.
3D packaging stacks multiple wafers (face down) together, directly using through-silicon vias for vertical stacking, connecting the electronic signals of different wafers above and below, and achieving true vertical packaging. Currently, more and more CPUs, GPUs, and memory are beginning to adopt 3D packaging technology.
At the 3D packaging stage, hybrid bonding technology is almost a must-have.
Hybrid bonding is one of the chip bonding technologies used in chip packaging processes, and the commonly used commercial technology is "Cu-Cu hybrid bonding." With Cu-Cu hybrid bonding, metal contact points are embedded in dielectric materials, and through a thermal treatment process, these two materials are combined, using the atomic diffusion of solid-state copper metal to achieve bonding. This method solves the challenges encountered in the previous flip-chip bonding process.
Hybrid bonding is not the only advanced packaging technology, but it provides the highest density of vertical stacking. The volume occupied by the microbumps in the packaging makes the stack too high to fit into a package with a GPU or CPU. Hybrid bonding not only reduces the height of the DRAM wafer but also makes it easier to remove excess heat from the packaging, as the thermal resistance between the layers of this packaging is smaller.
Compared with flip-chip bonding, hybrid bonding has many advantages. It allows for the realization of ultra-high I/O counts and longer interconnection lengths, eliminating the filling cost by using dielectric materials instead of bottom fillers for bonding. In addition, compared with chip bonding on the wafer, hybrid bonding has the thinnest thickness, which is particularly friendly for 3D DRAM packaging that requires stacking multiple layers of chips, as hybrid bonding can significantly reduce the overall thickness.Progress in 3D DRAM Manufacturing by the Three Major Manufacturers
Currently, the three major memory chip manufacturers, Samsung, SK Hynix, and Micron, are all researching and developing 3D DRAM, with corresponding manufacturing processes and packaging technologies being developed simultaneously.
Micron has been researching 3D DRAM since 2019 and has more than 30 patents related to 3D DRAM, which is 2 to 3 times the number of patents obtained by Samsung and SK Hynix.
In recent years, Samsung has been continuously researching 3D DRAM and has introduced the industry's first 12-layer 3D-TSV technology.
In 2023, at the "VLSI Symposium" held in Japan, Samsung Electronics published a paper containing research results on 3D DRAM and displayed images of the internal structure of 3D DRAM chips.
According to sources, in May 2023, Samsung Electronics established a development team within its semiconductor research center to mass-produce 4F2 structure DRAM. As the size of DRAM cells has reached its limit, Samsung wants to apply 4F2 to 10nm-level processes or more advanced DRAM processes. If Samsung's 4F2 DRAM storage cell structure research is successful, the die area can be reduced by about 30% compared to the existing 6F2 DRAM storage cells without changing the process.
It is reported that Samsung has stacked 3D DRAM up to 16 layers.
SK Hynix is developing IGZO channel materials for future DRAM, which can improve the refresh characteristics of DRAM. It is reported that IGZO thin-film transistors have been applied in the display panel industry for a long time due to their moderate carrier mobility, extremely low leakage current, and scalability of substrate size. It can become a candidate for future DRAM stackable channel materials.
Recently, at the VLSI 2024 summit held in Hawaii, SK Hynix released the latest research results on 3D DRAM, and the yield of its 5-layer stacked 3D DRAM has reached 56.1%. In addition, SK Hynix's experimental 3D DRAM has shown performance comparable to 2D DRAM, but a large amount of technical verification and optimization work is still needed before commercialization can be achieved.Conclusion
As a commodity in the chip industry, DRAM has always had a huge market share. Now, driven by the demand for high-performance computing, various new memory technologies and products have emerged in succession, adding more highlights to this already lively market.
With the development of AI servers, High Bandwidth Memory (HBM) has quickly become popular, and the manufacturing and packaging of related chips are hot topics in the industry at present. As applications develop and technology levels improve, 3D DRAM is likely to replace the current industry status of HBM in the next few years. Therefore, chip manufacturers and semiconductor equipment manufacturers are investing more and more resources in research and development, continuously building momentum.
In terms of chip manufacturing and packaging, 3D DRAM still needs to overcome challenges and it will take some time before mass production can be achieved. In this regard, SK Hynix points out that although 3D DRAM has great development potential, a lot of work still needs to be done before it can be commercialized. Currently, the performance characteristics of 3D DRAM are still unstable, and it needs to reach 32 to 192 layers of stacked memory cells to be widely used.
Comment