임베디드 시스템온칩 연구실

A Heterogeneous Computing System combines multiple processing units like CPUs, GPUs, and Processing-in-Memory (PIM) devices, often utilizing High Bandwidth Memory (HBM-PIM) or GDDR-PIM. Modern workload, particularly large models like transformer-based LLMs, face significant data transfer bottlenecks due to extensive data movement. By intelligently analyzing the dataflow, a heterogeneous system can assign different computational tasks to the most appropriate device (CPU, GPU, or PIM). This strategy minimizes data transfers across slower external bandwidth, allowing computations to happen closer to the data (especially on HBM-PIM), thereby effectively alleviating the memory bandwidth bottleneck and improving performance and efficiency for memory-bound operations. The extensive movement of data between the CPU and memory creates a performance wall. To overcome this, Near Memory Processing (NMP) and Processing in Memory (PIM) are gaining significant attention. This approach involves executing certain operations close to the DRAM, utilizing a dedicated NMP Core situated near the memory controller and DRAM banks, instead of solely relying on the CPU for all computations. This proximity drastically reduces data transfer latency and energy. Transformer architecture has become the foundation for many state-of-the-art models in NLP and computer vision tasks. While it demonstrates high performance by capturing long-range dependencies through the attention mechanism, its quadratic complexity and large number of parameters pose challenges for deployment in various environments. GPUs are commonly used to support inference for such Transformer-based architectures, but they suffer from issues such as kernel launch overhead, low utilization, high power consumption, and high cost. Theref