Scalable architecture to tackle memory bottleneck in many-core computers (2020-ongoing, Chapman)
The emergence of data-intensive applications, such as Deep Neural Networks (DNN), exacerbates the well-known memory bottleneck in computer systems and demands early attention in the design flow. Electronic System-Level (ESL) design using SystemC Transaction Level Modeling (TLM) enables effective performance estimation, design space exploration, and gradual refinement. However, memory contention is often only detectable after detailed TLM-2.0 approximately-timed or cycle-accurate RTL models are developed. A memory bottleneck detected at such late stage can severely limit the available design choices or even require costly redesign.
In this work, we propose a novel TLM-2.0 loosely-timed contention-aware (LT-CA) modeling style that offers high-speed simulation close to traditional loosely-timed (LT) models, yet shows the same accuracy for memory contention as low-level approximately-timed (AT) models. Thus, our proposed LT-CA modeling breaks the speed/accuracy tradeoff between regular LT and AT models and offers fast and accurate observation and visualization of memory contention.
Our extensible SystemC model generator automatically produces desired TLM-1 and TLM-2.0 models from a DNN architecture description for design space exploration focusing on memory contention. We demonstrate our approach with a real-world industry-strength DNN application, GoogLeNet. The experimental results show that the proposed LT-CA modeling is 46x faster in simulation than equivalent AT models with an average error of less than 1% in simulated time. Early detection of memory contentions also suggests that local memories close to computing cores can eliminate memory contention in such applications.
Project Publication:
The emergence of data-intensive applications, such as Deep Neural Networks (DNN), exacerbates the well-known memory bottleneck in computer systems and demands early attention in the design flow. Electronic System-Level (ESL) design using SystemC Transaction Level Modeling (TLM) enables effective performance estimation, design space exploration, and gradual refinement. However, memory contention is often only detectable after detailed TLM-2.0 approximately-timed or cycle-accurate RTL models are developed. A memory bottleneck detected at such late stage can severely limit the available design choices or even require costly redesign.
In this work, we propose a novel TLM-2.0 loosely-timed contention-aware (LT-CA) modeling style that offers high-speed simulation close to traditional loosely-timed (LT) models, yet shows the same accuracy for memory contention as low-level approximately-timed (AT) models. Thus, our proposed LT-CA modeling breaks the speed/accuracy tradeoff between regular LT and AT models and offers fast and accurate observation and visualization of memory contention.
Our extensible SystemC model generator automatically produces desired TLM-1 and TLM-2.0 models from a DNN architecture description for design space exploration focusing on memory contention. We demonstrate our approach with a real-world industry-strength DNN application, GoogLeNet. The experimental results show that the proposed LT-CA modeling is 46x faster in simulation than equivalent AT models with an average error of less than 1% in simulated time. Early detection of memory contentions also suggests that local memories close to computing cores can eliminate memory contention in such applications.
Project Publication:
- E. Arasteh, R. Dömer
Fast Loosely-Timed Deep Neural Network Models with Accurate Memory Contention
ACM Transactions on Embedded Computing Systems (TECS), vol. 23, issue 5, article 75, 32 pages, August 2024.
- E. Arasteh, V. Govindasamy, R. Dömer
BusyMap, an Efficient Data Structure to Observe Interconnect Contention in SystemC TLM-2.0
Proceedings of Design, Automation and Test in Europe (DATE) Conference, Valencia, Spain, March 2024.
- C. Raccomandato, E. Arasteh, R. Dömer
MapGL: Interactive Application Mapping and Profiling on a Grid of Processing Cells
Proceedings (research track) of the Design and Verification Conference in Europe, Munich, Germany, November 2023.
- C. Raccomandato, E. Arasteh, R. Dömer
Grid-based Mapping and Analysis of a GoogLeNet CNN using MapGL Editor
Proceedings (engineering track) of the Design and Verification Conference in Europe, Munich, Germany, November 2023.
- N. Farzan, E. Arasteh
Visualizing Transaction-Level Modeling Simulations of Deep Neural Networks
Fowler School of Engineering, Technical Report 23-01, August 2023.
- V. Govindasamy, E. Arasteh, R. Dömer
Minimizing Memory Contention in an APNG Encoder using a Grid of Processing Cells
Accepted for presentation at the International Embedded Systems Symposium, Springer, Lippstadt, Germany, November 2022
Completed Projects
- Explore parallelism in system level models by assessing PDES performance (2018-2020, UCI)
The emergence of data-intensive applications, such as Deep Neural Networks (DNN) demands early attention to finding parallelism opportunities for effective embedded system design and modeling. Electronic System-Level (ESL) design using SystemC Transaction Level Modeling (TLM) enables explicit modeling of parallelism early in the design flow. However, the choice of synchronization and communication mechanisms between concurrent modules has a significant impact on the available parallelism in the TLM models.
In this project, we propose and analyze a set of non-invasive standard-compliant modeling techniques to increase parallelism in IEEE SystemC TLM-1 and TLM-2.0 models. As shown in left figure below, we illustrate simulator parallelism, model parallelism and simulation speedup in a 3-dimensional space. As the red arrow indicates, both higher model parallelism and simulator parallelism achieve the maximum simulation speedup. Moreover, by increasing model parallelism opportunities in one dimension, the simulator can better leverage its parallelization capabilities for the maximum simulation speedup.
To demonstrate the effectiveness of our approach, we measure the performance of aggressive out-of-order parallel discrete event simulation (PDES) in the Recoding Infrastructure for SystemC (RISC) and analyze the parallelism in the models. In particular, we demonstrate the impact of varying synchronization mechanisms on the exposed parallelism using six modeling styles of a state-of-art DNN, GoogLeNet (middle figure). We further have quantified the improved parallelism in the improved SystemC TLM-1 and TLM-2.0 models by measuring the performance of aggressive out-of-order parallel simulation in RISC. The right figure shows top-level test benches for TLM-1 (a) and TLM-2.0 (b) that stimulus and monitor modules are connected using FIFO in case of TLM-1 and sockets to shared memory inside DUT in case of TLM-2.0.
The results support four hypothese (H1) less restrictive transaction types enable higher parallelism, (H2) abstract TLM-1 models carry less workload than memory accurate TLM-2.0 model (H3) higher speed in aggressive parallel simulation is a significant indicator of higher level of parallelism in design and (H4) improved RISC scheduling algorithms show higher simulation speedup.
Project Publications:
Simulator parallelism, model parallelism
and simulation speedup forms a 3-dimensional space
GoogLeNet network
Top-level TLM tech bench for DNN (a) TLM-1 (b) TLM-2.0
- E. Arasteh, R. Dömer
An Untimed SystemC Model of GoogLeNet
in "Analysis, Estimations, and Applications of Embedded Systems"
by M. Wehrmeister, M. Kreutz, M. Götz, S. Henkler, A. Pimentel, and A. Rettberg,
reprint of best papers at IESS 2019, Springer, February 2023. (ISBN: 978-3-031-26499-3)
- E. Arasteh, R. Dömer
Improving Parallelism in System Level Models by Assessing PDES Performance
Proceedings of Forum on Specification and Design Languages, Antibes, France, September 2021.
- E. Arasteh, R. Dömer
Systematic Evaluation of Six Models of GoogLeNet using PDES
Center for Embedded and Cyber-Physical Systems, Technical Report 21-03, September 2021.
- Z. Cheng, E. Arasteh, R. Dömer
Event Delivery using Prediction for Faster Parallel SystemC Simulation
Asia and South Pacific Design Automation Conference, Beijing, China, January 2020.
- D. Mendoza, Z. Cheng, E. Arasteh, R. Dömer
Lazy Event Prediction using Defining Trees and Schedule Bypass for Out-of-Order PDES
Design, Automation and Test in Europe (DATE) Conference, Grenoble, France, March 2020.
- R. Dömer, Z. Cheng, D. Mendoza, E. Arasteh
Pushing the Limits of Parallel Discrete Event Simulation for SystemC
A Journey of Embedded and Cyber-Physical Systems, Springer Nature, Switzerland, August 2020.
- E. Arasteh, R. Dömer
Untimed SystemC Model of GoogLeNet
Proceedings of the International Embedded Systems Symposium, Springer, Friedrichshafen, Germany, September 2019.
Controller architecture of enterprise solid state drive (Summer 2021, Samsung)
Design SoC and firmware architecture of enterprise SSD controller.
Design and evaluate resource allocation strategies to guarantee SoC performance in a multi-tenant cloud.
Design system-level contention models of a modern flash storage device.
ASIC design of speech recognition processor (Summer 2019, Syntiant)
High-level synthesis (HLS) design of cryptography block.
Evaluate performance of HLS RTL implementation to optimize speed and area.
Design system-level test bench of the next generation Neural Decision Processor (NDP).
ASIC design and verification of network video camera chip (2015-2018, Canon Inc. - Axis Communications)
Design virtual prototype for ARTPEC-7 .
Develop infrastructure for block, subsystem, SoC verification using SystemVerilog and UVM.
Formal verification to measure performance.
DSP software design for multi-standard 2G/3G/4G cellular modem (2012-2014, Ericsson Modem)
Design real-time OS and device driver for Embedded Vector Processor (EVP)
Develop low-level software to verify subsystems inside digital baseband including DSP, security and power management.
Troubleshoot system errors raised by either EVP software or hardware.
Software design for airbag bike helmet (2011-2012, Hövding)
Implement a secure in-field and in-production firmware upgrade to protect Hövding proprietary algorithm.
Design secure boot and customized bootloader.
ASIC design and verification of Video Display Controller (VDC), (2010-2011, ARM)
Design and synthesis of AMBA AXI VDC supporting paged memory with high average and peak latencies in 65nm CMOS process
Perfrom requirement analysis, RTL development, functional verification, synthesis for 65nm CMOS process and FPGA validation in hardware.
Develop Linux device driver, C-based user interface and MATLAB test scripts in software.
Project Publication:
- E. Arasteh
AMBA AXI Video Display Controller supporting paged memory
Master Thesis, Lund University, Sweden, February 2011.
- E. Arasteh, R. Dömer