9:00 - 9:10

Introduction

​

Neuromorphic Cross-stack Co-design for Efficient Intelligent Computing 

ROOM Granados II

session chairs

Melika Payvand (University of Zurich and ETH Zurich, CH)

Jae-sun Seo (Cornell Tech, US)

 

9:10-09:37

From Attention to KANs: Energy-Efficient In-Memory Computing for Next-Gen AI

John Paul Strachan (RWTH Aachen University, DE)

​This talk explores how in-memory computing (IMC) architectures can remove key performance and energy bottlenecks in modern deep learning—especially in transformer-based large language models (LLMs) and emerging architectures for scientific computing. First, we introduce an analog IMC self-attention mechanism built on charge-based “gain-cell” memories that store token projections in-place and execute dot-products in parallel within memory, substantially reducing the data-movement overhead inherent to attention with KV caching. However, analog gain-cell circuits impose non-idealities preventing direct use of pre-trained models, and thus we developed a light-weight initialization strategy that recovers GPT-2-level performance without training from scratch.

We then broaden to work addressing other major bottlenecks—including softmax operations, where precision demands and computational structure become increasingly problematic as sequence lengths scale. We present an IMC-optimized reformulation that shifts complexity away from sequence length, fuses softmax sub-operations with adjacent VMMs, and supports both efficient digital and spike-encoded analog implementations with negligible accuracy loss. Finally, we extend IMC beyond transformers to Kolmogorov–Arnold Networks (KANs), which trade linear layers for trainable nonlinear functions and can substantially reduce model size for scientific workloads. We present a flexible, cross-layer-optimized IMC accelerator that computes arbitrary nonlinear functions efficiently using read-optimized memristive arrays and a single-read evaluation scheme, delivering large improvements in energy and energy–delay compared to CPUs and prior accelerators.

 

​09:38 - 10:05

In-Memory Computing Architectures for AI at the Edge

Irem Boybat (IBM Zurich, CH)

The growing computational demands of AI inference are pushing edge platforms beyond the limits of conventional architectures, where data movement between memory and processing units dominates energy consumption. In-memory computing (IMC) offers a compelling path forward by performing computation directly within memory arrays, enabling significant gains in energy efficiency.

This talk presents a cross-stack perspective on heterogeneous IMC-based architectures for edge AI acceleration. At the system level, we present an architecture integrating IMC arrays with programmable multi-core accelerators, including ISA extensions for efficient attention execution. At the tile level, we discuss area-efficient periphery designs for distributed IMC architectures, addressing quantization-aware execution, partial-sum accumulation, and lightweight activation support. Spanning both levels, we present mapping and scheduling strategies that co-optimize workload placement across heterogeneous compute resources. Together, these contributions outline a path toward efficient and flexible deployment of AI inference at the edge.

​

10:06 - 10:33

Cross-stack optimization through a brain-inspiration lens

Melika Payvand (University of Zurich and ETH Zurich, CH )

Embedded and edge systems must interpret continuous sensory streams under strict power and memory limits. I will present how brain-inspired computational principles, such as event-driven communication, localized connectivity, and multi-timescale dynamics, enable efficient temporal processing on constrained hardware. Through examples from neuromorphic architectures and hardware–algorithm co-design, I will show how NeuroAI can guide the development of scalable, low-power computing substrates for real-time intelligence.

​

10:33-11:00

Nonlinear In-memory Computing for SNNs at the Edge

Arindam Basu (City University of Hong Kong, HK)

With the rapid increase in parameters and concomitant energy cost of modern Deep Neural Networks (DNN), it has become imperative to explore more energy-efficient architectures, especially for deployment on energy and area constrained edge devices such as smart glasses, hearables or medical implantable devices for brain interfaces. In this context, brain-inspired spiking neural networks (SNN) provide a fascinating new option with low-energy operation due to sparse binary activations, that reduce communication overhead and replace synaptic multiplications with additions.

Similar to ANN counterparts, In-memory computing (IMC) can be used in SNNs as well to remove the memory access bottlenecks in terms of throughput and energy. However, with synaptic operations becoming more efficient, the nonlinear operations such as dendritic computing and winner take all constitute a new “nonlinearity wall”. In this talk, I will show our recent work in embedding such scalar and vector nonlinear functions efficiently within memory leveraging nonlinear in-memory analog-digital converters. The architectures are general and have been demonstrated in both SRAM and RRAM substrates. Further, I will demonstrate how co-designing the algorithm and hardware lead to immense gains in energy and throughput with little sacrifice in task accuracy. I will also show the possible applications of these methods in improving the implementation of conventional ANN architectures like Recurrent neural networks and Transformers.

We envision the embedding of nonlinear operations within memory as an exciting new frontier for IMC research with potential to impact both SNN and ANN architectures.

​​

11:00 - 11:30

Coffee break

​

11:30 - 13:00

(no session) 

​

13:00 - 14:00

Lunch

​

​

Hardware/software co-design for accelerators for foundation models

ROOM Victoria 

session chairs

Kavya Sreedhar (Google, US)

Thierry Tambe (Stanford University, US)

 

 

14:00 - 14:35

TPU Hardware/Software Co-Design at Google

Kavya Sreedhar (Google, US)

AI is growing at an unprecedented scale, with the compute required to train AI models doubling more often than every five months. Given the scale of this problem, and that Moore’s Law is slowing down, there is a need to explore cross-stack optimization opportunities to design efficient AI chips. This talk will overview hardware/software co-design considerations at Google when designing Tensor Processing Units (TPUs) for AI applications.

 

14:35 - 15:10

A Structured Approach to the Hardware/Software Co-Design of an AI-Accelerator

Joel Emer (MIT / NVIDIA, US)

Over the past few years, efforts to address the challenges of the end of Moore's Law has led to significant rise in AI-focused accelerators. However, the well-understood pipeline stages used in the past for designing and optimizing general-purpose processors does not apply to these accelerators. As a result,there has not been a systematic way to express the range of design features used by these accelerators. This makes it difficult to understand the impact of each design choice and compare or extend the state-of-the-art.

In this talk, I will present a novel separation of concerns taxonomy to better understand and design AI-accelerators. One aspect of this approach is the use tensor algebraic notation to describe and optimize the computation. Second, in an analogous fashion to our prior work that categorized DNN dataflows into patterns like weight stationary and output stationary, this talk will try to provide a set of options that characterize the other attributes of tensor accelerators. Thus, rather than just presenting a single specific design, I will present a generalized framework for describing computations, dataflows and the placement of activity. In this framework, this separation of concerns is intended to better understand design choices and facilitate the exploration of the wide design space of tensor accelerators. Thus, using this common language, I will show how the choices invoked were used to generate our high-performance transformer accelerator.

​

15:30 – 16:00

Coffee break

​

16:00 - 16:35

Leveraging Natural Human Behavior for Efficient and Intelligent AR and VR Systems

Sai Zhang (NYU, US)

​Augmented and virtual reality (AR/VR) systems are emerging as a critical computing platform in modern life, with increasing impact across fields like education, healthcare and industrial applications. Despite their growing importance, AR and VR devices operate under strict constraints on latency, energy consumption, and computational resources, making efficient system design a fundamental challenge. A defining characteristic that distinguishes AR and VR from conventional edge devices is their direct and continuous interface with the human user, where perception, attention, and intention fundamentally shape system behavior. By leveraging natural human behavior such as gaze, head motion, and hand interaction as first class signals, AR and VR systems can adapt their computation to what truly matters to the user, enabling selective processing and more efficient use of limited resources.

 

16:35 - 17:10

Algorithm-Hardware Co-Design of Retention-Aware Differentiated Memory Architectures

Thierry Tambe (Stanford University, US)

Modern AI systems are increasingly limited not by arithmetic, but by memory. As frontier AI models become more capable, they require far more data to be moved, stored, and accessed efficiently. These workloads systematically generate large volumes of short-lived data that are written in memory, consumed, and quickly discarded, as well as long-lived data that must be retained reliably across much longer time scales. Conventional memory systems are poorly optimized to this behavior, as they are typically designed as one-size-fits-all storage, resulting in excessive energy consumption and increasingly limited density scaling. We propose to address this mismatch by developing a retention-aware computing stack that treats data persistence as a central design consideration by statically and dynamically matching short-lived and long-lived data to differentiated memory architectures and technologies, each optimized for the appropriate retention window. The result is a more efficient and sustainable foundation for accelerating large-scale AI systems in both datacenter and edge settings.

​

17:10 - 17:45

Disruptive Architectures via Diverse, Dense, and Deeply 3D Integrated Memory + Logic

Robert Radway (UPenn, US)

​Pervasive AI/ML demands ever-higher on-chip memory + logic capacity to minimize costly on- and off-chip data movement. Three techniques pave the way for the next generations of future system scaling: 1) Ultra-dense 3D integration to combine heterogenous memory + logic technologies (e.g., SRAM, ReRAM, Gain Cells, and beyond), matching distinct device characteristics to application needs for reliable operation, 2) New architectural design points leveraging dataflows unlocked by ultra-dense 3D memory-logic connectivity, 3) Illusion for energy-efficient multi-chiplet systems via new fine-grained mapping, scheduling, and power management techniques.