CNS-2420632

NeTS: SMALL: Develop Core Techniques and Applications of Learned and Disaggregated In-Network Lookups

2024-2028

PI: Prof. Chen Qian

University of California Santa Cruz

Project Summary

In-network lookups serve as fundamental functions and design blocks of numerous network protocols
and algorithms from the data link layer (e.g., MAC table lookups) to the application layer (e.g., lookups
in content distribution networks). Based on the PI’s expertise on in-network lookups and recent technology
trends, we have discovered new and important research issues that have not been addressed in prior
studies, including using machine learning models to replace traditional hash functions in lookup engines
and disaggregating lookup functions for heterogeneous devices. The objective of this project is to develop
LEarned and Disaggregated In-network Lookup Engines (LEDILE), a next-generation network framework
that provides memory and CPU efficiency, high throughput, effectiveness in handling network dynamics,
scalability to large networks, and compatibility with emerging network features.

This project will completely change the design and deployment of classic in-network lookup engines by
replacing a functional stage with a learned model and disaggregating lookup functions onto heterogeneous
devices. As such, the expected contributions of the proposed research are as follows: 1) Develop new
in-network lookups with perfect hashing and learned models. Learned models can effectively overcome
several existing limitations of traditional hashing; 2) Disaggregate in-network lookups in two dimensions:
divide a lookup engine into multiple stages and let each device execute one stage based on its resource
availability and divide the set of keys into shards and allow each lookup instance to handle a shard; 3)
Develop the LEDILE framework and its applications; 4) Evaluate the proposed algorithms, protocols, and
software framework on multiple platforms. The algorithms, protocols, software, and experimental tools
developed in this project will be made available to the public to attract researchers to work in this direction.

Project Publications

Yi Liu, Minghao Xie, Shouqian Shi, Yuanchao Xu, Heiner Litz, Chen Qian; Outback: Fast and Communication-efficient Index for Key-Value Store on Disaggregated Memory. In Proc. of the 51st International Conference on Very Large Data Bases (VLDB), 2025.

Minghao Xie, Chen Qian, Heiner Litz: En4S: Enabling SLOs in Serverless Storage Systems. in Proc. of ACM Symposium on Cloud Computing (SoCC) 2024. (Best Paper Award)

Yi Liu, Shouqian Shi, Ruilin Zhou, Yuhang Gan, and Chen Qian, Scalable, Fast, and Low-memory Lookups for Network Applications with one CRC-8, in Proc. of IEEE International Conference on Network Protocols (ICNP 2024), Accept rate 24%.

Yifan Hua, Jinlong Pang, Xiaoxue Zhang, Yi Liu, Xiaofeng Shi, Bao Wang, Yang Liu, and Chen Qian, Towards Practical Overlay Networks for Decentralized Federated Learning, in Proc. of IEEE International Conference on Network Protocols (ICNP 2024), Accept rate 24%.

Students

Graduate: Yi Liu (UCSC), Fei Fang (UCSC)

Software