NicOS: Toward Programmable NICs as Multi-Tenant Cloud Resources

Synopsis

Cloud service providers increasingly adopt programmable network interface cards (NICs) to offload infrastructure functions such as network virtualization, storage disaggregation, transport protocols, and security defenses from host CPUs. In parallel, recent studies show that offloading parts of application logic to NICs, including ML training and inference, data analytics, and key-value stores, significantly improves application performance. Despite this confluence, NICs remain underutilized because there is no effective way to virtualize and manage their resources, confining them to infrastructure roles. Existing approaches struggle with real-time variability in application deployment and resource demands, and lack effective resource, performance, and security isolation across applications and tenants.

This project envisions NicOS, a new operating system architecture that manages resources on programmable NICs in multi-tenant environments. NicOS introduces generalizable resource abstractions and runtime support that let tenants and operators implement and deploy their applications on shared NICs, with dynamic provisioning, elastic scaling, and isolation of NIC applications and tenants.

Our approach

Realizing NicOS requires addressing fundamental questions at the intersection of networking, computer systems, and security:

1. Resource abstractions. Virtual on-chip resource, memory, and communication-channel abstractions tailored to heterogeneous NIC architectures, with runtime resource management optimized for streaming workloads.
2. Elastic scaling. Proactive horizontal scaling based on predicted workload changes, a shared object abstraction for correct state management, and in-place dynamic resizing of data structures, all designed around NIC reconfiguration delays.
3. Performance and security isolation. A virtual channel-oriented scheduler that allocates resources fairly and prevents availability attacks, together with the identification of new attack vectors in multi-tenant NICs and lightweight privileged access control through compile-time or runtime policy checks.
4. Prototyping and validation. Prototypes on a software NIC, an existing NPU-based NIC, and a clean-slate FPGA-based design, evaluated with diverse NIC applications and realistic workloads.

Potential impacts

If successful, NicOS will enable programmable NICs to function as multi-tenant cloud resources, improving performance and cost efficiency for both infrastructure and tenant applications. The project will release open-source software and hardware prototypes of NicOS, along with example applications and benchmarks, to benefit practitioners and students. Research outcomes will be integrated into networked systems courses, including hands-on labs and projects built on NicOS, and the project will continue to mentor undergraduates and members of underrepresented communities. Collaboration with industry partners will help evaluate NicOS across diverse infrastructure settings and workloads.

The open-source software, hardware, data, and results will be available for public use under a permissive open-source license on this website.