Ultra-Low Latency NIC-CPU Interface Design and Realization

This thesis builds off an ultra-low latency network interface device, known as the Lightning NIC (or L-NIC) [7], that opts to bypass the entire processor memory system and cache hierarchy in favor of delivering network data directly into the processor's register file. L-NIC is itself inspired by the J-machine [4], an earlier experimental parallel computer that investigated register-based, rather than memory-based, interactions with data. We expect a particular class of self-contained, granular, cache-resident, sub-microsecond applications, known as nanoservices, to enjoy a substantial performance benefit from the lower latency this approach provides. In turn, nanoservices, by allowing for much finer-grained parallelism than has been previously possible, can be used to significantly accelerate many existing distributed applications.

In "The nanoPU: From Microservices to Nanoservices" [6], our group presents a modified RISC-V processor, known as a nanoPU, that supports this L-NIC approach to network data and is optimized for running nanoservices. We also present a set of candidate nanoservice applications, demonstrate an order of magnitude performance improvement over a baseline NIC, and discuss how other distributed applications with a high degree of parallelism might stand to benefit from a nanoservice-focused refactor and a network of nanoPUs.

Here, I expand upon several complete components of this larger effort, focusing specifically on the design and development of the nanoPU's hardware-software interface — that is, on the particular instructions and registers relevant to the integration of the L-NIC into the processor. This is itself a novel undertaking. The challenge, at its core, is to design a system that supports easily usable software interfaces in an environment where each individual instruction constitutes a noticeable latency increase and a single DRAM access in an executing nanoservice introduces an unacceptably high delay.