Nechama K.

Responsibilities:

• Research: Conducted a comparative study on "KV-Cache Offloading": Benchmarked vLLM with KVrocks (SSD) vs. DRAM to quantify cost-performance trade-offs

• Provisioned a high-throughput RAID0 array across dual 15TB NVMe SSDs specifically to accelerate KVrocks I/O operations

• Deployed the open-source vLLM framework on NVIDIA L4 GPUs to establish a baseline environment for Llama-3-8B inference

• Engineered the integration of vLLM with LMcache and KVrocks, enabling the chunk-based redirection of KV-cache blocks from GPU memory to local storage

• Calculated precise concurrency limits by analyzing HBM saturation points relative to token size and KV-cache footprint

• Optimized RocksDB parameters (compression, write buffers) to minimize storage latency under high concurrent user loads

• Enhanced the Python benchmarking script by implementing timeout exception handling to filter data skew and ensure accurate TPS metrics

• Synthesized benchmark data into comparative DRAM vs. SSD graphs plotting RPS/GPU vs. TPS/USER, illustrating the cost-performance trade-offs