Rivka W.

Responsibilities:

• Defined a generic schema for incoming telemetry structures, based on analysis of telemetry data received from NVIDIA GPUs.

• Research: Telemetry ingestion on the server using Kafka/RabbitMQ, batching incoming data for efficient storage in InfluxDB.

• Evaluated multiple acceleration approaches — threading, Numba, Spark, and Polars — and selected the fastest solution, ultimately reducing query latency from 4.88 ms to 0.00548 ms through direct database queries for fast dashboards.

• Executed complex SQL queries (JOIN, PIVOT, CTE) across multiple InfluxDB tables to aggregate raw telemetry for dashboards requiring combined metrics — such as correlating error counts with efficiency levels.

• Migrated profiling data and logs into structured SQLite metrics tables, improving query performance and overall data visibility.

• Integrated Grafana primarily for performance profiling and system monitoring.

• Developed pytest tests for key components to validate functionality and handle important edge cases.