FlashVector: Agent for Hierarchical Model Serving Stack Optimization
FlashVector agent optimizes all layers of Unity's ad-serving stack, delivering up to 2x model-server throughput and 1.98x latency speedup in production.
FlashVector is an agentic system that optimizes performance across GPU kernels, ML framework computation graphs, model servers, and on-demand feature processing. Deployed in Unity's Vector advertising platform, it achieved up to 2x model-server throughput increase, 1.98x latency speedup, and 1.6x feature-store throughput gain. Optimizations spanned NVIDIA Triton's C++ codebase and the Python feature transformation service, demonstrating extensibility beyond single-kernel tuning.
30