Hoss: Fast Oblivious Semantic Search with Heterogeneous GPU-CPU-TEE Architecture
Hoss uses heterogeneous GPU-CPU TEEs for oblivious semantic search, achieving up to 67x speedup over the Compass ORAM-based system.
Hoss is a first-of-its-kind oblivious semantic search system built on a heterogeneous CPU-GPU TEE architecture, hosting hot-path HNSW graph traversal in large GPU TEE private memory and offloading lower graph layers to CPU TEEs. It invokes oblivious primitives only for lower-layer accesses and adds host-access ORAM and data-dependent optimizations. Benchmarked against prior state-of-the-art Compass, the prototype achieves up to 67x speedup while maintaining high recall, with larger gains at scale.
- Leverages large GPU TEE private memory to bypass traditional ORAM bandwidth constraints
- Offloads lower HNSW layers to CPU TEEs, invoking oblivious primitives only there
- Prototype achieves up to 67x speedup over Compass with high recall
From the original article
1 min read at arxiv.orgSemantic search is widely deployed in modern AI systems, but protecting both data contents and access patterns remains challenging. The current state-of-the-art system, Compass, achieves oblivious semantic search by building an optimized ORAM over HNSW graphs. However, even with aggressive optimizations, it still incurs large overheads.
By Jianzhang Du · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.