QCATS: Query Context-Aware Transformer Slicing for Efficient Predictive Query Processing
QCATS slices transformer FFNs using query predicates, cutting in-database inference latency up to 4.42x.
QCATS is a query context-aware transformer slicing framework for sparse inference inside relational databases. It uses query predicates and metadata statistics to pre-select context-aligned feed-forward slices before execution, rather than exposing only tuple-level inputs. Offline expert construction and lightweight query-level routing select experts dynamically, with asynchronous CPU-GPU pipelines and routing-aware batching. On four predictive-query workloads with BERT-base and Qwen-0.6B, it reports up to 4.42x lower latency while matching dense-baseline accuracy.
- Selects FFN experts from query predicates and metadata before inference
- Routes at query granularity rather than per token or tuple
- Uses asynchronous CPU-GPU pipelines and routing-aware batching
- Up to 4.42x lower latency on BERT-base and Qwen-0.6B
From the original article
1 min read at arxiv.orgIn-database predictive query processing increasingly applies Transformer-based models within relational pipelines. However, existing in-database inference typically exposes only tuple-level model inputs to the inference runtime, leaving relational predicates and metadata statistics invisible to neural execution planning.
By Yueying Li · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.