Skip to content
NewA calmer, faster ZeroHour

Orbit AI multi-tool agent leads document RAG on 80 questions

Original title: Orbit AI: Evaluating Tool-Augmented Agents For Enterprise Question Answering

arXiv cs.CRby Rishabh Gupta ()1 min read
AI summary · x-ai/grok-4.7Summaries by Vani AI

Gemini 3.5 Flash Lite scored 86.25 percent with all sources; local Llama 3.2 full-agent accuracy was 52.50 percent.

Orbit AI is a LangGraph ReAct agent that retrieves from project documents, task systems and team conversations. On 80 held-out Phase 3 questions, Gemini 3.5 Flash Lite reached 86.25 percent with the full agent, 60.00 percent with document-only RAG and 23.13 percent without retrieval. Multi-source accuracy rose from 40.63 percent to 84.38 percent. The authors caution that Gemini ran via its API while Llama 3.2 ran locally on Ollama, and note a small artificial workspace plus similarity between Q53 and the system prompt.

Unconfirmed: The authors say the Gemini-versus-Llama gap cannot be pinned to one cause.

Key facts
  • Document-only RAG trailed the Gemini agent by 26.25 percentage points.
  • The multi-source gain for the full Gemini agent was 43.75 points.
  • A local Llama 3.2 setup scored 5.50 percent under rubric v2.2.
  • Authors say Q53 resembles a problem already in the system prompt.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at arxiv.org

Enterprise information is commonly stored across multiple platforms rather than in one centralized location. Answering questions about workplace activities may require information from several sources, including project documents, task management systems, and team conversations.

arXiv cs.CR

By Rishabh Gupta · arxiv.org

Read the full article at arXiv cs.CR

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.