Vercel Labs open-sources agent-desktop for agent computer use
Original title: Vercel Labs Open-Sources agent-desktop for Agent Computer Use
Version 1.0.2 under Apache-2.0 drives native apps on macOS, Windows and Linux without screenshots or a bundled model.
Vercel Labs released agent-desktop 1.0.2, an Apache-2.0 Rust CLI and C library that lets coding agents operate native apps through operating-system accessibility trees. Snapshots return JSON element refs instead of screenshots, and ref actions can run headless without moving the pointer. It supports macOS 13+, Windows 10 1809+ or Server 2019+, and Ubuntu 24.04+ on x64 and ARM64. The project reports large token cuts versus full-tree dumps but has no independent computer-use benchmark.
Unconfirmed: Token-savings figures are self-reported and not from an independent benchmark.
- Install is npm install -g agent-desktop, with x64 and ARM64 binaries.
- A skeleton snapshot reads three levels; drill refs open one region.
- On Apple Maps, the author reported tokens per look falling from 3,811 to 273.
- Ref actions auto-wait up to 5,000 ms and fail closed on stale refs.
From the original article
7 min read at marktechpost.com
Vercel Labs released agent-desktop, an Apache-2. 0 Rust CLI that gives any AI agent computer use on macOS, Windows and Linux by reading OS accessibility trees and acting on stable refs instead of pixels. The post Vercel Labs Open-Sources agent-desktop for Agent Computer Use appeared first on MarkTechPost.
By Michal Sutter · marktechpost.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.