Cognition helps Devin test its own work with GPT‑6 Astra
Cognition integrates GPT-6 Astra into Devin, its CLI, and desktop products to automate testing and provide evidence for code review.
OpenAI published a customer case study on Cognition, maker of the autonomous software engineer Devin, which serves customers from large banks to startups. Cognition uses GPT-6 Astra across its cloud agent, CLI, and desktop products; in one example, Devin uses Astra to test the iPhone game Otter Run and returns a simulator recording plus a report of passed checks and untested areas. Co-founder Walden Yan says Astra's ability to test and prove work functions as expected could reduce manual code review. The team also uses Astra to fix customer-reported bugs from screenshots and return verification screenshots faster.
- Astra testing is applied across Devin's cloud agent, CLI, and desktop products.
- Devin tested the iPhone game Otter Run, producing a simulator recording and coverage report.
- Customer-reported bug screenshots are fixed and verified automatically via Devin with Astra.
- Cognition expects less manual code examination and higher shipping velocity over time.
Full article327 words · extracted from openai.com · click to collapse
Cognition is the company behind Devin, an autonomous software engineer used by businesses around the world, from big banks to tech-native startups. As Cognition’s engineering teams write more code, reviewing that work has become a challenge. Cognition sees GPT‑6 Astra’s ability to test its work and show the results as a way to make code review more efficient.
“One of the big pieces that Astra improves on is its ability to test and prove that its work actually functions the way you expect.”
Walden Yan, Co-founder, Cognition
Testing software and showing the results
Cognition is applying GPT‑6 Astra across its product lineup. “We’ve been using it to make improvements across our product, including the core cloud agent that is Devin, but also our CLI and desktop products,” says Yan.
In one example, Devin uses Astra to test Otter Run, an iPhone game, and returns a recording of the game running in a simulator, alongside a report identifying checks that passed and areas left untested.
The recording shows the application’s behavior, while the report documents the scope of the testing. Engineers can use those outputs to inspect how the software works and understand what still needs attention.
Astra is also helping Cognition get back to customers “much quicker,” says Yan. When a customer sends a screenshot of a bug, the team can pass it to Devin using Astra, which fixes the issue and returns a screenshot showing the result.
Working toward less manual code review
Cognition sees stronger testing with GPT‑6 Astra as a path toward a more efficient review process. Having Devin test its work and provide evidence of the results could help engineers evaluate changes with less manual examination of the code.
“We expect over time that we have to manually look at less code and end up shipping more at the end of the day. This is one of the things we’re really excited about when it comes to GPT-6.”
Walden Yan, Co-founder, Cognition
Text extracted automatically; images, tables and formatting may be missing. Original: https://openai.com/index/cognition-devin-testing-with-astra