Search: “llm-apis”

30 stories

DomainAll Cyber AIWindow24h 3d 7d 30d AllIn the wild Important only Show duplicates

CategoryAll

SeverityAll critical 0 high 0 medium 0 low 0 info 0

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

llm 0.34

Version 0.34 of Simon Willison's llm CLI adds response-duration metrics to log output, plus bug fixes and faster log querying.

The open-source llm command-line tool for interacting with large language models released version 0.34. The headline change adds response duration in milliseconds and human-readable form to llm logs --usage Markdown output, plus a new duration_ms field in llm logs --short. The release includes several contributed bug fixes and a significant performance improvement to llm logs, alongside the related llm-openrouter 0.7.1 release.

Simon WillisonSep 2, 19:23 UTC · 14d agoAI tools & infra1

Towards Scalable and Cost-Efficient Vulnerability Detection: A Study on Automatic Query Generation

A study finds LLM-synthesized CodeQL queries improve average F1-score by 82% over baseline queries, offering scalable vulnerability detection versus direct LLM scanning.

Researchers conducted an empirical study evaluating whether LLMs can synthesize executable CodeQL queries from National Vulnerability Database vulnerability data. LLM-generated queries significantly enhanced baseline CodeQL suites, yielding an 82% improvement in average F1-score across a diverse set of real-world vulnerabilities. A cost-benefit analysis shows direct LLM-based scanning of entire repositories is often computationally and financially prohibitive, while LLM query synthesis offers a scalable and cost-effective alternative for large-scale vulnerability detection.

arXiv cs.CRSep 9, 16:31 UTC · 7d agoResearch1

Hackers Use LLMs to Generate Exploit Scripts and Automate Post-Exploitation Across Latin America

Unit 42 says Latin American attackers used LLMs to automate post-exploitation in campaigns hitting Mexican government, water utilities, and Brazilian financial firms.

Unit 42 identified two campaigns in Latin America whose operators used commercial LLMs (Claude, GPT-4.1) behind a self-hosted NextChat interface to generate and debug post-exploitation scripts. Cluster CL-CRI-1131 compromised a transportation organization, Mexican federal ministries, and water utilities in Mexico and Ecuador, using native Windows tools and Volume Shadow Copies to dump the SAM registry hive and NTDS.dit. Cluster CL-CRI-1163 targeted Brazilian financial organizations with job-themed phishing, custom RATs, and a Go-based reverse SOCKS5 tunneling utility called SockTz, with nine versions deployed within roughly two hours. Trend Micro tracks related AI-augmented activity as SHADOW-AETHER-040 and SHADOW-AETHER-064.

GBHackersupdated Sep 10, 08:06 UTC · 6d agofirst Sep 10, 06:02 UTC · 6d agoThreat actor in the wild 2 sources1

LLMs and Contextual Integrity

Bruce Schneier highlights two papers: the CIMemories benchmark shows frontier LLMs leak memory attributes up to 69%, and an RL method reduces inappropriate disclosures.

Bruce Schneier discusses contextual integrity in LLMs, referencing the CIMemories benchmark, which uses synthetic profiles with 100+ attributes per user to test whether models with persistent memory disclose sensitive information appropriately. Evaluation showed frontier models exhibit up to 69% attribute-level violations, with GPT-5's violation rate rising from 0.1% to 9.6% across 40 tasks and reaching 25.1% with repeated prompting, showing unstable leakage behavior. A second paper introduces a reinforcement learning framework trained on a synthetic 700-example dataset that substantially reduces inappropriate disclosure while maintaining task performance, with improvements transferring to the human-annotated PrivacyLens benchmark.

Schneier on SecurityAug 18, 10:40 UTC · 29d agoAI safety & security1

llm 0.35

llm CLI tool version 0.35 adds support for OpenAI's new GPT-6 Astra model exposed as gpt-6-astra.

Simon Willison released llm 0.35, which adds an OpenAI model definition for GPT-6 Astra under the model ID gpt-6-astra. No other release details were provided in the post.

Simon WillisonSep 7, 23:54 UTC · 9d agoAI tools & infra

Towards Tackling Application Logic Flaws through Autonomous Formal-Logic Modeling and Automated Reasoning

LL-Verifier combines LLMs with logic model checking to automatically discover logic flaws, uncovering vulnerabilities in 27 IoT access-control protocols.

Researchers present LL-Verifier, a framework that uses LLMs to autonomously convert natural-language protocol descriptions and security goals into formal logic models in a new logic language built on Maude, then applies logic model checking for exhaustive verification. The framework targets application-logic flaws that are tied to business semantics and hard to scale with manual analysis. Evaluation on 27 access-control protocols of widely used IoT devices uncovered a range of sophisticated logic vulnerabilities with security and privacy implications.

arXiv cs.CRSep 9, 17:58 UTC · 7d agoResearch1

Training a 3.8B LLM to 0.384 CORE for $998 – Hugo Vergnes

Independent developer Hugo Vergnes trained a 3.8B-parameter Llama-style model to 0.384 CORE on 65B tokens for $998 in 43 hours on rented B200s.

Hugo Vergnes trained little-lm, a 3.848B-parameter decoder-only LLM, on 65.3B tokens in 43 hours for $998 using rented NVIDIA B200s, scoring 0.384 on the CORE benchmark and beating nanochat d32 (0.310) at similar cost. The Llama-style architecture uses RMSNorm, RoPE, GQA with 24 query and 8 KV heads, relu-squared MLPs, QK-norm, and ResFormer-style value embeddings that account for 19% of parameters. Key wins included the Muon optimizer for matrix parameters, a trapezoidal learning-rate schedule with linear cooldown, FP8 training plus vocabulary padding for roughly 33% throughput gains, and the ClimMix dataset over FineWeb-Edu. The project, inspired by Karpathy's nanochat, was built as a config-driven YAML framework for small LLM training.

Hacker News · securitySep 10, 02:04 UTC · 7d agoAI research HN 28↑ · 2 comments1

llm-openrouter 0.7.1

Simon Willison released llm-openrouter 0.7.1, a performance fix for loading OpenRouter models in his LLM CLI plugin.

Version 0.7.1 of the llm-openrouter plugin addresses a performance problem when loading OpenRouter models in the LLM command-line tool. The fix was contributed by GitHub user waveplate. It is a minor maintenance release with no security implications noted.

Simon WillisonSep 2, 20:23 UTC · 14d agoAI tools & infra

Ask HN: Anyone still coding like 2021? Where do you work?

Hacker News users debate coding without LLMs, with one developer fired for refusing AI tools and others describing daily hand-coding practice to counter skill atrophy.

An Ask HN thread collects experiences of developers who still write code without LLM assistance. One contributor says he was fired for political reasons after refusing to use LLMs despite adequate stated performance, and observes fewer job ads now require LLM use. Others describe starting each day with a LeetCode problem or 30-60 minutes of hand-coding to stay sharp, contractual bans on AI-generated code for a government-adjacent embedded product over unresolved copyright issues, and inconsistent corporate policies where ChatGPT or Codex use flip-flops between allowed and blocked while a CIO mandates 70-80% AI-generated code next year.

Hacker News · AISep 9, 13:41 UTC · 7d agoAI industry HN 47↑ · 54 comments2

Meet Redis LangCache: A Managed Semantic Cache That Cuts LLM API Costs by Up to 90% and Returns Cache Hits Up to 15x Faster

Redis launches LangCache, a managed semantic cache matching LLM prompts by meaning, cutting API costs up to 90% and returning hits up to 15x faster.

Redis LangCache is a fully managed semantic caching service in public preview on Redis Cloud, accessed via a REST API with Python and JavaScript SDKs. It embeds incoming prompts, vector-searches stored entries, and returns a cached response when similarity clears a configured threshold, skipping the LLM call entirely. Redis claims up to 90% cost savings and up to 15x faster cache hits; a demo run showed 0.37 seconds versus 2.232 seconds direct inference (about 6x) with zero LLM tokens. Customer Mangoes.ai reports a 70% hit rate, 70% lower LLM spend, and 4x faster responses on a patient-care voice app.

MarkTechPostSep 10, 22:34 UTC · 6d agoAI tools & infra1

llm-gemini 0.34

llm-gemini 0.34 adds support for Google's new Gemini 3.8 Flash model with configurable low, medium and high thinking levels.

Simon Willison released llm-gemini 0.34, a plugin for the LLM CLI that adds the gemini-3.8-flash model, including low, medium and high thinking levels, and fixes an issue where async responses failed to record the resolved model version. The release coincided with Google's launch of Gemini 3.8 Flash, plus a Gemini 3.8 Flash Cyber variant restricted to trusted defenders. Willison noted the Flash tier's speed, low cost and competence at HTML and JavaScript generation tasks.

Simon WillisonSep 2, 16:39 UTC · 14d agoAI tools & infra

Harnessing LLMs for Automating BOLA Detection

Unit 42's BOLABuster methodology uses LLMs to automate detection of broken object-level authorization vulnerabilities, uncovering flaws in Grafana, Harbor, and Easy!Appointments.

Palo Alto Unit 42 details BOLABuster, a methodology combining large language models with heuristics to automate detection of broken object-level authorization (BOLA) flaws, which traditional fuzzing and static analysis struggle to find. The approach uses LLM reasoning to understand application logic, map endpoint dependency relationships, and generate and interpret test cases. It found CVE-2024-1313 in Grafana, CVE-2024-22278 in Harbor, and 15 CVEs in Easy!Appointments. The team is continuing to hunt for BOLAs in open-source and internal projects.

Palo Alto Unit 42Aug 17, 12:34 UTC · Aug 17, 2026Research CVE-2024-1313 CVE-2024-22278 CVE-2023-3285+14 CVEs

PlannerForge: LLM Agents for Scenario-Based Testing of Motion Planners in Autonomous Driving

PlannerForge unifies scenario-based testing of autonomous driving motion planners in one LLM-agent framework, outperforming prior baselines.

PlannerForge is an LLM-agent framework that covers the full scenario-based testing pipeline for autonomous driving systems, spanning scenario generation, selection, modification, routing, planner testing, plus new enhancement and benchmarking stages. In evaluations with 10 off-the-shelf LLMs, best-per-task scores range from 0.88 to 1.00, and open-source 20-35B backends such as Qwen3.6:35B match commercial APIs on most tasks. End-to-end chaining retains 83% (commercial) and 78% (open) of seed queries, beats Scenario Factory 2.0 on executable generation, and cost-tuning lifts planner success from 50.4% to 70.2% while cutting collisions from 19.0% to 8.4%.

Hugging Face daily papersSep 7, 20:00 UTC · 9d agoAI research

TuxBot v3: Inside an IoT Botnet Framework With LLM

Unit 42 uncovers TuxBot v3, an LLM-assisted IoT botnet framework with 17-architecture builds, Telnet brute-forcing, and DDoS capabilities.

Palo Alto Unit 42 identified TuxBot v3 Evolution, a modular IoT botnet framework derived from AISURU, Wuhan-lineage botnets, and MHDDoS. The C-based bot brute-forces Telnet with 1,496 credential pairs, targets over 30 IoT device families, and communicates with a Go-based C2 over encrypted TCP with multiple fallback mechanisms including DGA, P2P, and DNS TXT. LLM-assisted development left hallucinated crypto implementations and broken exploit modules in the analyzed samples, though roughly 70% of core functionality works. Researchers warn polished production builds likely exist, raising the threat potential.

Palo Alto Unit 42Aug 19, 12:03 UTC · 28d agoMalware1

Litelm: LiteLLM Without the Bloat

New open-source project Litelm launched on GitHub as a leaner alternative to the LiteLLM multi-provider LLM gateway.

Litelm is a community-published GitHub project positioned as a stripped-down alternative to LiteLLM, a widely used gateway/SDK for routing requests to multiple LLM providers. The post drew modest attention on Hacker News with 45 points and 17 comments. The listing provides no further technical details, benchmarks, or feature specifics.

Hacker News · AISep 11, 18:10 UTC · 5d agoAI tools & infra HN 45↑ · 17 comments2

How much of F-Droid is LLM generated?

A FOSS maintainer manually graded 102 F-Droid apps from the September 12, 2026 update batch, finding many show signs of LLM-generated code.

A student and FOSS app maintainer reviewed 102 apps pushed to F-Droid on September 12, 2026, assigning each a three-tier rating for likelihood of LLM-authored code (mostly AI >50%, hard to say/mostly human, no signs of AI). The heuristic relies on commit aesthetics, README and branding style, and the presence of agentic infrastructure like Claude Code or Codex, which automatically places an app in the 'mostly AI' tier. Example ratings include Amber (Nostr event signer) as mostly AI, and Aria for Misskey as showing no AI signs. The author stresses reliable detection of LLM-generated code from text alone is impossible, so ratings are approximate.

Hacker News · AISep 15, 09:47 UTC · 1d agoAI research HN 46↑ · 23 comments

Φ-Bench: Can Large Language Models Engineer the Infrastructure That Powers Them?

Researchers release Phi-Bench, a benchmark evaluating frontier LLMs on open-ended, long-horizon engineering and optimization of the LLM infrastructure stack.

Phi-Bench evaluates LLMs on open-ended engineering of the LLM infrastructure stack, derived from optimization problems studied in frontier research and grounded in real-world code repositories. Tasks range from localized kernel-level function completion to long-horizon implementation and end-to-end system optimization. Experiments on frontier LLMs reveal current capabilities and limitations on the path toward autonomous optimization of future AI infrastructure.

Hugging Face daily papersSep 8, 20:00 UTC · 8d agoAI research1

The Self-Expanding Stolen Inference Supply Chain: An AI Agent Harvesting and Re-Serving LLM Access, (Fri, Sep 11th)

An autonomous coding agent harvested LLM API access from poorly secured gateways and aggregated stolen inference capacity behind a self-hosted gateway

A SANS researcher observed a semi-autonomous coding agent finding weakly secured LLM resale gateways via FOFA queries, creating trial accounts with temporary emails and CAPTCHA solving, and exploiting weak authorization such as client-supplied group_id fields. The agent validated stolen keys using factorial code-logic tests, then loaded roughly 379 upstream endpoints into a self-hosted New-API gateway, disabling 341 fake or dead channels. Five model names including claude-opus-5 and gpt-5.6-sol were served via round-robin and failover, forming a partially self-expanding inference supply chain resembling an evolution of LLMjacking.

SANS Internet Storm CenterSep 11, 14:40 UTC · 5d agoThreat actor in the wild

When the prompt becomes the payload: A practical pen-testing guide for GenAI, LLM and RAG applications

CSO Online publishes a practical penetration-testing guide for GenAI, LLM, and RAG applications, covering prompt injection, retrieval poisoning, and tenant isolation testing.

The guide frames LLM applications as attack graphs spanning prompts, retrieval layers, vector stores, tools, identities, and downstream APIs, arguing that conventional web testing misses instruction-vs-data channel risks. It builds on OWASP prompt injection guidance (direct vs. indirect injection) and NIST's 2025 adversarial machine-learning taxonomy, noting that RAG and fine-tuning do not remove injection risk. Recommended practices include documenting trust transitions across components, using canaries and synthetic records to avoid test side effects, running multi-turn and obfuscated injection campaigns, and verifying chains from poisoned documents to observable state changes. It also details testing RAG pipelines via controlled document poisoning across metadata, OCR layers, and code comments, plus cross-tenant isolation checks on retrieved document IDs.

CSO OnlineSep 9, 10:00 UTC · 7d agoAI safety & security1

VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities

Introduces VEX-Bench, 75 expert-labeled real-world cases testing whether LLM agents can assess supply chain vulnerability exploitability; frontier models reach about 80% F1.

VEX-Bench is the first benchmark evaluating LLM agents on assessing whether upstream dependency vulnerabilities are exploitable in downstream projects, with 75 real-world expert-labeled cases across Python, Java, and Go mined from GitHub. Nine models across three agent harnesses were evaluated; GPT-5.5 and Claude Opus 4.6 reach approximately 80% F1 on binary vulnerability-status classification, but only GPT-5.5 surpasses 70% macro-F1 on fine-grained justification classification. The gap highlights the difficulty of moving beyond binary exploitability calls to explaining exploitability reasons, unlike prior benchmarks targeting zero-day settings.

arXiv cs.CRSep 7, 22:52 UTC · 9d agoResearch1

Retrofitting Code Using LLMs to Support Exceptional Behavior

EXCODER combines static/dynamic analysis with LLMs to retrofit exception-handling code, achieving 85.92% pass@1 with Qwen 2.5 Coder 32B on Java benchmarks.

The paper introduces the task of retrofitting existing code with Exception Related Code (throw statements, guarding conditions, try/catch blocks) so that given Exceptional Behavior Tests pass. EXCODER performs context engineering by integrating static and dynamic program analysis output with LLMs; it was evaluated on a benchmark built from 304 methods across 75 GitHub Java projects. Combined with Qwen 2.5 Coder 32B, EXCODER achieves pass@1, 5, and 10 rates of 85.92%, 86.18%, and 86.51%, roughly 13 percentage points over baseline, and manual inspection reveals remaining limitations.

arXiv cs.AI / cs.LG / cs.CLSep 9, 16:18 UTC · 7d agoAI research1

Context Engineering Inside the Harness: 4 Mechanisms That Beat Context Overflow and Goal Loss on Long-Horizon Tasks

Survey of four harness mechanisms—context budgeting, compaction, todo-state, and memory—that keep long-horizon LLM agents on task across 200+ tool calls.

The article details how agent harnesses, not larger context windows, solve context overflow and goal loss on long-horizon tasks, citing Chroma's Context Rot report showing 18 LLMs (GPT-4.1, Claude 4, Gemini 2.5, Qwen3) degrade on long inputs. Concrete implementations include LangChain Deep Agents offloading tool responses over 20,000 tokens to the filesystem and truncating old tool calls at 85% window usage, and Claude Code capping auto memory at 25KB while re-reading the 5 most recently modified files after compaction. OpenAI's Responses API now offers server-side compaction via context_management with a standalone /responses/compact endpoint, which Codex uses for long-running coding tasks. Manus reports a roughly 100:1 input-to-output token ratio per ~50-tool-call task, motivating todo.md state recitation to prevent goal drift.

MarkTechPostSep 13, 05:56 UTC · 3d agoAI research1

Large Language Models for HVAC Operations in Building Energy Systems: A Critical Review of Methods, Applications, and Deployment Readiness

Systematic review of 66 studies finds LLMs for HVAC operations are mostly research-stage, with no ready-now deployment and only four pilot-level studies.

A critical review of 66 peer-reviewed studies from 2023 to March 2026 examines LLMs for HVAC operations in building energy systems. Only four studies reach pilot-level evidence, none reports sustained operational deployment, and 63 of 66 are research-only. Conventional ML, MPC, and RL remain dominant for high-frequency control and short-horizon forecasting, and the evidence supports LLMs primarily as semantic and workflow layers rather than autonomous controllers.

arXiv cs.AI / cs.LG / cs.CLSep 4, 16:08 UTC · 12d agoAI research

LLM Agents as Computational Typologists

AUTOTYPOLOGIST is an LLM agent that performs evidence-grounded linguistic typology analysis over 25 open-source reference grammars.

The agent retrieves relevant grammar sections, analyzes interlinear glossed text (IGT), and iteratively reasons over typological hypotheses in a ReAct-style workflow. It was evaluated on typological feature coding against expert annotations and hypothesis testing against universals using 25 open-source reference grammars. Results suggest LLM agents can support scalable, inspectable crosslinguistic analysis but still require expert validation.

arXiv cs.AI / cs.LG / cs.CLSep 7, 17:35 UTC · 9d agoAI research1

The OWASP Top 10 for LLM Applications 2026: From Model Risks to Agentic Security

Akamai analyzes the OWASP Top 10 for LLM Applications 2026, which shifts focus from model-level risks to agentic AI security.

The OWASP Top 10 for LLM Applications has been updated for 2026, and Akamai published an analysis of the revised list. Per the title, the 2026 edition shifts emphasis from model-level risks toward the security of agentic AI systems, framed as a realistic security model. No article body was available, so the specific ranked risk entries cannot be enumerated.

Akamai BlogAug 14, 12:00 UTC · Aug 14, 2026AI safety & security

Plug 'n' Pray: Agentic LLM-based Detection of Potential Log File Exposures in Third-Party Content Management System Plugins

Agentic LLM analysis validates 79 log file exposures across 62 of the 300 most-installed WordPress plugins, covering 250M+ active installations.

Researchers built an agentic LLM-based framework combining static and dynamic analysis to automatically detect insecure log files created by WordPress plugins. Scanning the 300 most-installed plugins, which account for roughly 75% of all active installations in the official ecosystem, it produced 81 findings with 79 manually reproduced across 62 plugins. Insufficiently secured log files can disclose credentials and personal data and have led to website compromises. The authors derive a taxonomy of log path and protection patterns and best practices, finding multi-layered protection often absent.

arXiv cs.CRSep 15, 13:29 UTC · 1d agoResearch

When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk Control

Chain-of-Self-Questioning prompting cuts LLM wrong-answer commitments 32% relative while raising answered accuracy, holding across eleven model families.

The paper introduces Chain-of-Self-Questioning (CoSQ), a prompt-only framework that makes LLM answer commitment conditional on an explicit assessment of the information required to answer. On an 817-item TruthfulQA multiple-choice set, Grounded-CoSQ at τ=0.90 reduced mean unconditional wrong-commitment rate from 13.1% under chain-of-thought to 8.9% (a 32.1% relative reduction), while raising answered accuracy from 86.9% to 89.7% at 87.6% coverage. Improvements held across eleven open-weight and hosted model families and at every evaluated threshold, with convergent evidence from a Natural Questions short-answer evaluation.

arXiv cs.AI / cs.LG / cs.CLSep 15, 17:52 UTC · 1d agoAI research

[webapps] Langflow 1.8.4 - Path Traversal to Remote Code Execution

A path traversal to remote code execution exploit for Langflow 1.8.4, a popular LLM application builder, was published on Exploit-DB.

Exploit-DB lists a proof-of-concept exploit chaining path traversal to remote code execution in Langflow 1.8.4, an open-source tool used to build LLM applications and agents. The chain allows an attacker to write arbitrary files outside the intended directory and achieve code execution on the host. The provided text does not include a CVE identifier or reports of exploitation in the wild, but RCE in a widely deployed AI tooling product is notable for defenders.

Exploit-DBAug 31, 00:00 UTC · 17d agoExploit / PoC1

AI Coding Agents Are Installing Unknown/Untrusted Code on Corporate Networks

Researchers found 120 corporate llms.txt files pointing to unregistered packages, demonstrating AI coding agents install and execute attacker-controlled code on Fortune 500 networks.

Researchers at an Israeli stealth startup scanned 6,214 live domains belonging to defense contractors, Fortune 500 and Big Tech companies, finding 120 llms.txt files that pointed to unregistered code packages or domain names. After registering a handful of the unclaimed names, they received a phone-home beacon within an hour from a Fortune 500 company and dozens more over time. Parent-process chains showed coding agents including Claude, OpenAI's Codex and Nous Research's Hermes executed the installed packages. The researchers warn agents treating vendor docs as ground truth creates a SolarWinds-style supply-chain surface as agent adoption spreads across SaaS, cloud and endpoints.

Schneier on SecuritySep 4, 10:35 UTC · 12d agoAI safety & security in the wild1

Multi-Agents LLM Financial Trading Framework

TauricResearch's TradingAgents, an open-source multi-agent LLM framework for financial trading, gained traction on Hacker News with 94 points and 60 comments.

The TradingAgents GitHub repository from TauricResearch presents a multi-agent LLM framework for financial trading and surfaced on Hacker News with 94 points and 60 comments. It targets developers building agentic trading systems on top of LLMs.

Hacker News · AISep 8, 05:20 UTC · 8d agoAI tools & infra HN 94↑ · 60 comments