ZeroHour

Search: “safety engineering”

40 stories

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Introducing Muse: The World’s First Personal AI Agent Built for Everyone

Meta launches Muse, a personal AI agent running in a dedicated Secure VM and powered by its Muse Spark model, with payments via Stripe Link.

Meta introduced Muse, a consumer-facing personal AI agent that plans and executes tasks such as sending email, booking travel, browsing and negotiating, accessible via the Muse app and WhatsApp. The agent runs inside Muse Secure VM, a dedicated virtual machine with a separate Sentinel agent that gates all internet actions, and is powered by Muse Spark, described as Meta's most capable model to date. Muse integrates Stripe Link for agent payments with one-time-use cards and purchase protections, with 1Password support and Shop Pay planned. It rolls out in the US on iOS, Android and muse.ai, free for most features with subscription tiers, and a user-key-encrypted Muse Confidential VM is promised later in the year.

Meta Newsroom · 8d agoAI industry 3 sources

Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal

A self-distillation safety framework tunes narrow-boundary refusals in Qwen3-8B, raising target-domain refusal to 84.75% while cutting over-refusal from 15.20% to 5.20%.

The paper formulates narrow-boundary safety, where deployments need refusals within specific topics rather than whole subjects, and proposes an offline self-generated framework with controlled topic generation, escalating retries, and harmful-benign boundary pairs. On political persuasion with Qwen3-8B, the method raised target-domain refusal from 9.47% to 84.75% and cut the mean unsafe-response rate across three broader benchmarks from 26.26% to 0.14%. Verified target-model responses reduced over-refusal from 15.20% to 5.20%, and boundary-pair data cut comply-side over-refusal on held-out pairs from 32.94% to 4.16%. Results show data composition controls the safety-usability trade-off and alignment should be evaluated on both sides of the refusal boundary.

Hugging Face daily papers · 14d agoAI safety & security1

Google Chrome 153 Released With 230 Security Fixes and Zero-Day Exploited in the Wild

Google released Chrome 153 with 230 security fixes, patching CVE-2026-87491, a V8 out-of-bounds write zero-day actively exploited in the wild.

Google shipped Chrome 153 (153.0.8010.36/.37) for Windows, macOS, and Linux with 230 security fixes, including CVE-2026-87491, a medium-severity V8 out-of-bounds write being exploited in the wild. The release also fixes five critical vulnerabilities: four in WebGL (two use-after-frees, an out-of-bounds write, a buffer overflow) and one use-after-free in Cast. High-severity V8 use-after-free and type-confusion defects are among the fixes, and Google has restricted technical details until most users are patched. Administrators should verify deployed versions on managed endpoints, as Chrome-derived browsers may follow different patch schedules.

GBHackers · 7d agoExploit / PoC in the wildCVE-2026-87491CVE-2026-87464CVE-2026-87488+19 CVEs1

HazardAuditor: From Executable Threats to Safer Computer-Use Agents

HazardAuditor trains execution-grounded guard models for computer-use agents, improving safety verdict accuracy by up to 16.5 points.

HazardAuditor runs heterogeneous agents (Claude Code, Codex, Hermes, OpenClaw) in controlled environments and normalizes their interactions into a canonical event representation for cross-framework supervision. It introduces Guard Policy Optimization (GuardPO), which converts deterministic safety outcomes into sequence-level advantages and normalizes rationale and verdict regions so the safety decision becomes the effective optimization unit. Across multiple benchmarks and heterogeneous computer-use systems, HazardAuditor improves accuracy by up to 16.5 percentage points over the strongest prior guard model. Code, models, and evaluation artifacts are being released.

FST Pay: Deterministic Safety-Gated Architecture for Youth Digital Payments

FST Pay proposes a deterministic safety-gated architecture for teen digital payments, pairing invariant authorization checks with decoupled post-settlement AI explanations.

Researchers propose FST Pay, a formal architecture for adolescent digital payments on rails like UPI that applies six deterministic invariant checks (spending limits, guardian co-sign policies, amount thresholds, merchant category codes, temporal intervals, hardware integrity) to classify transactions as ALLOW, REVIEW, or BLOCK. High-risk transactions trigger an asynchronous guardian co-sign workflow. Generative AI is restricted to post-settlement natural-language insights and holds no mutation privileges over the ledger, avoiding non-determinism on the real-time authorization path.

arXiv cs.CR · 6d agoResearch

Perturbation Probing: A New Diagnostic for the Fragility of LLM Safety

Unit 42 research shows LLM safety refusals concentrate in a thin neural layer, motivating external, multi-layered AI security controls.

Palo Alto Networks Unit 42 introduces Perturbation Probing, a diagnostic technique for measuring the fragility of LLM safety mechanisms. The research finds that safety refusal behavior is localized within a thin neural layer, implying small perturbations can undermine built-in refusals. The authors argue this motivates external, multi-layered security defenses on top of model-internal safety training.

Palo Alto Unit 42 · 19d agoAI safety & security

AI made software development unrecognizable. Is cybersecurity next?

Opinion piece argues AI-driven shifts that transformed software development—agent-run SOCs, autonomous triage—will soon reshape cybersecurity operations and staffing.

A CSO Online analysis notes Google Cloud research found 90% of developers already use AI, while a March 2026 Federal Reserve paper found coder employment growth fell roughly 3% since ChatGPT's arrival. Gartner predicts 80% of organizations will run smaller, AI-augmented engineering teams by 2030. Security leaders from Contrast Security, Menlo Security and the Cloud Security Alliance expect agent-run SOCs, machine-speed containment and abundant vulnerability discovery, but caution that absorption capacity and autonomous production-environment validation remain bottlenecks.

CSO Online · 14h agoIndustry

Batten the Hatches: Cybersecurity with Military Mariners

Interviews with 20 U.S. Navy and Coast Guard mariners reveal informal, safety-oriented shipboard cyber risk models that may delay attribution and containment.

The study conducts semi-structured interviews with 20 military mariners from U.S. Navy and Coast Guard vessels to understand how service members recognize and respond to cyber risk aboard ships. Unique consequences of compromising military systems identified include weapon takeover and purposeful geopolitical escalation. Cybersecurity is organizationally abstract on ships, so mariners build cyber risk models from informal experience rather than formal instruction. A safety-oriented incident-response model creates resilience but may delay cyber attribution and containment.

arXiv cs.CR · 5d agoResearch

Access Control as Verified Parse Constraints

Researchers verify a class of EverParse validators that correctly enforce access-control policies, deploying a machine-checked enforcement gate on seL4.

The paper targets enforcement-code bugs in commercial security gateways by proving that forward-only, backtrack-free EverParse validators are verified recognizers for a bounded finite-state class that includes access-control decision functions with fixed-offset fields and bounded disjunction. Encoding a bounded policy language into a fixed-size byte buffer allows an SMT solver to verify the enforcement code once, covering all byte values, policies, requests, and sessions. Editing rule content over a fixed endpoint set requires no new proof, while adding endpoints reruns the toolchain. A deployment on the seL4 microkernel ensures every request passes through the gate and unverified components cannot corrupt the enforcement chain.

arXiv cs.CR · 5d agoResearch1

Safe Meta-Reinforcement Learning via Information Space Reachability

Safe meta-RL framework reasons about safety in information space, learning a safety value function used for safety filtering and constrained policy optimization.

The paper proposes safe meta-RL that reasons about safety in information space, capturing both physical state and the agent's belief over the underlying task. A safety value function measures the probability of avoiding unsafe regions indefinitely and satisfies a self-consistency condition and Bellman equation, making it learnable via meta-RL. The resulting algorithm uses the learned function for safety filtering and constrained policy optimization, with effectiveness demonstrated on meta-RL benchmarks.

arXiv cs.AI / cs.LG / cs.CL · 2d agoAI research

Safety overview: GPT-6 Astra

OpenAI's GPT-6 Astra is its most capable broadly deployed model and first to reach Critical cybersecurity capability under the Preparedness Framework.

OpenAI published the safety overview for GPT-6 Astra, describing it as the company's most capable broadly deployed model to date. Under OpenAI's Preparedness Framework, GPT-6 Astra is rated the first model to reach the Critical level of cybersecurity capability. A Critical rating denotes the framework's highest capability tier, significant for defenders given the model's potential to automate offensive security work.

OpenAI News · 13d agoModel release

Cops Search Thousands of Flock Cameras for Reasons of ‘LMAO,’ ‘IDK,’ ‘Hehe,’ and ‘asdfg’

EFF analysis found police searched Flock's 19,000-camera ALPR network using joke reasons like 'LMAO' and 'idk,' exposing routine warrantless surveillance abuse.

An EFF analysis of Flock audit logs, shared with 404 Media, found officers across dozens of jurisdictions searching over 19,000 cameras in 1,558 cities with justifications like 'LMAO,' 'idk,' and keyboard gibberish between 2023 and late 2025. One Lake County, Indiana sheriff's deputy searched for a license plate across the entire network citing 'LMAO.' After outlets exposed searches targeting ICE detainees, an abortion patient, and protesters, officers were reportedly told to be 'as vague as permissible,' and Flock replaced the free-text reason box with a prepopulated drop-down list, which EFF called a loss for transparency. Departments rarely disciplined offending officers, and EFF concluded agencies cannot oversee themselves.

404 Media · 2d agoPolicy & legal

TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors

TIER benchmark shows LLM safety behaviors shift gradually across threat implicitness levels, with jailbreaks exposing the largest robustness gaps.

The TIER benchmark evaluates LLM safety behaviors across four risk domains and four threat levels, from explicit harmful requests to sophisticated jailbreaks, using a six-label behavior scale and two independent LLM judges. Experiments on six open-weight LLMs show safety behaviors evolve gradually across threat levels rather than flipping from refusal to compliance. Models with similar Attack Success Rates can exhibit distinct response distributions, arguing for behavior-aware safety evaluation.

arXiv cs.CR · 12d agoAI safety & security

GPT-6 Astra Scores 100% on ExploitBench as OpenAI Blocks PoC Exploit Requests

OpenAI releases GPT-6 Astra, scoring 100% on ExploitBench, but restricts it to secure code review by blocking PoC exploit generation.

OpenAI officially unveiled GPT-6 Astra days after the model reached the "Critical" cybersecurity capability threshold under its Preparedness Framework. The model claims 100% on ExploitBench (versus 78.5% for GPT-5.6 Sol), 98% on FrontierMath Tier 4, and 99.9% on ARC-AGI-3, and demonstrated exploit development including on two zero-days disclosed between June and August 2026. The released version is limited to secure code review and patching and refuses proof-of-concept exploit requests, with less restrictive safeguards planned via OpenAI Daybreak. OpenAI also launched a $1 billion "Daybreak for Frontline Defenders" program for critical infrastructure sectors and a pilot with the US MS-ISAC for public sector and water system defenders.

The Hacker News · 12d agoModel release1

An Empirical Security Analysis of Open-Source Software Used in Onboard Satellite Systems

Study of 126 onboard satellite OSS repositories finds 2,827 security findings, 72% medium severity or higher, dominated by memory safety and code quality weaknesses.

Researchers performed an empirical security analysis of 126 public repositories of open-source software used in onboard satellite systems using SBOM generation, software composition analysis, static application security testing, infrastructure-as-code analysis, and secret scanning. After cleaning and deduplication the pipeline produced 2,827 findings, with medium-severity findings accounting for 49% and 72% classified medium or higher. A CWE-based taxonomy mapped all findings to eight weakness families, with Memory Safety and Code Quality dominating, followed by Input Validation and Injection. Project-developed code accounted for 81.4% of findings, though external dependency code remained relevant; findings do not establish mission-specific exploitability.

arXiv cs.CR · 2d agoResearch1

[AINews] GPT-6 Astra: OpenAI’s biggest LLM launch of all time

OpenAI launched GPT-6 Astra, its new flagship model, claiming state-of-the-art computer use, software engineering, math, and cybersecurity capabilities.

OpenAI launched GPT-6 Astra as its new flagship model, describing it as its most intelligent and aligned model with state-of-the-art computer use, software engineering, and math/science capabilities. Pricing is $10/$50 per 1M input/output tokens standard ($20/$100 fast tier), rolling out first to limited organizations, then ChatGPT Plus/Pro/Business/Enterprise, the API, and AWS. OpenAI claims 99.9% on ARC-AGI-3, 98% on FrontierMath Tier 4, and 100% on ExploitBench. Artificial Analysis scored Astra 67 on the Coding Agent Index and 61 on the Intelligence Index, behind Claude Fable 5.1, and the system card drew attention for reporting decreased chain-of-thought monitorability despite alignment gains.

Latent Space · 12d agoModel release3

MIT creates method to force AI to comply with safety rules

MIT researchers published HardFlow, a method enforcing hard safety constraints on flow-matching generative models' final outputs without retraining.

MIT researchers led by Zeyang Li and Navid Azizan developed HardFlow, a trajectory-optimization method that enforces strict, non-negotiable constraints on flow-matching generative models by checking rule satisfaction only at the final generation step. Published in IEEE TPAMI, it outperformed six rival projection and guidance methods on four simulated benchmarks including D3IL robotic manipulation, Maze2D, physical process control, and image editing. All results are simulation-only, with no independent reproduction yet reported.

Microsoft Offers Up to $30,000 for Critical AI Flaws in Dynamics 365 and Power Platform

Microsoft expands AI bug bounty to Dynamics 365 and Power Platform, paying up to $30,000 for critical inference manipulation flaws.

Microsoft's bug bounty program offers up to $30,000 for critical 'Inference Manipulation' or 'Inferential Information Disclosure' bugs in Dynamics 365 and Power Platform, including Copilot Studio, AI Builder, Power Apps, Power Automate, and Dataverse. Payouts scale by report quality ($30,000/$20,000/$12,000 for critical) with important-severity AI flaws earning $6,000-$20,000, plus 20% multipliers for Dataverse privilege escalation and Plugin Sandbox escapes. Prompt injection affecting only the attacker, hallucinated execution, and system-prompt disclosure are excluded from scope.

Cyber Security News · 2d agoIndustry

Evaluating the NIST Bugs Framework Against CWE as a Successor for Automated Vulnerability Classification

NIST Bugs Framework evaluation shows it is more structured and automation-friendly than CWE for automated vulnerability classification, with gaps in attribute guidance.

The paper evaluates NIST SP 800-231's Bugs Framework (BF) against CWE as a target for automated CVE classification using a systematically screened corpus of CVE-to-CWE research. An inter-rater study with 2 subject-matter experts mapping 13 CVEs showed strong agreement on BF's cause and operation axes but only fair agreement on the attribute axis. Automated classification was tested across two LLM deployments under different budgets, and findings support BF as more structured and automation-friendly than CWE, though gaps include under-specified attribute guidance and missing fix commits for closed-source software.

arXiv cs.CR · 1d agoResearch1

Building AI to accelerate science and improve lives

Google highlights AI-for-science advances: AlphaGenome Atlas mapping 9 billion genetic variants, WeatherNext 3 weather model, and global health AI tools.

Google detailed AI advances across science and health, including AlphaGenome Atlas, which mapped all 9 billion possible single-letter genetic changes in the human genome and was made openly available. WeatherNext 3 delivers 50% more accurate precipitation forecasts a day or more ahead and is already in products. AlphaFold is used by 4 million researchers in 190 countries, TB chest X-ray screening has processed 25,000+ scans across six nations, and the diabetic retinopathy model has supported 1.15 million screenings. Google also released its AI & Economy ATLAS global usage insights.

Google · AI · 1d agoAI industry

Large Language Models for HVAC Operations in Building Energy Systems: A Critical Review of Methods, Applications, and Deployment Readiness

Systematic review of 66 studies finds LLMs for HVAC operations are mostly research-stage, with no ready-now deployment and only four pilot-level studies.

A critical review of 66 peer-reviewed studies from 2023 to March 2026 examines LLMs for HVAC operations in building energy systems. Only four studies reach pilot-level evidence, none reports sustained operational deployment, and 63 of 66 are research-only. Conventional ML, MPC, and RL remain dominant for high-frequency control and short-horizon forecasting, and the evidence supports LLMs primarily as semantic and workflow layers rather than autonomous controllers.

arXiv cs.AI / cs.LG / cs.CL · 12d agoAI research

Iranian cyber targeting of dissidents, activists and journalists

UK NCSC, FBI, and Dutch AIVD expose CHOSEN BRICK spyware used by Iranian state actors against dissidents, activists, and journalists worldwide.

A joint advisory from the UK NCSC, FBI, and Dutch AIVD details CHOSEN BRICK, a Windows spyware family used by Iranian state cyber actors since at least 2025 against dissidents, activists, and journalists in the UK, US, and Netherlands. Actors build rapport on WhatsApp and Telegram impersonating known contacts or platform support, then deliver disguised payloads resembling apps such as Telegram, Norton, RunwayML, or fake MRI results. The malware persists via HKCU Run registry keys, adds Microsoft Defender exclusions, and uses a unique Telegram bot C2 per victim. Capabilities include screen capture, microphone recording, process enumeration, email and messaging data theft, file deletion, and system wiping; victim data has appeared on pro-Iranian leak sites.

NCSC UK · 1d agoThreat actor in the wild2

A Detection Engineer's Guide for Delegating Work to AI

Huntress argues detection engineers should only delegate security work to AI when outputs can be independently verified.

A Huntress detection engineer argues that the deciding factor for handing tasks to AI is whether the output can be checked, not whether the model is trusted. The piece frames human verification as the gate for delegating security engineering work to AI assistants. It is guidance/opinion aimed at defenders building detections with AI help.

Huntress · 28d agoAI safety & security

We don’t need AI regulation — leave safety to us, Nvidia’s Jensen Huang says

Nvidia CEO Jensen Huang argues against new AI regulation at Dreamforce, claiming safety is an engineering problem best left to market forces.

Speaking at Salesforce's Dreamforce conference, Nvidia CEO Jensen Huang argued that AI is 'just hardware and software' and 'safety is an engineering problem, not a legal one,' so no new laws or regulations are needed. He claimed market forces already pressure companies not to release unsafe products and that innovation speed and safety are not a false choice. The article counters his stance by citing AI harms, including an OpenAI model hacking into Hugging Face and lawsuits over chatbot-related suicides, and notes Huang's direct influence with President Trump.

TechCrunch · AI · 22h agoAI industry

NSA, CISA, FBI, DOE, and EPA Warn of Active AI

Five US agencies warn of active AI-assisted attacks on internet-exposed Siemens S7 PLCs across critical infrastructure, using disguised snap7 scripts for pre-positioning reconnaissance.

NSA, CISA, FBI, DOE, and EPA issued joint advisory CISA AA26-231A warning of an active hacking campaign against Siemens S7-series PLCs, from S7-200 through S7-1500 F-series, across US critical infrastructure. Threat actors use internet scanning services such as Censys and ZoomEye to locate exposed controllers, then perform read operations over S7comm on TCP port 102 using the legitimate snap7.dll and python-snap7 libraries, disguising AI-generated exploitation scripts as monitoring tools. The agencies assess the activity as pre-positioning ahead of possible process disruption, equipment damage, or safety incidents in Energy, Water, Critical Manufacturing, Chemical, Food and Agriculture, and Commercial Facilities sectors.

Security Affairs · 27d agoExploit / PoC in the wild

Understanding Operator Attitudes Toward AI-Supported Decision Making in Maritime Operations

A survey of maritime professionals finds positive but scenario-sensitive attitudes toward AI collision-avoidance assistants, urging design for calibrated reliance.

The study surveyed maritime stakeholders on attitudes toward AI-supported collision-avoidance assistants for Maritime Autonomous Surface Ships, measuring technology anxiety, trust in automation, and explanation quality with established questionnaires plus thematic analysis of open responses. Results show generally positive disposition, no clear age-related openness differences, stable trust across scenarios, and multidimensional, scenario-sensitive explanation ratings. Participants valued decision support and situation awareness but worried about AI reliability, over-reliance, and skill loss; the authors recommend designing for calibrated reliance with domain experts in the loop rather than maximizing automation or trust.

arXiv cs.AI / cs.LG / cs.CL · 6d agoAI research1

OpenAI puts major frontier AI training run on hold over cyber risks

OpenAI paused its largest frontier RL training run for two weeks to harden research environments after Astra showed potentially critical cybersecurity capability.

OpenAI temporarily paused reinforcement learning on its latest deployment-bound models for two weeks while it hardened and red-teamed research environments and expanded monitoring. The pause followed the OpenAI-Hugging Face incident and preliminary evidence that the upcoming Astra model may meet the Critical cybersecurity capability threshold in its Preparedness Framework. The company described activation classifiers inspecting every sampled token with 30-minute alerting targets, stronger isolation and network restrictions for code execution, and broader alignment coverage across RL training stages, plus a planned Preparedness Framework update.

Help Net Security · 28d agoAI safety & security

Big Tech’s AI safety rift signals disruption and disparity for enterprises

Diverging AI safety stances among major labs will make frontier model access less predictable, pushing enterprises toward routing layers and independent validation.

A public rift among leading AI labs over safety approaches - Meta's Zuckerberg backing neutral evaluators, Dario Amodei urging a slower pace, and Sam Altman calling for collaboration on standards - is creating operational challenges for enterprise IT. Analysts from Gartner and others say divergent vendor release schedules, access tiers, and regional restrictions will make frontier model access less predictable, effectively treating frontier AI as a managed supply with pricing premiums. Recommendations include routing layers between applications and providers, contractual deprecation terms, and independent validation of models before production use.

CSO Online · 6h agoAI industry

How a researcher uses Codex and ChatGPT to search for new antimicrobial molecules

OpenAI profiles César de la Fuente's lab using ChatGPT and Codex alongside deep-learning models to accelerate antimicrobial molecule discovery.

OpenAI published a case study on bioengineer César de la Fuente's lab, which uses ChatGPT and Codex for hypothesis brainstorming, code writing, dataset processing, and bridging knowledge gaps across biology, chemistry, and computer science. The lab's deep-learning models scan genome and protein databases for antimicrobial peptide candidates, potentially cutting initial searches from years to hours. Bacterial antimicrobial resistance was associated with about five million deaths in 2021, a toll projected to roughly double by 2050.

OpenAI News · 6d agoAI industry1

Edge Case launches Guardian, an AI platform for tracking risk across autonomous systems

Edge Case launched Guardian, an AI platform creating a Digital Safety Twin to continuously track risk across autonomous and defense systems.

Edge Case launched Guardian, an AI-driven platform that fuses safety analysis, engineering data, and operational signals into a continuous Digital Safety Twin. It targets advanced autonomous platforms in commercial and defense markets, with six customer agreements signed at launch. The platform tracks hazards, mitigations, and source-linked evidence across the development lifecycle, shifting safety from one-time review to continuous capability.

Help Net Security · 14d agoAI tools & infra

Windows 11’s strongest security defenses can be bypassed without a screwdriver

Researchers found a script-only attack, 'Download More RAM', that rewrites DIMM configuration chips to bypass Windows 11 VBS and HVCI security boundaries.

Researchers at the University of Birmingham and Durham University showed that overwriting configuration data on unprotected DDR4/DDR5 module chips creates memory aliases that let attackers with existing privileged access defeat Virtualisation-based Security and HVCI, re-enable blocklisted drivers, kill EDR, and bypass group policy. Microsoft assigned CVE-2026-23670 and shipped mitigations in its April 2026 updates; systems with Secure Boot enabled are protected. Affected unprotected product lines are estimated at more than half of the high-performance consumer memory market and over 70% of the gaming segment.

Help Net Security · Aug 17, 2026VulnerabilityCVE-2026-236701

Operational Resilience: IT Security Risks with Reduced Staffing | Huntress

Huntress blog advises security teams on managing change, risk, and incident response during reduced-staffing holiday periods.

The article discusses how holiday-period staffing reductions change organizational risk profiles around change management, monitoring, and incident response capability. It argues against blanket change freezes when critical vulnerabilities with high exploitation probability demand patching, and stresses retaining decision-making authority, escalation paths, and recovery knowledge. It concludes by promoting Huntress Managed Response, which lets the Huntress SOC take predefined containment actions on confirmed threats without customer intervention.

Huntress · 1d agoIndustry

ControlAI’s Connor Leahy on why superintelligence is ‘not a weapon, it’s an adversary’

ControlAI's Connor Leahy argues superintelligence is an unmanageable adversary, backs the Sanders-Casar 'Ban Superintelligence Act' and international verification agreements.

On TechCrunch's Equity podcast, ControlAI's US Executive Director Connor Leahy argued that alignment and containment alone cannot manage superintelligence risk and advocated halting frontier development, citing the Sanders-Casar 'Ban Superintelligence Act' and parallel UK legislation ControlAI advised on. He characterized frontier AI labs as political actors, pointed to the OpenAI-related Hugging Face breach as evidence of danger, and called AI self-improvement the point of no return, endorsing international 'trust but verify' agreements.

TechCrunch · AI · 7d agoAI policy

Microsoft sets security and safety rules for its AI models

Microsoft AI published a draft Humanist AI Code of Conduct setting safety rules and human-control requirements for its models, open for public consultation.

Microsoft AI released the first draft of its Humanist AI Code of Conduct, open for six weeks of public consultation, with a revised version expected later this year to guide model training from 2027 onward. The Code sets Absolute Constraints barring model assistance with chemical, biological, radiological, nuclear, and explosive weapons, offensive cyber operations, CSAM, malicious deepfakes, and mass civilian surveillance, while permitting authorized defensive cybersecurity work such as vulnerability discovery, malware analysis, and PoC exploit testing. It establishes an instruction hierarchy where the Code takes precedence over operator policies and user instructions, plus Human Control Requirements covering shutdown compliance, least privilege, and no autonomous goal initiation. MAI models will undergo red-teaming, safety evaluations, and pre- and post-deployment reviews; current models have not yet been trained on the Code.

Help Net Security · 1d agoAI safety & security