When LLM judges agree, should we believe them?
Amazon ICML paper uses Ising models to correct correlated LLM-judge votes, beating accuracy-weighted panels by 9-14%.
Amazon Science describes an ICML paper, "Dependence-aware label aggregation for LLM-as-a-judge via Ising models," addressing how correlated judge outputs inflate majority-vote confidence. The unsupervised method models pairwise dependence between judges, learning both reliability and similarity without human reference labels. Tested on relevance, toxicity, and summarization tasks with 10 judge models at temperature zero, it outperformed accuracy-weighted voting by 9% to 14%.
Cisco Advance Notification for Publication of September 16, 2026, Security Advisories
Cisco will publish security advisories with fixed software on September 16, 2026, covering BroadWorks, ISE, Nexus Dashboard, ASA, FMC, FTD and ThousandEyes.
Cisco PSIRT announced advance notification for security advisories to be published on September 16, 2026, along with fixed software releases. Affected products include BroadWorks CommPilot Application Software, Identity Services Engine (ISE), Nexus Dashboard, Secure Firewall ASA, Secure Firewall Management Center (FMC), Secure Firewall Threat Defense (FTD), and ThousandEyes Virtual Appliance. ISE, Nexus Dashboard and the Secure Firewall products receive security hardening releases, and the ASA, FMC and FTD advisories will be included in the same combined release.