RiskChainBench: A Benchmark for Obfuscated Platform Message Restoration and Evidence-Grounded Web Investigation
RiskChainBench benchmark evaluates models restoring obfuscated abuse messages and investigating risky websites, revealing execution failures as the main bottleneck.
RiskChainBench is a benchmark pairing 3,600 synthetic obfuscated-message restoration inputs from 600 source sessions with 600 human-labeled local web environments tied to pornography, fraud, and gambling services. Models must first restore message intent and destination, then act as VLM-driven web agents producing evidence-cited risk reports without domain-reputation cues. Across ten models, Entry Top-1 ranges from 35.2% to 95.2% and web decision accuracy from 26.3% to 62.8%. Execution failures account for 31.9% of web runs versus only 0.9% post-decision type errors, identifying stable exploration and risk judgment as key bottlenecks.
- Benchmark pairs 3,600 obfuscated-message restoration tasks with 600 labeled web environments.
- Evaluates VLM web agents producing evidence-cited risk reports without reputation cues.
- Entry Top-1 spans 35.2%-95.2% and web decision accuracy 26.3%-62.8% across ten models.
- Execution failures cause 31.9% of web runs; post-decision type errors only 0.9%.
- Benchmark, protocol, and resettable local sandbox are publicly released.
- Products
- RiskChainBench
From the original article
1 min read at huggingface.co
Platform abuse campaigns conceal redirection instructions with emojis, homophones, character decomposition, and redundant symbols, then route users through disguised links to services associated with pornography, fraud, gambling, or illicit transactions. Existing benchmarks evaluate obfuscated text and risky webpages separately, obscuring how target recovery affects downstream evidence acquisition.
By ZhuoXin Liu, Zhiming Ma, Ying Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.