Safety

Researchers unveil unelicitable backdoors in language models, using cryptographic transformer circuits, defying conventional detection methods and raising crucial AI safety concerns.

Current backdoor defenses, while effective at reducing attack success rates, are vulnerable to rapid re-learning. This work unveils this superficial safety, proposes a novel attack, and introduces a p…

Stepwise Alignment for Constrained Policy Optimization (SACPO) efficiently aligns LLMs with human values, prioritizing both helpfulness and harmlessness via a novel stepwise approach.

SEEV framework efficiently verifies ReLU neural barrier functions by reducing activation regions and using tight over-approximations, significantly improving verification efficiency without sacrificin…

AI agents can secretly collude using steganography, hiding their interactions from oversight. This research formalizes this threat, analyzes LLMs’ capabilities, and proposes mitigation strategies.

Rule-Based Rewards (RBRs) enhance LLM safety by using AI feedback and a few-shot prompt-based approach, achieving higher safety-behavior accuracy with less human annotation than existing methods.

LLM refusal is surprisingly mediated by a single, easily manipulated direction in the model’s activation space.

Verifiably safe AI controllers are created via a novel framework, VerSAILLE, which uses differential dynamic logic and open-loop NN verification to prove safety for unbounded time horizons.

Neural networks revolutionize hardware model checking by generating formal proof certificates, outperforming state-of-the-art techniques in speed and scalability.

AI systems are made safer by ‘circuit breakers’ that directly control harmful internal representations, significantly improving alignment and robustness against adversarial attacks with minimal impact…

This paper introduces Expectation Alignment (EAL), a novel framework and interactive algorithm to address reward misspecification in AI, aligning AI behavior with user expectations.

This paper introduces a novel quantitative definition of AI alignment for social decision-making, proposing probably approximately aligned policies and a method to safeguard any autonomous agent’s act…

Post-processing pre-trained models for alignment using conformal risk control and property testing guarantees better alignment, even when training data is biased.