Improving Alignment and Robustness with Circuit Breakers#

arxiv

AI system can take harmful actions and could be vulnerable to adversarial attacks. This paper proposed a way called circuit breaking to improve the safety of the system especially for the adversarial attacks the model to do harmful actions.

Current alignment methods such as refusal training is easy to be fooled by attacks.