基於區塊鏡射正交層和對數機率退火損失函數構造具可認證穩健性的深度神經網路

Abstract

Lipschitz 神經網路在深度學習中以具備可證明的穩定性而聞名。本研究提出一種新穎且高效的區塊鏡射正交（Block Reflector Orthogonal，簡稱 BRO）層，進一步提升正交層在建構更具表現力的 Lipschitz 神經網路架構上的能力。此外，我們從理論角度分析 Lipschitz 神經網路的特性，並引入一種新的損失函數，透過退火機制來擴大大多數資料點的分類間隔。這使得 Lipschitz 模型在分類結果的可認證穩健性上表現更佳。結合我們提出的 BRO 層與損失函數，我們設計出 BRONet ——一個簡潔且高效的 Lipschitz 神經網路，能達成目前最佳的可認證穩健性。我們在 CIFAR-10/100、Tiny-ImageNet 以及 ImageNet 上進行大量實驗與實證分析，結果顯示我們的方法優於現有的基準模型。

Lipschitz neural networks are well-known for providing certified robustness in deep learning. In this paper, we present a novel, efficient Block Reflector Orthogonal (BRO) layer that enhances the capability of orthogonal layers on constructing more expressive Lipschitz neural architectures. In addition, by theoretically analyzing the nature of Lipschitz neural networks, we introduce a new loss function that employs an annealing mechanism to increase margin for most data points. This enables Lipschitz models to provide better certified robustness. By employing our BRO layer and loss function, we design BRONet — a simple yet effective Lipschitz neural network that achieves state-of-the-art certified robustness. Extensive experiments and empirical analysis on CIFAR-10/100, Tiny-ImageNet, and ImageNet validate that our method outperforms existing baselines.