MixTrain: Scalable Training of Formally Robust Neural Networks
There is an arms race to defend neural networks against adversarial examples. Notably, adversarially robust training and verifiably robust training are the most promising defenses. The adversarially robust training scales well but cannot provide provable robustness guarantee for the absence of attacks. We present an Interval Attack that reveals fundamental problems about the threat model used by adversarially robust training. On the contrary, verifiably robust training achieves sound guarantee, but it is computationally expensive and sacrifices accuracy, which prevents it being applied in practice. In this paper, we propose two novel techniques for verifiably robust training, stochastic output approximation and dynamic mixed training, to solve the aforementioned challenges. They are based on two critical insights: (1) soundness is only needed in a subset of training data; and (2) verifiable robustness and test accuracy are conflicting to achieve after a certain point of verifiably robust training. On both MNIST and CIFAR datasets, we are able to achieve similar test accuracy and estimated robust accuracy against PGD attacks within $14\times$ less training time compared to state-of-the-art adversarially robust training techniques. In addition, we have up to 95.2% verified robust accuracy as a bonus. Also, to achieve similar verified robust accuracy, we are able to save up to $5\times$ computation time and offer 9.2% test accuracy improvement compared to current state-of-the-art verifiably robust training techniques.
Authors

Are you an author of this paper? Check the Twitter handle we have for you is correct.

Shiqi Wang (add twitter)
Yizheng Chen (edit)
Ahmed Abdou (add twitter)
Suman Jana (add twitter)
Ask The Authors

Ask the authors of this paper a question or leave a comment.

Read it. Rate it.
#1. Which part of the paper did you read?

#2. The paper contains new data or analyses that is openly accessible?
#3. The conclusion is supported by the data and analyses?
#4. The conclusion is of scientific interest?
#5. The result is likely to lead to future research?

Github
User:
None (add)
Repo:
None (add)
Stargazers:
0
Forks:
0
Open Issues:
0
Network:
0
Subscribers:
0
Language:
None
Youtube
Link:
None (add)
Views:
0
Likes:
0
Dislikes:
0
Favorites:
0
Comments:
0
Other
Sample Sizes (N=):
Inserted:
Words Total:
Words Unique:
Source:
Abstract:
None
11/07/18 06:05PM
13,036
2,969
Tweets
HubBucket: RT @arxiv_org: MixTrain: Scalable Training of Formally Robust Neural Networks. https://t.co/fGOC4UKE4x https://t.co/DkTWZyHp6z
PerthMLGroup: RT @Miles_Brundage: "MixTrain: Scalable Training of Formally Robust Neural Networks," Wang et al.: https://t.co/zquMoAAFTE
AssistedEvolve: RT @Miles_Brundage: "MixTrain: Scalable Training of Formally Robust Neural Networks," Wang et al.: https://t.co/zquMoAAFTE
arxiv_org: MixTrain: Scalable Training of Formally Robust Neural Networks. https://t.co/fGOC4UKE4x https://t.co/DkTWZyHp6z
nmfeeds: [O] https://t.co/OHs9QXxQ7C MixTrain: Scalable Training of Formally Robust Neural Networks. There is an arms race to defen...
EricSchles: https://t.co/HCmudKT08C this looks interesting. Has anyone done a Keras/pytorch implementation yet? I’m excited to play around with it on some data
EricSchles: RT @Miles_Brundage: "MixTrain: Scalable Training of Formally Robust Neural Networks," Wang et al.: https://t.co/zquMoAAFTE
JeanMarcJAzzi: RT @Miles_Brundage: "MixTrain: Scalable Training of Formally Robust Neural Networks," Wang et al.: https://t.co/zquMoAAFTE
vksbhandary: RT @Miles_Brundage: "MixTrain: Scalable Training of Formally Robust Neural Networks," Wang et al.: https://t.co/zquMoAAFTE
arxivml: "MixTrain: Scalable Training of Formally Robust Neural Networks", Shiqi Wang, Yizheng Chen, Ahmed Abdou, Suman Jana https://t.co/v8oEh2PPyM
BrundageBot: MixTrain: Scalable Training of Formally Robust Neural Networks. Shiqi Wang, Yizheng Chen, Ahmed Abdou, and Suman Jana https://t.co/LmhMvgLNuo
treasured_write: RT @Miles_Brundage: "MixTrain: Scalable Training of Formally Robust Neural Networks," Wang et al.: https://t.co/zquMoAAFTE
RyanDavidReece: RT @Miles_Brundage: "MixTrain: Scalable Training of Formally Robust Neural Networks," Wang et al.: https://t.co/zquMoAAFTE
Miles_Brundage: "MixTrain: Scalable Training of Formally Robust Neural Networks," Wang et al.: https://t.co/zquMoAAFTE
Images
Related