A Smoother Way to Train Structured Prediction Models
We present a framework to train a structured prediction model by performing smoothing on the inference algorithm it builds upon. Smoothing overcomes the non-smoothness inherent to the maximum margin structured prediction objective, and paves the way for the use of fast primal gradient-based optimization algorithms. We illustrate the proposed framework by developing a novel primal incremental optimization algorithm for the structural support vector machine. The proposed algorithm blends an extrapolation scheme for acceleration and an adaptive smoothing scheme and builds upon the stochastic variance-reduced gradient algorithm. We establish its worst-case global complexity bound and study several practical variants, including extensions to deep structured prediction. We present experimental results on two real-world problems, namely named entity recognition and visual object localization. The experimental results show that the proposed framework allows us to build upon efficient inference algorithms to develop large-scale optimization algorithms for structured prediction which can achieve competitive performance on the two real-world problems.
NurtureToken New!

Token crowdsale for this paper ends in

Buy Nurture Tokens

Authors

Are you an author of this paper? Check the Twitter handle we have for you is correct.

Krishna Pillutla (add twitter)
Vincent Roulet (add twitter)
Sham M. Kakade (edit)
Zaid Harchaoui (add twitter)
Ask The Authors

Ask the authors of this paper a question or leave a comment.

Read it. Rate it.
#1. Which part of the paper did you read?

#2. The paper contains new data or analyses that is openly accessible?
#3. The conclusion is supported by the data and analyses?
#4. The conclusion is of scientific interest?
#5. The result is likely to lead to future research?

Github
Stargazers:
0
Forks:
2
Open Issues:
0
Network:
2
Subscribers:
1
Language:
Python
Smoothing and acceleration for max-margin structured prediction
Youtube
Link:
None (add)
Views:
0
Likes:
0
Dislikes:
0
Favorites:
0
Comments:
0
Other
Sample Sizes (N=):
Inserted:
Words Total:
Words Unique:
Source:
Abstract:
None
02/10/19 06:02PM
34,834
5,169
Tweets
arxivml: "A Smoother Way to Train Structured Prediction Models", Krishna Pillutla, Vincent Roulet, Sham M. Kakade, Zaid Harc… https://t.co/FZ7RWVPTFN
arxiv_cs_LG: A Smoother Way to Train Structured Prediction Models. Krishna Pillutla, Vincent Roulet, Sham M. Kakade, and Zaid Harchaoui https://t.co/0gnuR6HWUb
BrundageBot: A Smoother Way to Train Structured Prediction Models. Krishna Pillutla, Vincent Roulet, Sham M. Kakade, and Zaid Harchaoui https://t.co/fHzt7M6jk1
Images
Related