Low-Shot Classification: A Comparison of Classical and Deep Transfer Machine Learning Approaches
Despite the recent success of deep transfer learning approaches in NLP, there is a lack of quantitative studies demonstrating the gains these models offer in low-shot text classification tasks over existing paradigms. Deep transfer learning approaches such as BERT and ULMFiT demonstrate that they can beat state-of-the-art results on larger datasets, however when one has only 100-1000 labelled examples per class, the choice of approach is less clear, with classical machine learning and deep transfer learning representing valid options. This paper compares the current best transfer learning approach with top classical machine learning approaches on a trinary sentiment classification task to assess the best paradigm. We find that BERT, representing the best of deep transfer learning, is the best performing approach, outperforming top classical machine learning algorithms by 9.7% on average when trained with 100 examples per class, narrowing to 1.8% at 1000 labels per class. We also show the robustness of deep transfer learning in moving across domains, where the maximum loss in accuracy is only 0.7% in similar domain tasks and 3.2% cross domain, compared to classical machine learning which loses up to 20.6%.
NurtureToken New!

Token crowdsale for this paper ends in

Buy Nurture Tokens

Authors

Are you an author of this paper? Check the Twitter handle we have for you is correct.

Peter Usherwood (add twitter)
Steven Smit (edit)
Ask The Authors

Ask the authors of this paper a question or leave a comment.

Read it. Rate it.
#1. Which part of the paper did you read?

#2. The paper contains new data or analyses that is openly accessible?
#3. The conclusion is supported by the data and analyses?
#4. The conclusion is of scientific interest?
#5. The result is likely to lead to future research?

Github
User:
None (add)
Repo:
None (add)
Stargazers:
0
Forks:
0
Open Issues:
0
Network:
0
Subscribers:
0
Language:
None
Youtube
Link:
None (add)
Views:
0
Likes:
0
Dislikes:
0
Favorites:
0
Comments:
0
Other
Sample Sizes (N=):
Inserted:
Words Total:
Words Unique:
Source:
Abstract:
None
07/17/19 06:02PM
7,045
2,081
Tweets
Memoirs: Low-Shot Classification: A Comparison of Classical and Deep Transfer Machine Learning Approaches. https://t.co/z3peOQIK8j
arxivml: "Low-Shot Classification: A Comparison of Classical and Deep Transfer Machine Learning Approaches", Peter Usherwood… https://t.co/xv9swkyiWK
arxiv_cs_LG: Low-Shot Classification: A Comparison of Classical and Deep Transfer Machine Learning Approaches. Peter Usherwood and Steven Smit https://t.co/EdKcQkzma8
BrundageBot: Low-Shot Classification: A Comparison of Classical and Deep Transfer Machine Learning Approaches. Peter Usherwood and Steven Smit https://t.co/5dEht8BpKl
Images
Related