What to do when you have small dataset

I'm trying to train a classifier with neural network, but I've got too small datasets. Each class has about ~1k examples. What is the best approach?

classificationsmall-dataset
3 votesJO218.00
3 Answers
KR4.00
2

You can try building a one-shot learning model with a siamese network. This model comprises of two similar networks in parallel and outputs the similarity score between the two input image from each network.  One shot learning model takes one image per class and there are few-shot learning models too.  These are some of the alternatives to consider when the training data size is limited. 

Paper with code: https://paperswithcode.com/task/one-shot-learning

Paper: https://arxiv.org/pdf/1703.03400v3.pdf

Code: https://paperswithcode.com/paper/model-agnostic-meta-learning-for-fast#code

Reply
sK2.00
1

I would suggest (if data is labeled)

1. Cross validation using leave one out

2. Try to increase the training data by some data augumentation techniques (for example of data is images, mirror the image, shift  the image, rotate)

3.Another good technique id bootstrapping.

Reply
JW229.00
2

There is a paper called "Overfitting Mechanism and Avoidance in Deep Neural Networks" by Shaeke Salman and Xiuwen Liu for the cases when you have a small dataset. I thought they do something like ensembling, but the approcah is different. They rely on the examples, which are classified with high probobility, like different models are confident for these examples. The other part called sensetive examples, because different models give different results. Take a look at the paper

https://arxiv.org/pdf/1901.06566.pdf

Reply
Couldn't find what you were looking for?and we will find an expert to answer.
How helpful was this page?