You can try building a one-shot learning model with a siamese network. This model comprises of two similar networks in parallel and outputs the similarity score between the two input image from each network. One shot learning model takes one image per class and there are few-shot learning models too. These are some of the alternatives to consider when the training data size is limited.
Paper with code: https://paperswithcode.com/task/one-shot-learning
I would suggest (if data is labeled)
1. Cross validation using leave one out
2. Try to increase the training data by some data augumentation techniques (for example of data is images, mirror the image, shift the image, rotate)
3.Another good technique id bootstrapping.
There is a paper called "Overfitting Mechanism and Avoidance in Deep Neural Networks" by Shaeke Salman and Xiuwen Liu for the cases when you have a small dataset. I thought they do something like ensembling, but the approcah is different. They rely on the examples, which are classified with high probobility, like different models are confident for these examples. The other part called sensetive examples, because different models give different results. Take a look at the paper