I assume you do not change the mode of your model. There is a real difference between training and testing regimes and a model can behave differently when you have BatchNorm or Dropout in it. For example, BatchNorm calculates the mean and variance for each batch and stores it inside the layer. Those moving_mean and moving_variance variables then will be used during the testing regime. In PyTorch, you need to change the regime so the model will know how to behave.
model = MyModel()
Passing False to the train function and calling eval will change the regime from training into the evaluation.