There is a feature in tensorflow called benchmark. It lets you to have full report about the performance of your network. Shows by ms per operations. Also collects which operation is slowest and many other details.
First you should build benchmark_model in tensorflow. Take a look how you can do it. https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark