how to get the consistent or reproducible results of benchmarks
Hi, I am interested in the benchmarks and would like to further understand the method of recording running time and comparing time differences. Do the benchmarks record running time for each benchmark by using time.time()
or something? Is it reliable to measure time by using time.time()
instead of timeit
module? And how do you design the environment to ensure the consistent or reproducible result? Could you give some references (online links or papers) about the approach of performance comparison?
Thank you!
Edited by anonymous