In this thread
http://www.mini2.com/forum/showthrea...threadid=21584 I used a GTech/Pro Comp to try to compare the stock and Rogue filter on my
MCS. There were variations from run to run that could be attributed to the GTech but there was also a noticable drop in power as the engine got very hot and also a 5hp change from cleaning the filter. This makes me question the performanc claims from modifications that were based on just two dyno runs.
It seems as though the only way to reliably see what a change has done is to repeatedly test until consistent results are found, make the change, repeatedly test again until there are new consistent results. Even better would be to then go back to the original configuration, test again and verify you get the same results as before. The last step would check for effects from air temperature changes during the day.
This makes it really tough to compare changes that are not easy to switch - new exhausts, new ECU or supercharger pully. The 5hp change from simply cleaning the filter shows that comparing two different cars tells us very litle unless the change is large.
It also shows how easy it is for a manufacturer to claim large increases from their products - it is easy to pick a good result for their change and a poor one for stock. e.g. Test theirs first and then test stock when the engine is really hot. They don't have to be deliberately deceiptful to be tempted to publish one of the results that is slightly better than average.
Any comments on this? Which of the performance claims being quoted are based on rigorous testing?