[Keith]

In this thread http://www.mini2.com/forum/showthrea...threadid=21584 I used a GTech/Pro Comp to try to compare the stock and Rogue filter on my MCS. There were variations from run to run that could be attributed to the GTech but there was also a noticable drop in power as the engine got very hot and also a 5hp change from cleaning the filter. This makes me question the performanc claims from modifications that were based on just two dyno runs.

It seems as though the only way to reliably see what a change has done is to repeatedly test until consistent results are found, make the change, repeatedly test again until there are new consistent results. Even better would be to then go back to the original configuration, test again and verify you get the same results as before. The last step would check for effects from air temperature changes during the day.

This makes it really tough to compare changes that are not easy to switch - new exhausts, new ECU or supercharger pully. The 5hp change from simply cleaning the filter shows that comparing two different cars tells us very litle unless the change is large.

It also shows how easy it is for a manufacturer to claim large increases from their products - it is easy to pick a good result for their change and a poor one for stock. e.g. Test theirs first and then test stock when the engine is really hot. They don't have to be deliberately deceiptful to be tempted to publish one of the results that is slightly better than average.

Any comments on this? Which of the performance claims being quoted are based on rigorous testing?

[jlm]

this has come up before. I think the thing to keep in mind is that there are at least two kinds of potential errors with dyno measurements:

1. inconsistency due to not controlling variables for each run (such as maintaining constant: intake air supply, intercooler temperature, hood closed, etc.)

2. inconsistency from day to day due to variables not under control (such as humidty, ambient temp, altitude, dyno erratta, etc.)

applying the results has its own problems since almost no compared results have been measured under the same condtitions and to be statistically believable, an average of many runs has to be used, not the best, not the worst.

fat chance you will find any mfgr dyno claims presented in this light.

I have spent a few days with Helix using their dyno on several minis. It is possible to satisfy #1 and we were able to make many runs to find a stable, believable result, but you really need to do it all on the same day to satisfy #2 and even then its not the Bureau of Standards, its only comparative for that day subject to the limitations of the dyno testing (limited airflow, no ground effect affecting airflow, airlow not velocity related.)
the dyno is a useful tuning tool, but the track (and the g-techpro, to a certain extent) tell us more real results.

[Keith]

Thanks. And that's the problem - are there any performance comparisons we can trust? Grinder has published some great info at http://www.mcshp.org/ but I don't know the sources of the data. I suspect I am like most owners and want to choose modifications based on which does the most. Is there any good data to help make the decisions? If not, how does anyone decide what to do?