I'm seriously considering purchasing my first MINI, an R56 S with a LSD.
#1. I know CR cautions its readers about buying the first year of a new model or models that have been completely redesigned. Given the track record of MINI / BMW, is the redesigned Mini likely to be more or less reliable than the 06 model?
For those who don't subscribe, the following is CR's overall reliability assessment by year:
"Used Car Verdicts"
- 02: "much worse than average"
- 03: "much worse than average"
- 04: "average"
- 05: "worse than average"
- 06: "average"
#2. I've read the new engine is a joint venture between BMW & Peugeot / Citroen. Is such a collaboration likely to be more or less reliable than the old BMW / Brazilian Chrysler joint venture engine?
#3. Since Peugeot & Citroen are not sold in the USA, how does the Peugeot/Citroen reputation for quality control and reliability compare to:
* Chrysler?
* Honda/Subaru/Toyota?
#4. Is the turbo-charged S likely to be any more reliable than the super-charged S?
Thanks!
Last edited by R56SMAN : Dec 11th, 2006 at 10:38 AM.
Reason: See note about links.
I wouldn't consider a word I read in CR, it's complete random sampling, and you really have no idea if that was really how they went about getting their information. The Mini is not an extremely complex car, some have been dependable, others unreliable, just as w/ any other car. Overall this is one of the most reliable vehicles on the market. Just the same it's wise to be careful when making the decision to buy a first year model, because little is known about how the car will stand up or what problems will occur. Great car no matter how you look at it.
1.1. What is unique about CR's survey? Consumer Reports' auto reliability information is unique in several respects:
We bring this data to the information marketplace with no fear or favor. Because we have no clients beyond our readers, we can report all the data, not just the autos with top results. We needn't worry about losing advertising in our magazine, so we can interpret the data with total independence.
Our subscribers tend to be well-educated and appreciative of objective, independent research, which makes them unusually qualified to provide valuable data about their experiences.
The sheer size of our reliability survey enables us to look at owners' experiences with hundreds of thousands of cars. Unlike other reliability surveys, we are able to look at car models in several variants, in some cases reporting them separately by type of engine, drive types, trim lines or other classifications.
1.2. How does CR's survey compare with other reliability surveys?
The timeframes used in collecting data also differ among surveys. J.D. Power's IQS is based on the first 90 days of ownership of new vehicles only, and its VDS (Vehicle Dependability Study) is exclusively based on 3-year-old vehicles. CR's survey asks about subscribers' experiences with their vehicles over the course of the previous 12 months and, for the first time, CR's 2006 survey covers 10 model years--from brand-new models to models that are 10 years old, providing a more complete profile of the life of a model. Our Predicted Reliability, which forecasts how well a new model is likely to hold up, is based on the cumulative experiences of owners of the three latest model years.
1.3. Where is the data from? Consumer Reports obtains its reliability data from an Annual Questionnaire that is sent to subscribers to ConsumerReports.org and Consumer Reports magazine. In all, the survey was sent to almost 6.7 million subscribers in 2006, and we received responses on about 1.3 million vehicles.
1.4. How is the survey conducted?
The survey is conducted each spring. In the questionnaire, we ask subscribers to note any problems with their cars that occurred between April 1 of the previous year and March 31 of the current year. They are asked to identify problems that they considered serious (because of cost, failure, safety, or downtime). We ask them to include problems covered by warranty, but not the ones resulting from accident damage. We also ask them not to include replacement of normal maintenance items (like brake pads, batteries, and mufflers) unless they were replaced much sooner or more often than expected. Respondents check off problems from a list of trouble areas, ranging from the engine and transmission to climate system, brakes, electrical system, and power accessories. See the full list of trouble spots.
1.5. How current is the data?
All our reliability information is completely updated annually. We begin sending out each year's survey in the spring. By late summer, we have collected and organized responses, and we complete our analysis and update the information online by November. The new information first appears in print in the Consumer Reports New Car Preview, on newsstands in November. Subsequent auto publications, such as the Used Car Buying Guide, also use this new information. In the pages of Consumer Reports, we update Predicted Reliability and Recommendations in the vehicle Ratings beginning in the road tests in the November issue. Full results are published in the following April issue of CR. All reliability information we publish is based on subscribers' experiences with cars in the one-year period immediately preceding the survey.
1.6. What expertise does CR's survey staff have? CR's Annual Questionnaire is constructed and implemented by the staff of our National Research Center. The staff of this department includes professional social scientists, some with more than 30 years of experience in constructing, conducting, and interpreting large-scale surveys. The automobile section of the survey is developed in consultation with CR's automotive engineers and statisticians to ensure that we are capturing the most important aspects of auto reliability. The data is analyzed by professional statisticians and survey analysts.
Our decisions about the construction of the survey and the analysis and interpretation of the data are continually reviewed with an eye on giving consumers the most valuable and useful information possible.
6. How accurate is CR's reliability information?
6.1. Is this a scientific survey?
There are generally two criteria that social scientists use to evaluate the quality of a survey: its validity and its reliability. Validity refers to whether the survey actually measures what it says it does. Reliability refers to whether the information generated by the survey would be repeated if the survey were to be conducted again.
We have strong evidence that our survey is both valid and reliable. The questions in the survey are designed professionally by experts in CR's National Research Center, in consultation with our automotive engineers and statisticians. Members of our survey team have more than 30 years of experience in conducting all sorts of consumer surveys. The survey uses an aided response technique that leads respondents through well-defined specific items and gives each respondent the same perspective in answering the questions. The data we report tracks well with other sources of repair and reliability information available on the market. Although we know that auto manufacturers pay close attention to our reports, they have not formally disputed our survey findings, which often identify problems that the manufacturers see in the warranty experience of their vehicles. We conduct a validation test every year and, in more than 30 years, have not found any evidence of bias. From year to year, our subscribers' reports of their problem experiences are fairly consistent; when there is a difference on a particular model, we can often attribute it to known issues with a particular component of a car.
6.2. Is the survey based on a representative sample?
A sample is considered to be representative of a population if the relevant characteristics of the population are reflected in the sample. So, considering the population of interest is critical in evaluating the quality of a sample. Our survey sample is drawn from the population of subscribers to ConsumerReports.org and to Consumer Reports magazine. While all subscribers are invited to participate in the survey, participation is voluntary, and there is always the possibility that those who respond are unique in some particular way. For example, subscribers have sometimes questioned whether those who respond are those who have a complaint to make about their cars.
To address this concern, we conduct a validation test every year. A representative sample of 8,000 to 10,000 subscribers are mailed the same questions about problem experiences with their cars at the same time as all subscribers are asked to complete the main survey. Using a combination of incentives and follow-up mailings, we attain at least a 40 to 50 percent response rate on this validation sample. The validation sample is known to be representative of the subscriber population; by comparing responses from this sample to responses of the main sample, we can assess whether the main sample is representative of the population overall. In more than 30 years we have not found any biases on any of the questions on any of the topics.
One reason for this is that our survey is an omnibus survey asking subscribers not only about their cars, but about a dozen other products, about major services they have used (such as insurance, hotels, and health plans), as well as suggestions for CR. The survey also requests participation in our board of directors election. Many subscribers return surveys reporting that they had no problems at all with their cars in the past year. This is true for all makes and models of cars. So, owners with complaints about their cars are not the only ones who return the survey.
Any survey has some sort of sampling frame that limits the people being surveyed. We choose our subscribers as our sampling frame. On average, CR subscribers tend to be more educated and affluent than the general population. With the growth of Consumer Reports online, a wider demographic range of individuals has been surveyed in recent years. However, our reliability questions do not ask respondents about their attitudes or opinions about the reliability of their cars, where one might expect different groups of individuals to have different perspectives. Instead, we ask for factual information about whether specifically defined problems occurred; these types of questions are less sensitive to the nature of the characteristics of the sample itself.
Further, our results track well with other sources of reliability information available on the market, and auto manufacturers have not formally disputed our survey findings, which often correspond to problems that the manufacturers see in the warranty experiences of the population of car owners at large.
6.3. Is the survey biased toward Japanese cars?
In our survey of CR subscribers, Japanese vehicles are popular. Also, many Japanese models have had relatively low rates of problems in our survey. But the fact that we received responses on more than 300 makes and models from nearly all domestic, European, and Korean manufacturers shows that our subscribers do not exclusively favor Japanese vehicles and that they buy a wide range of vehicles of all makes and models.
Unlike some other magazines or surveys, we do not take advertisements from any outside manufacturer, so we have no vested interests in the outcome of our survey. We have no agenda other than communicate accurate results of our survey. We do not consider country of origin in our analyses leading to our reliability ratings.
Some Japanese models in our survey have scored below average in reliability, and some American models have scored above average. Those findings provide evidence against pro-Japanese bias on the part of our subscribers.
In recent surveys, we've seen a decline in reliability in the first-year introduction of redesigned or new models from Japanese manufacturers that usually have outstanding reliability. Our subscribers report similar teething problems for new Japanese models that they report for models from other manufacturers.
For example, in our 2005 survey, the newly redesigned 2005 Acura RL, Honda Odyssey, and the newly introduced Scion tC all had only average reliability. In the 2006 survey, those models continued to have average predicted reliability. We also found that some Nissan products such as the Infiniti QX56, Nissan Armada, and Nissan Quest to be among the most unreliable new models.
Some newly introduced models from domestic manufacturers seemed to have countered the common negative first-year trend in our 2006 survey. GM's Buick Lucerne and Cadillac DTS, Ford's Fusion, Mercury Milan, and Lincoln Zephyr all had top reliability--on par with some of the most reliable models from Japanese automakers.
6.4. Your survey results do not match with my experience. Is your survey wrong?
The CR reliability information reflects the problem rate, or percentage of cars that experienced problems, across at least 100 car owners. Even in the most unreliable models, some individual car owners are lucky and experience few or no problems during the year covered by the survey. For example, in one of the worst models in our recent surveys, about 75 percent of the owners reported problems in at least one trouble area over the previous year; of course, this means that about 25 percent of owners reported no problems. Your neighbor or friend might be one of those lucky owners. Of course, the opposite can happen as well--even in a model that tends to be quite reliable, there is an occasional "lemon."
6.5. Since the average number of problems is small for most models, is Consumer Reports overemphasizing differences that may not be important?
Beyond statistical significance, we believe these differences are also meaningful to car buyers. We think that car buyers should expect a new car to be entirely problem-free in its first months or years of service. While the difference between a and a may be small, a pattern of several less-than-perfect trouble spots in a brand new car should be cause for concern and does not bode well for a model's long-term reliability. We have not yet seen a single model in our survey that is entirely problem-free. More than that, the worst new model in the 2006 survey, the redesigned for2006 Mercedes-Benz M-Class (V8), has four times as many problems as the average model, and 10 times as many problems as the Toyota Highlander Hybrid, which debuted in the same year. Those differences among models are important for car buyers to consider in choosing a car. We present these scores for trouble spots primarily to allow consumers to compare the relative incidence of problems among models. While there are no guarantees, you can improve your odds of buying a reliable car if you choose a model that has had a lower rate of problems in the past.
6.6. Some people maintain their cars differently from others. How does this affect the Ratings?
Areas of concern in late-model cars include the electrical system (starter, alternator, battery, wiring, and lights), body integrity (squeaks, rattles, and leaks), body hardware (manual locks, seats, and door mechanisms), and power equipment (power windows, locks, and seat adjustments). Maintenance does not affect those items. Problems in those areas might more likely reflect the inherent design or quality.
6.7. How do you account for mileage differences?
Vehicles with higher mileage will most likely experience more problems than vehicles of the same age with lower mileage. We adjust our analyses to minimize differences among models due to varying mileage. Our data are mileage-standardized by dividing cars of each model into groups of high, average, and low mileage, and employing the statistical technique of direct standardization.
6.8. How do you know that manufacturers don't "stuff the ballot box?"
In most other surveys that draw their samples from lists of registered car owners, the researcher can control who is mailed a survey. In the Consumer Reports survey, buying a subscription to either the magazine or to ConsumerReports.org gives you a vote on two cars. Some subscribers have wondered whether a manufacturer could just arrange to have their employees fill out questionnaires saying that their cars are reliable, as a way to influence our Ratings. Hypothetically, this is a potential weakness in our survey.
However, there are a number of ways that we can protect against this potential for fraud. For obvious reasons, we do not want to describe in detail the actions we take in this regard. We are confident that no manufacturer has succeeded.
6.9. If you state that first-year models are less reliable than later-year models, how can you still recommend some new Japanese models in their first year?
It is true that most newly introduced or redesigned models have more problems than later model years of that design. This happens even to models from the most reliable manufacturers, such as Toyota/Lexus and Honda/Acura. But despite the decline in reliability due to the new design, reliability of first-year models from those manufacturers were still at least on par with the average model and within our reliability criteria for recommendation. In the second year, those models often improve and compare well to other models of the same age.
Anything that we could say would be pure speculation as we won't know until people start throwing some serious miles on them, but the tuners on this site that have looked at the new engine seem to think it will be a reliable engine.