NCL Ranks Bottom 2 In Customer Satisfaction Survey

[Cruisercl]

A couple of things -

 

The survey is really all about marketing, which often incorporates bias (in my experience).

 

The same size is small in comparison to the known population.

 

The results are statistically insignificant; but difficult to tell unless the numbers are analyzed, the study is powered and inferential instead if descriptive statistics were used. This speaks to the strength of the results.

 

I would be interested to see if they asked the equal amount of people from each cruise line or mixed the sample. (Did I miss that?)

 

JMO. :)

 

Happy cruising all!!!

[KeithJenner]

The same size is small in comparison to the known population.

 

I'm sorry to keep on about this, but I just can't help responding when people comment about sample sizes (which happens just about every time polls or surveys are discussed).

 

There are many potential issues with this survey, but sample size really isn't one of them. Margin of error calculations are basic maths, and you really don't need big samples.

 

A sample of 3,000 people, regardless of population size, gives a margin of error of +/-1.8% with a confidence level of 95%. In laymans terms that means that if a poll says that 50% of people agree with something then 19 out of 20 times the actual number in the population as a whole would be between 48.2% and 51.8%. The other one time it would be just outside that range. That is pretty good.

 

A popular sample size for political polls (which survey populations measured in hundreds of millions) is 2,000 people and samples of 1,000 are often accepted.

 

There are a number of things about this survey which may be questionable (sample selection, questioning, interpretation), but sample size isn't one of them.

[CanadaTraveller]

There are a number of things about this survey which may be questionable (sample selection, questioning, interpretation), but sample size isn't one of them.

 

 

I agree. Checked out the website an did find this quote:

 

"Results are based on measures of customer satisfaction and quality by consumers that are proprietary to J.D. Power and Associates. Awards and performance ranking are based on numerical scores, and not necessarily on statistical significance."

 

Rankings of importance are done through having the respondents rating importance rather than correlation to an overall question. Results are reported in an index. I can't see if the overall result is an index of all the indexes or has its own questions. Reliability and Validity results can not be found, which is probably due to the proprietary nature of the survey. All surveyed have had experience with the cruise line rated. So, at the end of the day, I take it with a grain of salt, but the sample size is really good.

 

What throws me is the cost and service scores for NCL.

 

Sent from my iPad using Tapatalk HD

[doones2]

I dont pay much attention to surveys although this one involved 3000 people which is quite good. We mainly base our choice on convenience and price. I have never been on Disney and dont plan to. I am way past princesses and mickey mouse and I imagine there is alot of children. I have been on celebrity, rccl, princess and ncl and I liked them all for different reasons. I dont think anyone should only cruise with one company. They are missing out

[younglegend]

just because you disagree with the results, doesn't mean the survey's methodology is flawed. are any of you are qualified to say whether or not the sample was large enough or chosen correctly? Personally, I can't imagine that a research organization like JD Power would put their reputation on the line by producing a flawed survey.

[Cruisercl]

Love, love. love the stats discussion! It totally depends on what you are looking for and its significance and purpose.

 

The numbers are what they are but the analysis is the question. Is this a validated tool for a survey? (I still the sample is not well defined.) If not, it really is a marketing tool. Irresponsibility comes to mind; not all of us think this is OK because we know better.

[younglegend]

Love, love. love the stats discussion! It totally depends on what you are looking for and its significance and purpose.

 

The numbers are what they are but the analysis is the question. Is this a validated tool for a survey? (I still the sample is not well defined.) If not, it really is a marketing tool. Irresponsibility comes to mind; not all of us think this is OK because we know better.

 

Agreed. It depends on what you are looking for and to that end the survey results shouldn't really matter if you already know what you want/like.

 

But questioning the validity of the survey (and integrity of the surveyor) only because you disagree with the results is irresponsible.

[QuattroRomeo]

KeithJenner and Canadatraveller- if the total number surveyed is within acceptable range for such statistical data, I assume according to experts in that industry, then how does one know the other variables used in a proprietary survey? It is hard to know what to think about these results. I am not a flag waver for any one particular cruise line, but the results for price and service do seem weird. I would never consider a Disney cruise, but if I did, maybe I would like the service and or food, etc. Same with lots of others regarding various lines. Does that matter? If we don't know the questions or the source of the sample it can be misleading. Which I guess is why so many of us find these things questionable. What information should the average person look for in such surveys?

[Cruisercl]

Agreed. It depends on what you are looking for and to that end the survey results shouldn't really matter if you already know what you want/like.

 

But questioning the validity of the survey (and integrity of the surveyor) only because you disagree with the results is irresponsible.

 

The article I read, did not talk about the credibility of the results and the full methodology. I really never said I disagreed with the results. IMO, when collecting research data and it is published, there is a (ethical) responsibility to the public, (in very public survey results), that it is good and non-biased data. The (my) concern that this kind of data is not represented well by the media, such as a survey report/article, like this is.

[coka]

The numbered results all look pretty close together to me which means most are more alike than different...with exception to the first and last. That is just me, though.

 

coka

[WildcatRock]

Interesting survey results. Last year on our cruise on the Star the Disney Fantasy berthed next to us in Costa Maya. Wow, what a beautiful ship! We went up on the top deck of the Star and watched the Fantasy pull away. As she did she sounded her horn and it blew " When You Wish Upon A Star". The sound resonated for miles. Incredible.....but not for me. I love NCL, and have no desire to cruise any other lines - which is probably short sighted on my part. I know no other cruise experience than NCL, and I love it. These surveys don't and won't impact my opinion and love for NCL, although I would assume it might impact some consumers. As has been discussed many times on this forum, cruising comes down to what we want our individual cruise experiences to be, and NCL is all I need.

 

 

Sent from my DROID4 using Forums mobile app

[mpk218]

Give me the answer your looking for and I will give you the right question.

500 , 3000 , 4,000,000 , 50,000,000. It does not matter. been there,Done that , and still do .

 

If you didnt make them or love them or know them , you really should't trust them ... first rule of a survey.. lol

 

mpk

[oceanseagle12]

On a 1000 point scale, all lines in the mid 800s. To me, that says more about the industry as a whole than it informs about the differences between the lines.

 

That being said, I do have a question for those talking about sample size and margin of error. Doesn't the fact customers are being asked about one line make the total sample size less relevant? Meaning that this is effectively 8 different surveys, so the number of responses for each is like 400?

[KeithJenner]

Love, love. love the stats discussion! It totally depends on what you are looking for and its significance and purpose.

 

The numbers are what they are but the analysis is the question. Is this a validated tool for a survey? (I still the sample is not well defined.) If not, it really is a marketing tool. Irresponsibility comes to mind; not all of us think this is OK because we know better.

 

As I have said many times, there are lots of potential problems with the survey. All I have been saying is that 3,000 is a adequate sample.

 

I'd also like to be clear that I am not in any way defending this survey. I understand the potential issues and personally don't take any notice of such polls. However the issue about sample sizes is a very common misconception.

[KeithJenner]

KeithJenner and Canadatraveller- if the total number surveyed is within acceptable range for such statistical data, I assume according to experts in that industry, then how does one know the other variables used in a proprietary survey? It is hard to know what to think about these results. I am not a flag waver for any one particular cruise line, but the results for price and service do seem weird. I would never consider a Disney cruise, but if I did, maybe I would like the service and or food, etc. Same with lots of others regarding various lines. Does that matter? If we don't know the questions or the source of the sample it can be misleading. Which I guess is why so many of us find these things questionable. What information should the average person look for in such surveys?

 

Unfortunately we just don't know the answers to most of your questions, so all we can do with surveys like this is be very wary.

 

The main issue (and I have no idea) is whether the company has any motives behind its data. If not then I generally find such surveys to give a reasonably accurate results overall. But I certainly wouldn't rely on them.

[KeithJenner]

On a 1000 point scale, all lines in the mid 800s. To me, that says more about the industry as a whole than it informs about the differences between the lines.

 

That being said, I do have a question for those talking about sample size and margin of error. Doesn't the fact customers are being asked about one line make the total sample size less relevant? Meaning that this is effectively 8 different surveys, so the number of responses for each is like 400?

 

Yes, absolutely.

 

We don't know enough about the methodology to say but there has to be a suspicion that some individual samples have higher margins of error.

 

As I've said, there are many potential issues with this survey.

[sparks1093]

The numbered results all look pretty close together to me which means most are more alike than different...with exception to the first and last. That is just me, though.

 

coka

 

I look at it the same way so it's not just you:).

 

So if my understanding is correct 80% of the cruising public are satisfied with the product of whichever line they sailed with. I don't see that as a bad number at all from an industry standpoint. (And of course each line can determine for itself how much it is willing to invest to raise their individual rating. I am sure they could figure out how much each additional percentage point would cost them and how much that increase would return in extra revenue.)

[jingle5616]

The numbered results all look pretty close together to me which means most are more alike than different...with exception to the first and last. That is just me, though.

 

coka

 

You hit it spot on!

[SteveH2508]

I have just found another website (the best cruise line dot com (remove spaces)) which ranks the cruise lines on things like CDC scores, Coastguard inspections, Better Business Bureau complaint resolution etc.. These seem to be more reliable and objective metrics and the scoring system is explained.

 

It tells a VERY different story (NCL ranks 3rd in the top 25).

[Mrs. Squirrel]

I just shared the JD Power survey results with my DH. We are both scratching our heads. :confused:

 

Apparently, we have lower expectations than those cruisers surveyed, because we love NCL! Our NCL cruises have been the most enjoyable out of our Celebrity, RCCL and Carnival trips. Go figure.

[kylenyc]

NCL does not rank next to last... Here's what I wrote in a related thread. Would welcome insights from any statisticians.

 

TLDR: JD Power is a for profit marketing research firm. Likely the only thing we can say is that Disney ranks above the pack (the intent of which is so JD Power is paid a license if Disney wishes to reference the survey results). We can't say how any of the other cruise lines rank in relation to each other because the results aren't statistically significant (due to sampling size and that JD Power doesn't publish margin of error or survey method).

 

I don't see the margin of error in the JD Power Cruise Line Satisfaction 2013. This makes make the report less meaningful to meaningless.

 

My statistico maff is rusty but if we take the cruise lines they marked as average (3 out of 5 dots), there's no way to conclude whether Norwegian is at the top or bottom of the pack. There's no way to know whether people are more or less satisfied among Celebrity, Princess or Norwegian because the results more than likely fall within the margin of error.

 

Example - JD Power scores for cruise lines rated "About Average" 3 of 5 circles

Celebrity 828

Princess 826

Average 824

Norwegian 817

*On a scale of 0 to 1000

 

This does not mean Norwegian was the bottom of the average. The difference between Celebrity and Norwegian scores is 1.1% which I bet is well within the margin of error. We can't even say how close these cruise lines are to being "Better than most" because JD Powers isn't revealing the specifics.

 

Further, the difference between "About Average" and "Better than most" is only 1.4%!!! ... Again to me well within the margin of error making the entire survey meaningless to me. Which means we don't know if Celebrity, Princess, Norwegian (or even Carnival) are actually ranked better or worse than each other.

 

Statistico maff subject to independent peer review.

 

P.S. It's also interesting to see JD Power's terminology. "About Average" ... not average. "Better than most" not "Above average". These are very important distinctions in statistical analysis and survey methods. For example, the results do not say "XYX" passengers are "Below Average, Average, Above Average", OR, "Very Dissatisfied, Dissatisfied, Neither, Satisfied, Very Satisfied" because I because I bet most cruise lines fall within "Average" and JD Powers wouldn't be able to publish any meaning results at all.

 

 

I'd add that the one thing that might be statistically significant is Disney coming in first to gain the 5 out of 5 (which is again interestingly labeled "Among the best"... not "The Best", "Very Good", or "Very Satisfied") And there they only scored mere 4.5% above the "Among average" pack. But that's where the real purpose comes in because now JD Power can get Disney to pay for the coveted "Ranked best in customer satisfaction according to JD Power and Associates" title and emblem.

 

"To be able to use the J.D. Power logo and to quote the survey results in advertising, companies must pay a licensing fee to J.D. Power". Follow the money.

 

So factoring in margin of error, how much better is Disney than the rest of pack? 0%, 4.5%, -4.5% compared to the average?

 

[OceanBlue123]

So you are arguing that the facts and statistics are wrong because you disagree with them? The facts do not match my opinion so the facts cannot be right? That is not the side I'd be betting on.

[Cruisercl]

NCL does not rank next to last... Here's what I wrote in a related thread. Would welcome insights from any statisticians.

 

TLDR: JD Power is a for profit marketing research firm. Likely the only thing we can say is that Disney ranks above the pack (the intent of which is so JD Power is paid a license if Disney wishes to reference the survey results). We can't say how any of the other cruise lines rank in relation to each other because the results aren't statistically significant (due to sampling size and that JD Power doesn't publish margin of error or survey method).

 

Hi Kyle!

I understand your statement because I am familiar with statistics; how they are measured, what 'statistically significant' means, and your reference to the method and the tool used by JD power. The survey is based on opinions, not facts, and the sample size (number of responses) is really small in comparison to the population of cruisers, which JD also stated with their results.

I think that if someone is not familiar with statistics and how it works there may be some misunderstanding; yes, I am sure that is it. :)

[Cruise_More_Often]

All I know is that I've cruised on HAL (our first, long time ago), Royal Caribbean and Princess, and the ONLY reason I would choose any of them for a cruise is that Norwegian doesn't go where I want to go when I want to go. To me, Norwegian has and will continue to be (unless they get rid of all their smaller ships) my first choice for cruising.

[KeithJenner]

the sample size (number of responses) is really small in comparison to the population of cruisers

 

There we go again.

 

Just a question for those of you who are dismissing this because the sample is apparently too small for a population of 14 million. If there were only 20,000 cruisers a year, would you think that 3,000 was too small a sample then?