Huge downturn in CC reviewer scores (this one's for the nerds)

[hankandteri]

This one's for the other nerds on this board. If you don't like numbers, you might want to sit this one out.

 

I wondered how actual cruisers were responding to the changes Celebrity management has implemented since August, and I was pretty sure Celebrity wouldn't share that with me, so I looked at the most recent reviews here on CC. Bottom line is you can see the reaction to the changes instantaneously in the reviewer scores and those new scores are pretty ugly.

 

Celebrity's average all-time rating from Cruise Critic reviewers based on 19,000+ reviews is 4.1 out of 5, so very good. Other mass market lines like Carnival, HAL, Royal and Princess hover around 3.8 or 3.9. NCL and MSC are a bit lower (3.6 and 3.3 respectively). For comparison, Viking, a premium line, is 4.6.

 

I looked at the passenger scores for the last 60 Celebrity reviews posted on CC, (all the reviews from September and the last few posted in August), and the average score for that time period had fallen from the 4.1 baseline to 3.2. That's may not sound like a lot, but it's a brutal drop, and puts X below MSC's overall rating, so mediocre at best. Sixty reviews is a decent sample size, but I didn't have the time or interest to keep going to determine exactly when the dropoff in scores began or what the curve looks like over a larger timeframe.

 

That 3.2 average score is way lower than the overall scores for any other mass market line and on a different planet from the premium and luxury lines! To be fair, I didn't check the recent review scores for any other line to see if they had also dropped precipitously recently. I bet a few others have also had a measurable drop (looking at you NCL, Princess, and Royal).

 

A lot of the CC reviews are posted by first time cruisers and new-to-Celebrity cruisers, so it's not just loyalists reflexively opposed to change contributing to the ugly number.

 

This feedback has to be showing up in their current post cruise surveys, so if anyone in Miami knows basic math, Celebrity must know that their passengers are less than delighted by the recent changes they have made.

 

On a related note, is it time for CC to take a look at all the five star ratings they have assigned to the ships in Celebrity's fleet given the huge disconnect between the official ratings and those of their current passengers?

[Rocky4]

Wow... nicely done.  I love facts versus emotional rants.  It looks like a trend starting and will be interesting to see how it holds up and whether it is directed at one class of ship or cabin.

[~Jewels]

Sad trend, but appears to match what I have been reading here and via social media. Not what it once was, actually quite far from it!

[CHEZMARYLOU]

Thanks. We have 25 cruises with Celebrity since 1997 and our ratings mirror your numbers.

[Jeremiah1212]

Two things...

 

Comparables to the competition would be poignant. I don't have time to do it but I would imagine all of them have taken a dip recently. 

 

In direct contrast, Celebrity says their owns NPS is higher than it has even been. 

[~Jewels]

1 minute ago, Jeremiah1212 said:

In direct contrast, Celebrity says their owns NPS is higher than it has even been. 

They cannot keep track of anything, so I doubt they could possibly back that notion up with real data. Perhaps a segment of respondents like new to X passengers only.

[Guest ldtr]

Just now, hankandteri said:

This one's for the other nerds on this board. If you don't like numbers, you might want to sit this one out.

 

I wondered how actual cruisers were responding to the changes Celebrity management has implemented since August, and I was pretty sure Celebrity wouldn't share that with me, so I looked at the most recent reviews here on CC. Bottom line is you can see the reaction to the changes instantaneously in the reviewer scores and those new scores are pretty ugly.

 

Celebrity's average all-time rating from Cruise Critic reviewers based on 19,000+ reviews is 4.1 out of 5, so very good. Other mass market lines like Carnival, HAL, Royal and Princess hover around 3.8 or 3.9. NCL and MSC are a bit lower (3.6 and 3.3 respectively). For comparison, Viking, a premium line, is 4.6.

 

I looked at the passenger scores for the last 60 Celebrity reviews posted on CC, (all the reviews from September and the last few posted in August), and the average score for that time period had fallen from the 4.1 baseline to 3.2. That's may not sound like a lot, but it's a brutal drop, and puts X below MSC's overall rating, so mediocre at best. Sixty reviews is a decent sample size, but I didn't have the time or interest to keep going to determine exactly when the dropoff in scores began or what the curve looks like over a larger timeframe.

 

That 3.2 average score is way lower than the overall scores for any other mass market line and on a different planet from the premium and luxury lines! To be fair, I didn't check the recent review scores for any other line to see if they had also dropped precipitously recently. I bet a few others have also had a measurable drop (looking at you NCL, Princess, and Royal).

 

A lot of the CC reviews are posted by first time cruisers and new-to-Celebrity cruisers, so it's not just loyalists reflexively opposed to change contributing to the ugly number.

 

This feedback has to be showing up in their current post cruise surveys, so if anyone in Miami knows basic math, Celebrity must know that their passengers are less than delighted by the recent changes they have made.

 

On a related note, is it time for CC to take a look at all the five star ratings they have assigned to the ships in Celebrity's fleet given the huge disconnect between the official ratings and those of their current passengers?

an apples to oranges comparison. You need to compare the recent Celebrity scores, to the scores for the other cruise lines for the same period. Comparing recent for one to historical for others is not good practice.

 

I expect that you will find all lines have dropped in the same period.

[Jeremiah1212]

6 minutes ago, ~Jewels said:

They cannot keep track of anything, so I doubt they could possibly back that notion up with real data. Perhaps a segment of respondents like new to X passengers only.

 

NPS is real data. And it's very simple data. I'm pretty sure they can figure it out. 

[hankandteri]

2 minutes ago, Jeremiah1212 said:

Two things...

 

Comparables to the competition would be poignant. I don't have time to do it but I would imagine all of them have taken a dip recently. 

 

In direct contrast, Celebrity says their owns NPS is higher than it has even been. 

 

I agree with your first point (to a degree) and I said so in my post. If someone had the time and energy it would be interesting to see how the other guys are doing right now compared to their overall ratings. My gut tells me that lines like Carnival, which are actually enhancing their offerings, are doing pretty well while lines like NCL, which are going down the same path as Celebrity, would also have significant dropoffs.

 

As for the second point, if X is saying that, I don't believe them. They would have to be manipulating the data in ways that obscures reality without resorting to lying. Trust me, I know all the tricks and have had to employ them myself over the years.  I'm convinced the CC review scores are painting a very clear picture of what the customer response really is.

[Jeremiah1212]

2 minutes ago, hankandteri said:

As for the second point, if X is saying that, I don't believe them. They would have to be manipulating the data in ways that obscures reality without resorting to lying. Trust me, I know all the tricks and have had to employ them myself over the years.  I'm convinced the CC review scores are painting a very clear picture of what the customer response really is.

 

Does it though? How many people go out of their way to provide a glowing review? But if they have an axe to grind...they'll take any attention they can get. 

[hankandteri]

5 minutes ago, Jeremiah1212 said:

 

Does it though? How many people go out of their way to provide a glowing review? But if they have an axe to grind...they'll take any attention they can get. 

Comparing all reviews to recent reviews is apples to apples in that regard. You'd have axe grinders and glowers in both cohorts.

[Georgia_Peaches]

6 minutes ago, ldtr said:

an apples to oranges comparison. You need to compare the recent Celebrity scores, to the scores for the other cruise lines for the same period. Comparing recent for one to historical for others is not good practice.

 

I expect that you will find all lines have dropped in the same period.

If I'm understanding correctly, the OP compared scores of the last 60 reviewers to the average historical rating for Celebrity and found a drop in overall rating scores for Celebrity.   The next set of data, should someone be inclined, could look at the most recent 60 reviewers of MSC, RCL, NCL, etc and compare the same thing...has the over all rating score given on _____ cruise line dropped, gone up, remained the same since _____ time frame.  Once the data from several lines are collected then one could begin to draw conclusions as to whether it's just Celebrity who has dropped of if it appears to be an industry wide trend.  Maybe I misunderstood you point (or OP's) but looking at changes in data over a period of time for a given cruise line is not apples to oranges.

[prmssk]

9 minutes ago, hankandteri said:

As for the second point, if X is saying that, I don't believe them. They would have to be manipulating the data in ways that obscures reality without resorting to lying. Trust me, I know all the tricks and have had to employ them myself over the years.  I'm convinced the CC review scores are painting a very clear picture of what the customer response really is.

Cruise critic represents a fairly small minority of guests sailing.  I don't know that it is truly representative of how passengers as a whole feel about Celebrity and other lines.

 

And I agree with previous points that I'm not sure that it is very relevant to compare Celebrity's recent ratings to historical ratings of other lines.  It is an apples to oranges comparison that doesn't at all reflect the reality that ALL cruise lines (including Carnival) have been making cuts, changes, adding fees, etc.

[Georgia_Peaches]

OP, I'm definitely a data nerd 😎...so thank you!  Agree that 60 reviews is an OK sample size but there's not really enough data points (star rating alone) to determine if the change in over all rating is linked to recent changes implemented by Celebrity.  No doubt you know (if you are a data nerd like me) that correlation does not necessarily equal causation. But I love what you've done and find this extremely fascinating!

[hankandteri]

13 minutes ago, ldtr said:

an apples to oranges comparison. You need to compare the recent Celebrity scores, to the scores for the other cruise lines for the same period. Comparing recent for one to historical for others is not good practice.

 

I expect that you will find all lines have dropped in the same period.

Again, I admitted that I didn't compare current vs. historical averages for any other line in my initial report. I was only concerned about recent Celebrity reviews.

 

Just for fun, I just did the same exercise for Carnival and HAL (last 60 reviews vs. historical average) Carnival also dropped and by way more than I was expecting. They went from 3.8 to 3.0--almost as much as X dropped. HAL, on the other hand, went from 3.9 to 3.8, which isn't much of a change at all.

[hankandteri]

10 minutes ago, Georgia_Peaches said:

If I'm understanding correctly, the OP compared scores of the last 60 reviewers to the average historical rating for Celebrity and found a drop in overall rating scores for Celebrity.   The next set of data, should someone be inclined, could look at the most recent 60 reviewers of MSC, RCL, NCL, etc and compare the same thing...has the over all rating score given on _____ cruise line dropped, gone up, remained the same since _____ time frame.  Once the data from several lines are collected then one could begin to draw conclusions as to whether it's just Celebrity who has dropped of if it appears to be an industry wide trend.  Maybe I misunderstood you point (or OP's) but looking at changes in data over a period of time for a given cruise line is not apples to oranges.

I just did Carnival (3.8 to 3.0) and HAL (3.9 to 3.8)

 

 Any other nerd here can do what I did for any cruise line. It only takes a couple of minutes. Cruise Critic loads 10 reviews on each screen so all you need to do is add the total number of dots for the first six screens and divide by 60 to get the current score I was measuring. 

[Georgia_Peaches]

10 minutes ago, prmssk said:

Cruise critic represents a fairly small minority of guests sailing.  I don't know that it is truly representative of how passengers as a whole feel about Celebrity and other lines.

Agree.  In fact, I've met cruisers who say they are avid but have never heard of Cruise Critic.  To really cast a wider net for data purposes one would have to look at other reviewing platforms that offer a rating system similar to CC.  

 

But, I understand what the OP was looking at and it is indicative of something going on with how at least CC consumers are feeling.

[prmssk]

3 minutes ago, hankandteri said:

Just for fun, I just did the same exercise for Carnival and HAL (last 60 reviews vs. historical average) Carnival also dropped and by way more than I was expecting. They went from 3.8 to 3.0--almost as much as X dropped. HAL, on the other hand, went from 3.9 to 3.8, which isn't much of a change at all.

All the cruise lines have been making changes on a different schedule.  It has been interesting to switch between the cruise critic boards to see which board has the most negative threads at that moment (something that is constantly changing).  If you want to truly use data to find some trends both within a cruise line and as a comparison to other cruise lines, I think you need to be looking at a much larger sample over a greater time period.  5-6 weeks of time isn't very much time in the grand scheme of cruising.

[Georgia_Peaches]

1 minute ago, hankandteri said:

I just did Carnival (3.8 to 3.0) and HAL (3.9 to 3.8)

 

 Any other nerd here can do what I did for any cruise line. It only takes a couple of minutes. Cruise Critic loads 10 reviews on each screen so all you need to do is add the total number of dots for the first six screens and divide by 60 to get the current score I was measuring. 

Exactly!  It would not be difficult to collect the data across lines.  I've never thought to do it but do find it interesting.  

[Georgia_Peaches]

2 minutes ago, prmssk said:

All the cruise lines have been making changes on a different schedule.

I remember that Carnival and NCL were among the first to implement once a day cabin service.  That was a while ago...so indeed, changes are coming a different times...which no doubt impact the ratings.

[hankandteri]

Virgin Voyages has a historical average of 4.1 and their most recent 60 reviews averaged 4.5, so a significant uptick in recent passenger satisfaction. Not every cruise line is disappointing their customers right now.

Edited October 10, 2023 by hankandteri

[Georgia_Peaches]

2 minutes ago, hankandteri said:

Virgin Voyages has a historical average of 4.1 and their most recent 60 reviews averaged 4.5, so a significant uptick in recent passenger satisfaction.

I wonder how many of those reviewers jumped ship from Celebrity and other lines due to changes?  

[hcat]

Unless Celebrity managers will receive and utilize these stats..not sure of its use... Folks not liking X right now  will feel validated. Others will see flaws in the methodology, sample etc.

 

Reviews, feedback and posts are what we go by esp from "trusted"  Cruise Critic  posters.   And  hopefully we will soon have our turn.

 

Does make for interesting conversation..!

 

 

[hcat]

7 minutes ago, hankandteri said:

Virgin Voyages has a historical average of 4.1 and their most recent 60 reviews averaged 4.5, so a significant uptick in recent passenger satisfaction. Not every cruise line is disappointing their customers right now.

..how will this help improve X?

[hankandteri]

23 minutes ago, Georgia_Peaches said:

OP, I'm definitely a data nerd 😎...so thank you!  Agree that 60 reviews is an OK sample size but there's not really enough data points (star rating alone) to determine if the change in over all rating is linked to recent changes implemented by Celebrity.  No doubt you know (if you are a data nerd like me) that correlation does not necessarily equal causation. But I love what you've done and find this extremely fascinating!

Yes!!!!! You totally get it. This was simply a quick and dirty exercise to see if anything popped out, and it did. I'll leave it to others to determine the reasons and meaning (if any). If anyone really cared, they could track these results over time and also check against potential variables like seasonality of reviews (is the average review higher in some seasons across all years?) and all kinds of other things. I didn't do any of that; but if I were in the executive suites in Miami, I would assign a bright intern to the task of getting all this data for X and their major competitors and start building out a spreadsheet with some serious crosstabs built in.