This analysis of customer reviews for Kindle Fire vs Asus Transformer Prime shows the attributes of customer experience that are the most important to the customers and how their experience meets their expectations. Click on the image of dashboard to make it larger.
This analysis is based on 82,620 customer reviews of 318 smart phones published online by December 15th 2011.
To insure statistical representation and accuracy of results, we have focused on 42 smart phones that were reviewed at least 100 times this year. That may mean that some phones that were introduced toward the end of the year did not qualify for this report.
We have studied before the correlation between number of reviews published online and a number of units shipped, and therefore found it important to use it for comparison.
The most customer-reviewed phones of 2011 are HTC Thunderbolt (5,579), Apple iPhone 4-16GB (4,106) and LG Ally (2,514).
HTC got a hold on the position of the most reviewed brand in the smart phones category largely based on popularity of the Thunderbolt.
The customer’s enthusiasm for Android smart phones and the availability of a large number of models from multiple brands produced very unbalanced distribution of reviews (75%). 
However, the Android OS enthusiasm did not translate into customer satisfaction lead as Windows phone customers’ expectations were exceeded by their experience with a wider margin. One of the possible reasons is the relatively weaker support of Android by the developer’s community that translates into the availability of applications.
It appears that Nokia’s decision to migrate their phones to Window OS is a wise one considering Symbian satisfaction scores.
Our Market Intelligence Analysis of the smart phone segment indicates that the following Attributes of customer experience are most important to them:
In terms of overall satisfaction, Blackberry Style 9670 has earned the top customer satisfaction rating (1.60) and HTC Rhyme (1.59) came within a statistical tie, while Motorola Citrus (0.72) and Droid 2 Global (0.82) are on the very bottom of the list.
To get more specific insights into the dynamics of the smart phone customer perceptions, we sampled a market segment by analyzing the most experienced (i.e., most reviewed) models representing different operating systems. We picked the models that are close to each other in a number of customer reports to make it more comparable.
More details and customer feedback verbatim are available via access to the dynamic dashboard for this segment on request.
How come there often seems to be no direct connection between the things we choose to measure and the goals we are hoping to achieve? Here are a few examples:
According to Clayton Christensen, a professor in Harvard Business School and brilliant scholar of Innovation, the root of this problem is the quality of education offered in our business schools. He makes a great point illustrating how wrong choice of key metrics leads to deconstruction of enterprises and entire industries. Clayton is famous for his efforts to re-focus marketing “a job customers hire products to do” as opposed to product’s specs.
As consumers, we all know that our experience with “products” depends on many factors that are not connected to or even correlated with its specifications, functions and features. Quite often customers are more influenced by how easy it is to deal with the supplier or how reliably a product performs, or how simply and consistently it delivers the outcome we require. Yet when we try to measure customer satisfaction, we ask them to score their opinions about characteristics of the product itself. I do appreciate the elegant simplicity of NPS (Net Promoter Score) methodology and its well-documented correlation with profitability, but what specific action can it suggest to a product manager whose product earns a low score?
Steve Blank, Silicon Valley entrepreneurial marketing genius and the author of The Four Steps to the Epiphany book, seconds Christensen’s opinion about the quality of our business schools and is working on the development of an alternative curriculum that is focused on customer development as opposed to financial engineering. Blank is preaching the importance of customer involvement into a product development that appears to be a no-brainer to me, but apparently is a relatively challenging concept to most marketing professionals according to Kristin Zhivago.
The choice of measurements we make has a dramatic influence on the probability of a startup success, according to Eric Ries—a creator of the Lean Startup movement—who has very interesting thoughts on creativity and innovation. Eric thinks that we prefer to use “vanity” metrics that make us feel good instead of helping us to make quality decisions.
So it appears that according to the experts, institutional indoctrination and lack of intellectual honesty are two major reasons for the gap between organizational goals and performance measurements that negatively affect our probability to succeed in business.
I would like to suggest that our compensation system methodology is the third leg of this proverbial stool. Since a majority of the workforce is not compensated for producing results aligned with a long term goals of organizations they work for, we instead end up measuring what is easy to measure and makes us look good.
We are proud to announce the a new release of Opinion Mining software is now available to our customers and registered users. The release includes the following functions, features and enhancements:
I would like to thank our customers who provided valuable feedback that helped us develop these new capabilities.
I keep struggling with the definition of what is an innovative blockbuster product (or service), and this is yet another attempt: A truly innovative product is the one that delights its customers by anticipating their needs before they knew they have them. In other words, if you want to develop a blockbuster product, you should stop trying to better serve the existing needs of your customers and instead try to discover needs that customers may not realize they have and address them.
Traditionally, companies use customer feedback to assess satisfaction with existing products and to validate product developer’s ideas for the improvements. One of the most popular methods used for collecting customer feedback are survey and panels, where the questions asked or topics moderated tend to reflect interests of product development team and focus on how customers experience their product.
I would like to pose that truly innovative product developers use a different perspective to discover the needs customers cannot articulate in controlled or moderated environment – the perspective of holistic experience of a job the customer “hired” the product in question to do.
The journey starts with the understanding of what the “job” they want to do is and what a desirable outcome is. The next step is to imagine how this whole experience can be simplified in its entirety, which may or may not involve your product. I use the word “simplified” because it is an ultimate description of improvement in a context of “desirable outcome.” Terms we usually use to describe improvements – Better, Faster, Cheaper – are traps anchoring us to the incremental changes of status quo.
The complete customer experience starts with a notion that the desired outcome can be achieved, and goes through discovery of components required, acquisition of the components and/or materials and skills all the way through a process of applying them. Your product may be just one of many in that process, but if you can make it easier to find at the conception stage, simpler to understand that it is the best alternative to get the job done at the acquisition stage, and require less skill and/or effort to operate, that will make your product a lot more successful. However, truly innovative products do often have an element of disruption that does not easily fit into organizational structures. If you are a drill product manager, and survey satisfaction of a drill purchasers, the ideas of alternative wall anchoring to hung pictures will not likely come up. However, even if it does, how does it help you or your department? I wonder if a celebrated genius of Steve Jobs could only manifest itself because he operated from above of organizational hierarchy.
The question is, “Can Customer Feedback help to create innovative products?” If you define Customer Feedback as the results of survey or other structured information-gathering method, the answer is NO. The best outcome of these exercises is reduced uncertainty about your assumptions (i.e., confirmation of what you already know). The probability of discovering an idea that could lead to the conceptualization of an innovative product is extremely low, but could be improved somewhat by allowing open-ended questions and a lot of unstructured comments.
I define Customer Feedback as any and all customer-generated content available about a product/service in any form customers chose to communicate it. That includes company and public forums, customer support notes and call transcripts, company sales notes, customer’s Facebook comments, and customer videos and reviews published online. The wider Customer Feedback “fishing” net is cast, the higher probability of innovative ideas discovery. Combine it with the right analysis methodology that does not tie you up with pre-conceived keywords and ontology, and your chances are looking even better.
This is a new analysis of customer feedback, which is available online, about their smart phones. At this time we are tracking and analyzing comments from 37,110 customers on 136 mobile smart phones. I decided to filter out the phones which were not updated with new customer comments during the last 30 days to insure that these phones are still available on the market. The resulting Product Reputation report is available at Market Intelligence.
I selected the most reviewed phones for each operating system to take a close look at what attributes are important to the smart phone customers. As customers keep posting their reviews and forum comments about their experience with the phones they chose, Reliability remains the most important specific attribute that dominate the conversation as 15.22% of all opinions mined is focused on it.
Let’s face it, anybody who buys a smart phone and pays for the service expects to be able to use their phone every time they want to. Apple clearly outshines competition by exceeding customers’ expectations of Reliability by 10%. HTC-HD7 (Win7 OS “representative”) meets their customers’ expectations: however Android (HTC Thunderbolt), Symbian (NokiaN8) and RIM’s Blackberry Curve 9330 are a disappointment to customers who selected to purchase these phones.
Overall customers are satisfied with their decisions to various degrees, but Apple iPhone users are reporting that the phone exceeded their expectations by 42%. Not surprisingly they also are the most satisfied with the choice and quality of the Applications available to them, their Usability and Web Browsing Experience. Since I personally have never purchased an Apple product, nobody can accuse me in the Apple bias: however this phone has earned a remarkable reputation by managing not to disappoint its user in a single attribute of customer experience.
Nokia N8 leads customer satisfaction in Battery Life, exceeding expectations by 14%, Call Quality, Music Player experience, Sound and Video Quality. However, it also disappoints their customers where it really counts – poor Customer Support, inadequate Keyboard, Operating System experience and Web Browsing.
Below is the list of top 19 Customer Experience Attributes by their importance to the customers as they opined in their comments and reviews. Our methodology does not utilize surveys, focus groups, panels or other forms of leading questions/bias forming market research tools. I have filtered out any Attribute with importance below 0.35% that may be very valuable for Product Marketing analysis, but not very meaningful for general consumption. The complete list is available on request:
The selection of phones for the comparative analysis would vary based on criteria important to a person who conducts the research – I wanted to compare a single representative phone per operating system and you may want to find the best Android phone for example. The Attributes and their importance may vary based on such choice as customers “conversations” could yield substantially different results.
“You cannot manage what you cannot measure.” Those are well known and accepted words of wisdom that have been taught to thousands of MBA students for decades. There is a lesser popular truth though—that measuring wrong things can really hurt your business. Definition, selection and design of appropriate, balanced and concise metrics, as well as the processes for continuous delivery of these metrics, are the key management challenges for every organization.
1. Focusing on the “wrong” metrics will create unintended results.
One of the more graphic examples of “wrong” metrics is a ratio of successful convictions, widely used to measure success or reputation of public prosecutors. This one causes US taxpayers to waste hundreds of millions of dollars in court costs, compensation for wrongful imprisonment and lost productivity every year. Essentially this metric is measuring a percentage of tried cases that result in conviction against the total number of cases tried by the prosecutor. The higher the percentage of convictions, the more “successful” the prosecutor is considered to be regardless of how well justice is served, how many lives are destroyed and how much financial damage is inflicted.
Another example is the evaluation of a Product Manager performance based on the attainment of product forecast goals. While I am familiar with a popular definition of a Product Manager’s role as a “Product CEO,” the organizational reality does not often support this definition, as product managers rarely have administrative authority to enforce their decisions and act mostly as influencers. It is intellectually dishonest to keep them accountable for a result of a sum of aggregated decisions made by a multitude of people, but most importantly it does not help to bring desired improvements in products performance.
2. Focusing on unbalanced metrics will promote bad behavior.
Performance is often measured by a singular metric, yet people are rarely expected to behave one-dimensionally. Everyone knows that a lot of digressions will be forgiven to a salesman who consistently makes his quota, even though his lack of desire and/or skill to forecast costs your company serious hits to profit margin. Imagine it is the end of a quarter and you are deeply discounting your product in a desperate attempt to make your company revenue numbers, just to see your “best” performer bringing in a “bluebird” deal you had no visibility of. He just caused you to give away profits, and you did not need to sacrifice for “please sign today” deals. A secondary measurement attached to accuracy of forecast and associated with commission structure can dramatically improve a company’s profitability.
3. Concise metrics promote action.
Conversely, the convoluted metrics are a waste of time, expense and opportunity. Many Customer Satisfaction measurements are falling into this category because they are often too general, and the best cause of action you may take is to do more studies. Most companies do not even consider competitive influences on their customer’s assessment of their satisfaction with their products or services. Unless feedback from customer analysis of every key component of customer experience is continuously conducted, and in relation to competitive options available to the customers, it is very difficult to figure out why overall Customer Satisfaction is moving higher or lower, who should take any action and what kind of action should be taken.
In conclusion, I would like to suggest that any performance metric has to be evaluated in a holistic model as it is very easy to come up with a clever way to improve one aspect of a specific performance at a detriment of the long-term well-being of the company as a whole.
Social media monitoring quickly becomes a “commodity” with hundreds of companies’ rummaging through fire hose streams of communications published, re-published and re-tweeted every second of a day. Brands want to know what people think about them and are prepared to pay for this knowledge. But why is this so? What is the value of knowing that people communicate a positive sentiment about your brand today?
I would speculate that most companies make this investment without specific strategy or process on hand, and some companies do it to manage the reputation of their brands or in other words, to do PR damage control and risk mitigation. A very few do so to systematically improve their customers’ experience.
Most of Social Media chatter has relatively low value for opinion mining efforts, which need to be given attention if you want to extract actionable knowledge for systematic change.
Furthermore, it is important to understand the differences between the types of communications that use Social Media channels:
The opinion mining operation that is focused on the Voice of the Customer “ore” delivers significantly higher yield compared to the overall Social Media buzz in terms of actionable knowledge. It is possible because it provides a very close correlation to specific products and often describes holistic customer experience with these products. I refer to “holistic customer experience” in the context of the customer’s effort required to achieve a desired outcome. An example of a desirable outcome is a new roof for the house or a quality audio experience while exercising in a gym.
We consider all steps – from the initial purchase research to the selection, purchase, delivery and setup, and to a completed realization of the desired outcome – measure it as a difference between the customer’s expectations and their perception of reality (their actual experience). Examples of VOC “ore” include—in ascending order—customer forums, blogs and customer reviews published online.
Recent Forrester’s research found that 14% of executives surveyed said that their companies don’t solicit customer feedback at all, while 56% of the respondents said—or were not sure if—their companies do not have a formal VOC program. However, the most shocking finding is that nearly one out of every four executives said that they seldom or never use customer feedback to change a business process (source: http://www.mediapost.com/publications/?fa=Articles.showArticle&art_aid=142811).
The Temkin Group research identified it as the one of Top 10 Customer Experience Incompetencies, as shown in the table on the left.
There is a good reason why so many companies find it difficult to mine Social Media for improving customer experience: most content generated by customers is unusable by corporate information systems that are built to process structured data.
The following steps have to be taken to address the big challenge of translating seemingly anecdotal evidence into a scalable, corporate process:
PHT95C82Y63H
The tablets market segment is fun to watch. While there is no doubt that Apple “owns” the segment it is interesting to note that it has the lowest satisfaction score compared to the competition.
Given the enthusiasm of Apple fans and popularity of the original iPad, I wonder if the bar was set too high for many iPad 2 purchasers. Digging deeper into the details of Customer Experiences we can see that a lot of negative comments are focused on quality of the camera embedded into the tablet.
Indeed the “focused” listening provides specific metrics that show difference between customer expectations and their actual experience with this attribute of Customer Experience.
The analysis shows that Blackberry Playbook is a clear leader when it comes to Customer Experience with picture quality. Considering widely held belief (which I do not share) that Apple does not do market research it would be interesting to see if they address the camera/picture quality issue in the next edition of this popular product.
We used Opinion Miner® software to analyze 1,103 customer generated reviews, published online before May, 29 2011, of the tablets listed above to generate these findings. The scores are calculated to the two point scale from 0 (unacceptable) to 2 (delighted) with 1=100% satisfied (i.e. experience matches expectations).
Amplified Analytics Blog Updates