This analysis of customer reviews for Kindle Fire vs Asus Transformer Prime shows the attributes of customer experience that are the most important to the customers and how their experience meets their expectations. Click on the image of dashboard to make it larger.
This article was originally published at CustomerThink.com and being re-posted with some updates and modifications.
Marketers used market segmentation methods for a very long time. However, as our abilities to collect and manage information continues to improve, the new methods of segmentation become available to enable more targeted marketing efforts for marketers and better products and services for consumers. One of the most commonly accepted strategies utilized is demographic segmentation based on an assumption that a specific group (based on age, gender, etc) is a primary consumer of your product or service. Sometimes this assumption is based on the product purchase history. Regardless of the validity of an assumption, it does not often provide an insight on “WHY” this demographic segment would select the product in question or “HOW” they would use it. In other words, there is a lot of guessing that has to take place or additional segmentation strategies to be deployed. In my opinion, the popularity of demographic strategy lay mostly in its low cost and ease of access as behavioral and psychographic segmentation requires a lot of research that translates into high cost and time-to-market constraints.
The advances in technology start to offer new opportunities for market segmentation based on automated analysis of customer-generated content which is becoming available with the proliferation of social media and the rise of Social Consumer. Essentially instead of assuming what demographic group would be the ideal target for our marketing efforts, we could look at a group that already expressed their interest by purchasing specific types of products or services and learn “WHAT” elements of their experience were important to them.
Joel Rubinson, one of my favorite authorities in the field, posted this on Google+ ”as I review materials for the NYU social media class I am about to teach, I believe that Facebook will lead to the end of demographic targeting for media. Of course, content consumption and sharing behavior also enable this but Facebook will be the catalyst. Why not target on interests and actions? Thoughts?”
Most companies of any size use online survey techniques in an attempt to engage their customers, but the method does not support discovery of customer perspective; it validates assumptions of the company based on questions posed and deemed important. Again, the primary driver of survey method popularity is not the quality of the output and ability to provide better market intelligence, but the cost of implementation. I would suggest there are better alternatives today to learn unbiased market segment knowledge in applications of Opinion Mining technology to unsolicited customer-generated content.
The Opinion Mining approach offers much better quality of market segment intelligence and often rivals Survey approach in terms of implementation complexity and cost. I would like to offer an example to illustrate my point. Let’s look at tablets market segment defined by a few popular products in this category; however, non-like products that compete for the same wallet share can be used to get valuable insights:
These products were selected based on their popularity that manifested itself in a number of their customer-generated content references available online in a form of customer reviews, forum comments or social networks product page messages.
The first level of Customer Intelligence gained by Opinion Mining of this customer content is a list of customer experience attributes, sorted by their importance. The importance is measured as a percentage of total number of unsolicited opinions expressed by the customers. This answers the questions – WHAT is important to the customers and HOW important that is.
|
Attribute |
Importance |
|
usability |
12.02% |
|
reliability |
10.28% |
|
quality of construction |
8.92% |
|
display |
6.21% |
|
specifications |
3.58% |
|
portability |
3.49% |
|
audio quality |
3.08% |
|
price/value |
2.64% |
|
applications |
2.18% |
|
battery |
2.17% |
|
video and camera |
1.73% |
|
customer support |
1.53% |
|
performance |
1.51% |
|
operating system |
1.29% |
|
web experience |
0.87% |
|
flash |
0.86% |
|
connectivity |
0.27% |
|
build quality |
0.24% |
|
screen resolution |
0.2% |
|
replaceable battery |
0.19% |
|
color quality |
0.15% |
The next level allows the measuring of the difference between customer expectations and their experience and measures HOW well the customers’ needs are met. We use a two-point scale to visualize that difference (0=unacceptable, 1=experience meets expectations, 2=delighted); however, the measurements can easily be converted to any scale of choice without losing their meaning or accuracy. The chart below focuses on the top four attributes of customer experience by their importance to illustrate the approach.
There are practical implications of these measurements as they reflect on marcom messaging that have created customer expectations the product needs to meet. In the example above, most of the products exceeded the expectations of their customers in attributes most important to them by a significant margin. As an illustration, I would suggest that perhaps messaging about usability of these products could leverage customer sentiment to assure consumers who are hesitant to make a purchase and increase their products market adoption. That calls for a next level of intelligence that provides an answer to WHY customers feel this way and provide a context in which they express their opinions.
Above is a very small sample to illustrate the use of words and expressions (in square brackets) people to describe their opinions, and how they are attributed to a specific element of customer experience. These words, expressions ad even quotes can be used to fortify marketing messaging. Think of the very successful marketing campaign by Tempur-Pedic. 
The flip side of the coin – early understanding of root causes of customer disappointment – can help to alleviate larger problems, turn the problem situation around or even present an opportunity for differentiation as illustrated below.
Looking deeper reveals a lot of unhappiness about compatibility:
And even deeper analysis will provide a context that is invaluable for taking an advantage of the opportunity (click on the image below to make it larger):
To sum it up – this type of market intelligence can be produced within a few hours at cost of a few hundred dollars without any installation, implementation or training investment which makes it difficult to ignore as an alternative or addition to survey and panels approach. As GPS technology thought us – multiplicity of signal sources results in better decision quality.
This analysis is based on 30,670 customer reviews of 108 tablets published online by December 24th 2011.
To insure statistical representation and accuracy of results, we have focused on 18 tablets that were reviewed at least 100 times this year. That may mean that some tablets that were introduced toward the end of the year did not qualify for this report.
We have studied before the correlation between number of reviews published online and a number of units shipped, and therefore found it important to use it for comparison.
The most customer-reviewed tablet of 2011 are Amazon Kindle Fire (3,572), Apple iPad 1 and 2 (2,302 combined) and HP TouchPad (970).
Amazon Kindle Fire announcement dominated media and not surprisingly received a disproportional number of customer feedback after it was released. It’s value proposition and content availability are highly anticipated to make long expected dent in iPad market supremacy. The number of reviews seem to predict that this is the case.
Even though Kindle Fire OS is build on the Android platform, it has sufficient proprietary layer to view it separately from a more common versions.
From the date of TouchPad introduction the WebOS customers were the most satisfied lot (1.39) and the uncertainty about its future does not seem to extinguish their enthusiasm. Apple iPad customers are very close behind at (1.35). It is interesting to note that iPad 2 version of iOS have substantially improved its overall average satisfaction score.
The Kindle Fire OS experience score falls 0.03 points behind the Android, which is well within the margin of error, while both somewhat exceeded their customers’ expectations.
Our Market Intelligence Analysis of the tablet segment indicates that the following attributes of customer experience are most important to them:
You can get more detailed explanation of Attributes and Importance by watching this short video.
In terms of overall satisfaction, Samsung Galaxy tablet (7″) has earned the top customer satisfaction rating (1.53) and Huawei Ideo 7 tablet (1.50) came within a statistical tie, while Archos 7 Home tablet (0.82) and Velocity Micro Cruz tablet (0.95) are on the very bottom of the list.
To get more specific insights into the dynamics of the tablet customer perceptions, we sampled a market segment by analyzing the most experienced (i.e., most reviewed) models representing different operating systems.
More details and customer feedback verbatim are available via access to the dynamic dashboard for this segment on request. Watch this video for navigation tips.
This analysis is based on 82,620 customer reviews of 318 smart phones published online by December 15th 2011.
To insure statistical representation and accuracy of results, we have focused on 42 smart phones that were reviewed at least 100 times this year. That may mean that some phones that were introduced toward the end of the year did not qualify for this report.
We have studied before the correlation between number of reviews published online and a number of units shipped, and therefore found it important to use it for comparison.
The most customer-reviewed phones of 2011 are HTC Thunderbolt (5,579), Apple iPhone 4-16GB (4,106) and LG Ally (2,514).
HTC got a hold on the position of the most reviewed brand in the smart phones category largely based on popularity of the Thunderbolt.
The customer’s enthusiasm for Android smart phones and the availability of a large number of models from multiple brands produced very unbalanced distribution of reviews (75%). 
However, the Android OS enthusiasm did not translate into customer satisfaction lead as Windows phone customers’ expectations were exceeded by their experience with a wider margin. One of the possible reasons is the relatively weaker support of Android by the developer’s community that translates into the availability of applications.
It appears that Nokia’s decision to migrate their phones to Window OS is a wise one considering Symbian satisfaction scores.
Our Market Intelligence Analysis of the smart phone segment indicates that the following Attributes of customer experience are most important to them:
In terms of overall satisfaction, Blackberry Style 9670 has earned the top customer satisfaction rating (1.60) and HTC Rhyme (1.59) came within a statistical tie, while Motorola Citrus (0.72) and Droid 2 Global (0.82) are on the very bottom of the list.
To get more specific insights into the dynamics of the smart phone customer perceptions, we sampled a market segment by analyzing the most experienced (i.e., most reviewed) models representing different operating systems. We picked the models that are close to each other in a number of customer reports to make it more comparable.
More details and customer feedback verbatim are available via access to the dynamic dashboard for this segment on request.
How come there often seems to be no direct connection between the things we choose to measure and the goals we are hoping to achieve? Here are a few examples:
According to Clayton Christensen, a professor in Harvard Business School and brilliant scholar of Innovation, the root of this problem is the quality of education offered in our business schools. He makes a great point illustrating how wrong choice of key metrics leads to deconstruction of enterprises and entire industries. Clayton is famous for his efforts to re-focus marketing “a job customers hire products to do” as opposed to product’s specs.
As consumers, we all know that our experience with “products” depends on many factors that are not connected to or even correlated with its specifications, functions and features. Quite often customers are more influenced by how easy it is to deal with the supplier or how reliably a product performs, or how simply and consistently it delivers the outcome we require. Yet when we try to measure customer satisfaction, we ask them to score their opinions about characteristics of the product itself. I do appreciate the elegant simplicity of NPS (Net Promoter Score) methodology and its well-documented correlation with profitability, but what specific action can it suggest to a product manager whose product earns a low score?
Steve Blank, Silicon Valley entrepreneurial marketing genius and the author of The Four Steps to the Epiphany book, seconds Christensen’s opinion about the quality of our business schools and is working on the development of an alternative curriculum that is focused on customer development as opposed to financial engineering. Blank is preaching the importance of customer involvement into a product development that appears to be a no-brainer to me, but apparently is a relatively challenging concept to most marketing professionals according to Kristin Zhivago.
The choice of measurements we make has a dramatic influence on the probability of a startup success, according to Eric Ries—a creator of the Lean Startup movement—who has very interesting thoughts on creativity and innovation. Eric thinks that we prefer to use “vanity” metrics that make us feel good instead of helping us to make quality decisions.
So it appears that according to the experts, institutional indoctrination and lack of intellectual honesty are two major reasons for the gap between organizational goals and performance measurements that negatively affect our probability to succeed in business.
I would like to suggest that our compensation system methodology is the third leg of this proverbial stool. Since a majority of the workforce is not compensated for producing results aligned with a long term goals of organizations they work for, we instead end up measuring what is easy to measure and makes us look good.
I was talking to one of my customers about her experience trying to introduce the use of metrics into the business processes she is managing. Janet is in the gourmet food marketing business and was hoping to use analytics for discovering the patterns of shoppers’ consumption of her products by the time of day, as well as an impact of promotional events on the sales results. The food business, in her words, is a very fragmented environment and even the simplest business process tends to involve a number of companies to perform.
Clear understanding and measurement of the metrics, which Janet is interested in, would bring substantial financial benefits to all of the participants in this process, and yet they passively resist any attempt of implementation. Her frustration level was rising as Janet was describing the excuses she was getting from her customers and partners. They were not saying no to her proposal and even promised to make some information available, but ultimately no progress was ever made. Let’s make it clear that a cost is not a factor, as Janet’s company offered to underwrite the implementation.
“So why do ‘go-get-them’ people usually become so passive-aggressive when the analytics are involved?” Janet asked me. This question made me look back on my experiences, and it occurred to me that they invariably are similar to Janet’s. For over decades of my business career, I was charged with development and implementation of KPI’s many times in large and small companies engaged in different industries, but the outcome is always the same – passive resistance.
There are a few business processes that universally accept and practice metrics. The most common example are Sales and Call Center processes, but anyone who has managed sales forecasting will tell you that the efforts required to drive it are very substantial.
Recent explosion in web analytics technology brought to us a myriad of products that capture, measure and present dashboards of transactional data that may correlate to specific business process performance, but are very far removed from actionable KPI metrics that most of us need to manage business. Even marginal improvement in measuring performance of advertising investment disrupted the entire industry and created new multi-billion dollar players like Google. Imagine what could be done if we could measure actual impact of a given decision on the bottom line results. However, that would not be likely to happen anytime soon because of fundamental characteristics of human behavior – we will go to extraordinary lengths to avoid personal accountability.
The numbers can shine a light on our performance and quality of our decisions that can be too bright and harsh. Our organizational structures and compensation systems are too binary, with a few exceptions, to compensate for actual performance. Too commonly we get and keep our jobs not for delivering exceptional results, but for “fitting in” and showing up on time, for being efficient and working long hours, but not necessarily effective in producing the “right” results.
The key to successful, productive adoption of analytics into organizational fabric is careful selection of only those metrics that measure elements of a process that can be proactively managed by the parties involved to their performance benefit. The fewer relevant, actionable KPI metrics that help to take meaningful actions is much better than dashboards full of charts and numbers you have no control over. Relevancy beats ease of generation and drives user adoption.
I keep struggling with the definition of what is an innovative blockbuster product (or service), and this is yet another attempt: A truly innovative product is the one that delights its customers by anticipating their needs before they knew they have them. In other words, if you want to develop a blockbuster product, you should stop trying to better serve the existing needs of your customers and instead try to discover needs that customers may not realize they have and address them.
Traditionally, companies use customer feedback to assess satisfaction with existing products and to validate product developer’s ideas for the improvements. One of the most popular methods used for collecting customer feedback are survey and panels, where the questions asked or topics moderated tend to reflect interests of product development team and focus on how customers experience their product.
I would like to pose that truly innovative product developers use a different perspective to discover the needs customers cannot articulate in controlled or moderated environment – the perspective of holistic experience of a job the customer “hired” the product in question to do.
The journey starts with the understanding of what the “job” they want to do is and what a desirable outcome is. The next step is to imagine how this whole experience can be simplified in its entirety, which may or may not involve your product. I use the word “simplified” because it is an ultimate description of improvement in a context of “desirable outcome.” Terms we usually use to describe improvements – Better, Faster, Cheaper – are traps anchoring us to the incremental changes of status quo.
The complete customer experience starts with a notion that the desired outcome can be achieved, and goes through discovery of components required, acquisition of the components and/or materials and skills all the way through a process of applying them. Your product may be just one of many in that process, but if you can make it easier to find at the conception stage, simpler to understand that it is the best alternative to get the job done at the acquisition stage, and require less skill and/or effort to operate, that will make your product a lot more successful. However, truly innovative products do often have an element of disruption that does not easily fit into organizational structures. If you are a drill product manager, and survey satisfaction of a drill purchasers, the ideas of alternative wall anchoring to hung pictures will not likely come up. However, even if it does, how does it help you or your department? I wonder if a celebrated genius of Steve Jobs could only manifest itself because he operated from above of organizational hierarchy.
The question is, “Can Customer Feedback help to create innovative products?” If you define Customer Feedback as the results of survey or other structured information-gathering method, the answer is NO. The best outcome of these exercises is reduced uncertainty about your assumptions (i.e., confirmation of what you already know). The probability of discovering an idea that could lead to the conceptualization of an innovative product is extremely low, but could be improved somewhat by allowing open-ended questions and a lot of unstructured comments.
I define Customer Feedback as any and all customer-generated content available about a product/service in any form customers chose to communicate it. That includes company and public forums, customer support notes and call transcripts, company sales notes, customer’s Facebook comments, and customer videos and reviews published online. The wider Customer Feedback “fishing” net is cast, the higher probability of innovative ideas discovery. Combine it with the right analysis methodology that does not tie you up with pre-conceived keywords and ontology, and your chances are looking even better.
This morning, I had an interesting experience. Among many emails, tweets and webinar offerings, I managed to expose myself blindly to two that focused on one challenge I have to overcome the most – talking too much and listening too little. Interestingly enough, neither of the messages was specifically targeting individual or style shortcomings, and the subject line of these presentations was not about listening skills. If this is not a moment of serendipity, then I don’t know what is. BTW it also synch with my favorite definition of serendipity: “…is when you come to look for a needle in a stack of hay, and you end up finding a farmer’s daughter.”
The first piece of content is a video from http://www.entselling.com/ that talks about challenges of entrepreneurial selling and is not focused on the selling or listening style at all. It is very good and I strongly recommend it to any startup team, but the listening piece resonated with me the most. I’ve been trained on the importance of this skill for selling many years ago, and judging by my performance at the time, I have even learned to apply it. However, as it may be obvious to people who know me, it is not one of my natural qualities 
. The more I get excited about the subject of conversation, the less patient I get with listening to my conversation partners, particularly if I think I already figured out what they are trying to communicate.
I may be right about that, but it doesn’t create a great conversation experience, nor does it make them feel that they have been heard and that I actually do understand their concerns or problems I am proposing to address with my product or service. Apparently, it is a very common problem undermining many startup founders who are understandably excited about their creations to a detriment of their potential customers’ comfort, and subsequently a sales success. Maybe I should start looking for a startup founders “shut up and listen” support group. Please let me know if such a group exists.
The second piece is even more interesting and was presented by Rebel Brown at the Defy Gravity webinar sponsored by TreeHouseInteractive. Rebel is a very dynamic and passionate speaker, and she was talking about many marketers trying to use social media as traditional content broadcasting channel. The main lesson I took out of this presentation is about a challenge of institutionalized listening that needs to become a part of personalized conversation if a brand wants to be successful in social media. I suppose that no brand will be able to survive without social engagement with their customers, as the customers are creators of a brand.
“Advertising can help you sell good products, but only your customers can help you build a great Brand!”
The challenge is in learning what is important to your customers and communicating with them about this, as opposed to focusing on your product or your brand. I think the most difficult part is to not assume that we already know what it is, and not be afraid to learn from these communications.
Rebel also made a great point about the practice of counting followers and “likes” as a result of social media efforts. I will paraphrase it here as, “Do not confuse tactical metrics with actual meaningful results.” Let’s face it – these only exist because they are easy to count. Their relevance to business outcomes is very questionable, and the only thing they help to learn is how to manipulate or game the counting mechanisms.
I don’t think a “real” marketer can be helped by a mere support group or volunteer 12-step program. Perhaps there is an opportunity for a true “rehab.” All you need is a recovering marketing celebrity lending their name to this venture.
“You cannot manage what you cannot measure.” Those are well known and accepted words of wisdom that have been taught to thousands of MBA students for decades. There is a lesser popular truth though—that measuring wrong things can really hurt your business. Definition, selection and design of appropriate, balanced and concise metrics, as well as the processes for continuous delivery of these metrics, are the key management challenges for every organization.
1. Focusing on the “wrong” metrics will create unintended results.
One of the more graphic examples of “wrong” metrics is a ratio of successful convictions, widely used to measure success or reputation of public prosecutors. This one causes US taxpayers to waste hundreds of millions of dollars in court costs, compensation for wrongful imprisonment and lost productivity every year. Essentially this metric is measuring a percentage of tried cases that result in conviction against the total number of cases tried by the prosecutor. The higher the percentage of convictions, the more “successful” the prosecutor is considered to be regardless of how well justice is served, how many lives are destroyed and how much financial damage is inflicted.
Another example is the evaluation of a Product Manager performance based on the attainment of product forecast goals. While I am familiar with a popular definition of a Product Manager’s role as a “Product CEO,” the organizational reality does not often support this definition, as product managers rarely have administrative authority to enforce their decisions and act mostly as influencers. It is intellectually dishonest to keep them accountable for a result of a sum of aggregated decisions made by a multitude of people, but most importantly it does not help to bring desired improvements in products performance.
2. Focusing on unbalanced metrics will promote bad behavior.
Performance is often measured by a singular metric, yet people are rarely expected to behave one-dimensionally. Everyone knows that a lot of digressions will be forgiven to a salesman who consistently makes his quota, even though his lack of desire and/or skill to forecast costs your company serious hits to profit margin. Imagine it is the end of a quarter and you are deeply discounting your product in a desperate attempt to make your company revenue numbers, just to see your “best” performer bringing in a “bluebird” deal you had no visibility of. He just caused you to give away profits, and you did not need to sacrifice for “please sign today” deals. A secondary measurement attached to accuracy of forecast and associated with commission structure can dramatically improve a company’s profitability.
3. Concise metrics promote action.
Conversely, the convoluted metrics are a waste of time, expense and opportunity. Many Customer Satisfaction measurements are falling into this category because they are often too general, and the best cause of action you may take is to do more studies. Most companies do not even consider competitive influences on their customer’s assessment of their satisfaction with their products or services. Unless feedback from customer analysis of every key component of customer experience is continuously conducted, and in relation to competitive options available to the customers, it is very difficult to figure out why overall Customer Satisfaction is moving higher or lower, who should take any action and what kind of action should be taken.
In conclusion, I would like to suggest that any performance metric has to be evaluated in a holistic model as it is very easy to come up with a clever way to improve one aspect of a specific performance at a detriment of the long-term well-being of the company as a whole.
The discussions on importance of innovation are all over Social Media. The calls for innovating ourselves out from the current economic malaise are coming from the President of Consumer Electronics Association to the President of the United States. In the words of Louis XIV (or was it Mel Brooks?) – “It’s good to be the King!” – for the rest of us it would be helpful to put some definitions around these terms. I do not pretend to be an expert on the subject of innovation, but I like to be specific and want to offer some ideas for discussion.
So what differentiate commercially successful product or service from the innovation?
I would like to propose that successful products gain market traction, meet their sales forecasts and generate anticipated profit margins. Innovative products re-shape the market place, create new categories, and generate blockbusting profits. Innovative products successfully defeat the competitors’ assaults for long periods of time.
Development and introduction of successful products or services is a very challenging and risky endeavor, as we are well aware.
The Recent Portfolio Management Benchmark Survey sponsored by Planview, reported that only 52.3% of products meet with commercial success, while 21.2% were “killed prior to launch”. They did not specify the type of products or industries covered by this survey, but my personal experience pegs the success ratio well under 40% mark.
The risk level for innovation is even higher. It is estimated that only 1 out 3,000 new innovative ideas becomes a commercial success. We also know that most innovative products rarely have anything to do with technological inventions, but have everything to do with the scale of market adoption. Peter Dreker, the father of modern Management Science, wrote in his book “Innovation and Entrepreneurship”, that a 15 year “gestation” period is the average time observed between the time of an original invention and the time of its commercial realization.
We all know examples of such innovations as Ford T, Microsoft Word, iPod and iPad to name a few that dominated and still dominate their product categories. These are very different products, however the thought process, methods and techniques of the people who are behind the creation of these products, are a mystery we want to discover.
I would like to propose that the key difference between really good Product Managers and the Innovators is in a way they perceive and understand the markets they target.
While a Product Manager segments the markets in terms of demographics or personae for which they develop a product, an Innovator is focused on the Customer Experience of people, who struggle to use existing products to do their chores, and interprets these struggles into the definition of innovative vision for new generation products.
In other words they concentrate on improvement of EXPERIENCE as oppose to improvement of a PRODUCT.
Amplified Analytics Blog Updates