This week we analyzed Customer Reviews for Bluetooth Headsets. As of this date we monitor 30 products in this category and analyzed 2,977 reviews written by their customers. However some of these products have not accumulated enough reviews to produce statistically representative and accurate metrics, so we filtered them out of the competition. The second round disqualified any product that failed to meet Customer Expectations with its Functionality, Reliability or Support.
Motorola H680 2010 Piplzchoice Award winner 44.5% above average Customer Satisfaction in its Category The winners are chosen by their customers
Last weekend we have passed another milestone – we are now monitoring product reputation of over 20,000 products and analyzed over 2,000,000 customer reviews.
The next few weeks will bring new functional releases as well as changes to the structure of our website that hopefully make it easier to use. Thank you for your feedback and recommendations.
Say you’re a product manager responsible for a line of MP3 players. One of the players is not selling well, in spite of various promotional activities including two price reductions within the last six months. You still can’t find lift.
As with any product development cycle, you conducted focus groups and researched the market to determine the optimal feature set for your target audience, at a compelling price point. The research didn’t yield any unexpected or actionable results.
In addition to handling your regular workload, you have several hundred online customer reviews collected over the last 60 days to plow through. It’s vital to read these reviews but you simply don’t have the bandwidth to go over them all with an attention they require. You need an easy way to filter out relevant customer themes that provide quick, current, actionable insights from customers.
Competitive products offer almost the exact same features as your MP3 player at a similar price. You’re now working on a next- generation player but aren’t clear on what the “must have” features should be for this version. Not only is your market data ambiguous, but also it’s now stale after all this time.
Sound familiar?
Enter Amplified Analytics. Using AAI’s Product Reputation Market Intelligence Reporter (PRMIR), which is based on semantic analysis of customer reviews and behavioral economics models, product managers and key decision makers can quickly segregate and analyze key performance indicators (KPIs) like Customer Satisfaction (CSI) with a product functionality, reliability and a quality of support.
Top category selections, such as MP3 players, on the PRMIR data entry screen are, easily identifiable and simple to find. Users can see the ratio of reviews to products, using a significant product sampling (in the case of the MP3 player, 75:1). All listings are date stamped, so that users know precisely when data has been updated. In just four mouse clicks, a product manager is able to generate meaningful functionality rates for his or her product;
The PRMIR interface allows customers to make multiple selections of competing manufacturers and filter the number of reviews and ranges for several performance indicators, including Customer Satisfaction Index (CSI), Product Functionality Score (PFS), Product Reliability Score (PRS) and Product Support Score (PSS).
When recalculating the CSI factoring out this specific design issue, the MP3 player in question outscores the competition by 4.2%. An up-to-date analysis with easily importable data is available in less than 15 minutes; the entire process takes less time than a normal lunch hour. Most important, stakeholders walk away with accurate data and tangible feedback to ensure customer satisfaction and profitability of future products.
Product requirements development without true customer input is one of the most common reasons for a product failure to achieve its profitability expectations. The common practice of capturing this input is traditional market research methods of using customer panels and consumer focus groups. The results of these efforts are only as good as specific professionals conducting these studies that often lead to a very high costs, inconsistent results and unscalable processes. In my opinion, these techniques can be much more beneficial later in the Product Development process, at the Test and Customer Evaluation stage, when specific product assumptions can be validated.
Regardless of methodology you follow for development of a new product, at some point of the process you have to decide what segment of your market you are planning to compete in. That often involves a need to create and analyze a list of existing products your new product targets to displace, to identify target customers and what mechanisms to use for collection of their needs.
Careful analysis of experiences the customers had with the products you target to compete with, can provide invaluable insights into their original needs without any color commentary or interpretation. The critical, and often disregarded part, is that these people have actually spent their own money to satisfy their needs which make them uniquely qualified to provide you with very valuable information. Many of them, estimated 1%-2% total customers, have bothered to leave their comments and reviews on countless social media venues, such as retailer and manufacturer websites, user forums and customer communities.
Availability of customer generated product experience content allows all product development personnel to have direct and affordable access to understanding customer requirements.
The more intelligence you can find about your product segment of the market the higher is a certainty of your new product profitability forecast.
This week the Market Intelligence Report for Portable Vehicle GPS category shows that Garmin nuvi 260 3.5-inch Portable GPS Navigator enjoys the highest aggregate satisfaction ratings from its customers for its Functionality, Reliability and Support.
The runners up are respectively:
#2. Magellan RoadMate 1400 4.3-inch Portable GPS Navigator, and
#3. Harman Kardon GPS-810 4.3-inch Widescreen Bluetooth Portable GPS and Media Player.
You can extract reports like this for any category of products we track by registering with us at the home page and clicking on “Try Us Now” orange button.
As any business seeks to better understand customer needs and behaviors, it’s no secret that Social Media has opened more doors to CRM opportunities than ever before. Last week while reading a recent marketing blog, I was amazed to observe that the writer failed to suggest the current trend of social networking as a frontline method for creating a relationship with customers.
Social Media is providing a colossal platform allowing us to hear what our customers are saying. It is quickly becoming one of the best ways to engage a customer and gain valuable insight into their experience with our products as well as those of our competition. Are you listening?
This explosive technology could permit any business to identify competitive threats or opportunities through information that might not otherwise be detected without listening to thousands of customers. Historically, formal focus groups were utilized as the most common means of collecting this data in-person from the end user. Perhaps one could imply at this juncture that social media is quickly becoming the new focus group.
Consider for a moment that while traditional focus groups draw in customers to discuss their experiences, so are Social Networks providing the same information. Is there really a significant difference? The value of a focus group depends largely on quality of questions posed to the participants with all the biases that are incorporated into a question. The main disparity is that social media presents a very public review of a product or company’s benefits and even shortcomings. However, we must not ignore the exponential numbers of consumers who are vocalizing this valuable data. It is often more candid than any focus group could provide.
Getting connected with them is just part of the solution. Connecting & engaging within these social mediums is relatively easy. Nevertheless, just like any other ‘marketing” effort, its success is not realized without measurement. Therefore, the opportunity exists in figuring out what to do with the unstructured data.
Fortunately there is technology available to “interpret” this valuable data. Utilizing a multi-dimensional analysis, we convert various forms of feedback into an actionable plan then we take it one step further. We are examining customer ratings across the market of nearly 15,000 products (shameless self promotion). Many of the companies who have attempted their own translations had to invest very significant amounts of money into text mining implementation projects that allow only to handle feedback about their own products. With more than 1.4 million reviews, our database can deliver satisfaction scores from real world consumers about your products as well as that of your competition.
Self help author and motivational speaker, Robert Kiyosaki, was quoted last year as saying ‘I am a bit old to focus on social media now but I spend an average of two hundred thousand dollars monthly through hired employees or consultants on social media, online reputation etc’. While the use of social media as a marketing tool is still in its early stages, let’s not ignore this novel opportunity to act on customer feedback.
I found an interesting post by Jeffrey Henning today. The article is touching on emotional attachment people have for different measurement scales used in Market Research. I can see how easily it come into play as we try to find one method to fit variety of research projects. Every method or tool has it’s limitations and therefore the challenge is to find the most appropriate one for the task at hand. Jeffrey quotes Brad Borther who provides an excellent advise:
Ten-point scale: “A five-point scale is totally inappropriate for customer satisfaction studies. Why? It lacks enough granularity and robs companies of a burning desire to take corrective action. It commonly leads executives to believe that ‘80% rate us four or five; that’s great, let’s move on,’ without realizing that it simply means that 80% are at least somewhat satisfied. Further, many people will never rate anything a ‘five,’ resulting in ‘four’ including those who are really very satisfied and those who are only somewhat satisfied. To avoid this topping effect, use at least a 10-point scale and count nine and 10 ratings as fully satisfied. This will also allow easier analysis of what bottom-line effects satisfaction has, since such tools as regressions work better with a more granular score.” – Brad Bortner, principal analyst with Forrester Research, “Best Practices: Why Customer Satisfaction Studies Fail“
Since our approach to measuring Product Reputation (delta between Customer Expectations and Customer Experience) is focused on competitive position of multiple products within their category, and our method does not require to ask people to measure it, I have decided to use “0″ to “2″ balanced scale with 2 decimal points for more granularity. It is interesting how infrequently people want to challenge a value of our methodology or accuracy of our analysis, compared to the selection of the measuring scale. By now I gave up any attempts to change their mind. We arrive to the scores using our algorithms to analyze Customer comments and reviews, not by asking them to measure according to any scale, therefore is much easier for us to recalculate Product Reputation scores to appear in a customer “favorite” scale. The integrity of the finding is not compromised by the conversion.
I wish all religious wars could be settled that easily.
The input and collaboration of many creates value in most cases, but probably not in all. One of the best examples of the concept of “crowdsourcing” is Wikipedia, but there are some troubling signals that have come out of this social experiment:
More than 49,000 editors left Wikipedia’s English-language edition during the first three months of 2009, compared with only 4,900 for the same quarter a year earlier, according to the Journal, quoting Spanish researcher Felipe Ortega, who analyzes Wikipedia’s online data. Though the service still boasts about 3 million active contributors, volunteers are leaving more rapidly than new ones are joining, the Journal said.
I fancy myself as being relatively well informed, and have joined, as a volunteer a few months ago, but upon reflection saw nothing particularly valuable to contribute to the existing entries. How many in a “crowd” makes “crowdsourcing” meaningful?
Wikipedia co-founder Jimmy Wales discussed the site in an interview with Silicon.com earlier this month. With 13 million articles now written and edited by volunteers, Wales sees conflict among multiple contributors as the exception.
“We really tend to use less inflammatory words–try to stick to basic facts and so on. And that’s come about over time. You have people come together [on Wikipedia] with different viewpoints but in general they tend to be trying to work in good faith to collaborate and compromise with other people.”
Wales also pointed out that most articles are written by a small number of people.
“One of the things that’s important to know about Wikipedia is that the entries that are edited by hundreds of people are really anomalies,” he told Silicon.com.
So at what point does the wisdom of the crowd turn into madness of the mob? I am not envious of Wales as he seems to manage as explosive a process as nuclear fusion, when it comes to the emotions and egos involved, but I am very grateful that he does, as many studies have shown that Wikipedia’s authority is every bit as high, if not higher, than one of traditional encyclopedias. The accuracy is the context in which authority of an encyclopedia is judged.
The debate over the accuracy–and quality–of survey research conducted online is flaring at the moment, at least partly in response to a paper by Yeager, Krosnick, Chang, Javitz. Levendusky, Simpson and Wang: “Comparing the accuracy of RDD telephone surveys and Internet surveys conducted with probability and non-probability samples.”
In my opinion the methods employed to conduct the research are secondary to the findings, the researcher attempts to discover. This opinion usually draws very heated arguments from purists who are concerned that “biases” cannot be avoided if the research is “tainted” by pre-conceived expectations. I totally agree – biases cannot be avoided, or even tried to. Without biases the results of research is meaningless and it is a lot more useful to introduce the power of the context and some structure into the process.
Meaningful, representative and actionable results of market research are more important than its marginal accuracy.
