By Jeff Cartwright-Smith, Ph.D.,
Vice President, GfK Market Measures
I’m talking, of course, about the thorny question of measuring “Importance.” How many times have clients proposed survey questions like this?
Using a scale from 1 to 7, where 1 means NOT AT ALL IMPORTANT and 7 means EXTREMELY IMPORTANT, how important is ___ in your choice of an agent to treat [disease condition]?
As patented blockbusters age and generics intrude, clients need to know where to focus their increasingly limited development and marketing resources. Should the focus be on addressing convenience of administration? Freedom from side effects? Duration of efficacy?
Self-reported attribute importance rating scales are very popular, but usually yield very disappointing differentiation. If we are wise in screening potential candidate attributes, we will only choose the most important questions to be rated by physicians. We can’t be surprised that they agree and tell us that, yes, all the attributes we chose to be rated ARE important! We learn little from such uniformly high importance ratings.
Attribute ranking scales have their own serious problems, including the practical issue of trying to get respondents to reliably rank 20 or 30 or more items.
Using derived importance techniques to assess importance is also popular, but we find it is equally disappointing for the complex decision process that physicians face. Correlations or regression betas between ratings of critical brand attributes and market shares are rarely above 0.20, and often negative. It is not fun to explain nonsignificant and inexplicable findings to clients!
Readers of this column will recognize this as a case for the use of a new measurement technique known as MaxDiff Scaling, invented by Dr. Jordan Louviere. Shiv Raman, Senior Vice President and Chief Marketing Scientist of GfK V2, nicely introduced this technique in the
July 2007 issue of Topline. MaxDiff Scaling is a form of self-explication, but simply asks respondents to perform a very easy task: Report the most important, and least important, of a set of four or five attributes. Here is an example from a study on seizure disorders:
Clients have previously had to take it on faith that MaxDiff results will be more useful than results from attribute rating or derived importance exercises. No longer! We recently conducted two studies on seizure disorders that had some remarkable characteristics:
- The studies were done in the same month
- Attribute importance was assessed in three different ways:
- Self-explicated importance ratings (1-7 scale)
- Derived importance (correlations)
- MaxDiff Scaling of importance
- The sample definition was the same (U.S. general neurologist and primary care physician treaters of seizure disorders)
- The patient population referenced was identical (seizure disorder patients)
- An identical set of attributes was assessed (ex.: “Rapid onset of action,” “Effective as monotherapy”)
The results were eye-opening.
Most of the ratings ranged from mean score of 5 to 6, as we expected. Which attributes were the key ones? Which were less important? The technique failed to give us real direction, and without surprise, the client was disappointed.
Interestingly, one attribute was quite different: Attribute J, “Appropriate for children (4+ years)”. Since most neuros and PCPs were known to prefer to have pediatric neurologists treat such patients, this finding was hardly a useful piece of news about these physicians.
Once again, the client was disappointed. Derived importance could not reveal much useful discrimination among the attributes.
This finding of uniformly low correlations has been replicated countless times across therapeutic areas. Derived importance analyses invariably fail in the case of correlated attributes and a complex decision process. But that defines our world!
How did MaxDiff handle this task?
The results of MaxDiff Scaling seemed to be a revelation to the client. At last, here was a method that provided differentiation and clarity of results as originally sought by the client. Mean rated importance scores conveniently sum to 100.
There is no doubt which of these attributes is considered by the physicians as the most important; no doubt which are the three most important attributes; or which are considered the least important attributes.
We are now extending MaxDiff in new ways to enable efficiently “weighing” the importance or appeal of 50 or more attributes. This task was previously nearly impossible for any methodology. MaxDiff is proving to be reliable in its results and simple to administer. We are also finding that we can use physicians’ MaxDiff scores to produce benefit segmentation for our clients at a fraction of the cost of traditional market segmentation procedures.
MaxDiff Scaling is a form of self-explication, of course, and remains subject to criticism for that reason. We still often recommend choice modeling as perhaps the psychometrically ideal way to measure prescribing factor importance, if time, sample and budget will allow for it. But the “last great unsolved problem in market research” may at last now have a simple, practical solution.
