 |
|||
|
GfK HealthCare | July 2011 | |
|
|||
Best Practices for Sampling MCOs  By Camm Epstein, Ph.D., Vice President, Market Access Market research requires that we ask the right people the right questions. Managed care organization research requires that we ask the right people at the right organizations the right questions. The focus of this paper will be on sampling the right organizations. The Universe Market access in the U.S. is largely determined by a relatively small group of payers (about 470 MCOs and about 60 PBMs) that vary in size and in aggregate cover more than 200 million lives. The graphic below illustrates the number of MCOs in the U.S., and the size of each bubble represents the number of lives each covers. US MCOs  Beyond number and size, payer decision making varies (because of forces such as purchaser demands, provider relations, population differences). That said, MCO decision making is often internally consistent over time and across analogs. MCOs also vary in terms of service areas, plan designs, corporate structures and local market dominance. Sample Size The right number of MCOs should reflect several factors including the research questions/objectives, the desired level of confidence and, of course, time and money. Time and money often limit the sample size. Even when money is no object, many timelines do not support the additional time needed to recruit larger sample sizes. Also, there are typically diminishing returns. For example, it might take an additional week to grow a payer sample from 40 to 80 and another week to grow it from 80 to 100. Research Goals If time and money are not limitations, then the research goals should be a key driver when designing a sample of MCOs. If the goal of the research is to obtain directional insights into current and/or likely future payer behavior, as is generally the case when conducting marketing analyses and testing messages, then studying 10 to 20 MCOs is sufficient. Researching 10 MCOs will identify the common range responses and directional patterns, and 20 MCOs can boost confidence in the results and identify segments that will likely emerge through a quantitative survey. If the goal of the research is to describe the market with greater precision or identify and size payer segments, then studying 40 to 80 MCOs is warranted. Researching 40 MCOs will likely permit two behavioral segments to emerge (e.g., more and less aggressive management), and 80 MCOs can permit the identification of a smaller third segment. If the goal of the research is to predict payer behavior under different scenarios (e.g., under different market conditions, in response to different product profiles) using choice modeling, then studying 80 to 100 MCOs is warranted. Generalizable Results Nearly all payer research is based on convenience samples and the results may not be generalizable. However, probability sampling of MCOs is achievable if the universe is identified and a weighting factor (e.g., number of covered lives) is known for each organization. The size of a representative sample of MCOs then depends on the desired confidence level and confidence interval. As the table below illustrates, the sample size increases as the desired confidence level increases and/or as the desired confidence interval decreases. MCO Sampling Guidelines  It is important to note that the numbers in the table above are estimates based on the average of 20 random samples as the numbers vary for each sample drawn. It is also important to note that these numbers represent particular MCOs selected. For example, a representative sample is not simply 92 MCOs, it is the right 92 MCOs. Interestingly, the right 25 MCOs can be a representative sample (assuming an 80 percent confidence level and 10 percent confidence interval were acceptable). This is very similar to much qualitative MCO research based on 20 MCOs. The implication is that with a little more time and money, and rigorous sampling methods, MCO research can move from directional to representative. This challenges descriptions of MCO research as being qualitative simply because of the sample size and highlights the differences between payer research and patient/physician research. Whereas payer research based on 25 MCOs might be representative, research based on 25 patients or physicians is only directional. A frequently cited barrier to drawing representative samples of MCOs is the nonresponsiveness of some MCO decision makers. In fact, some decision makers are unwilling to participate in research and this makes it very difficult to study some MCOs directly. For example, Kaiser is frequently mentioned as an MCO that cannot be studied via primary research. Yet nonresponse is a problem that plagues nearly all survey research, and widely accepted methods are available to address nonresponse. For example, finding replacement organizations that match the missing organizations on key behavioral dimensions (e.g., formulary management, PA criteria) is one such approach. While this approach is clearly suboptimal to directly accessing the organization drawn in the sample, it is far superior to the common approach of simply ignoring the problem. Myths and Misconceptions To further highlight best practices associated with MCO sampling, it is useful to review some of the common myths and misconceptions. Stratification A few of the more sophisticated payer researchers attempt to achieve more representative results by stratifying payers. While sampling MCOs and weighting results based on these strata can yield results that mirror the market on the dimensions used for the stratification, it may not be aligned along other important dimensions that were not used for the stratification. Other than measures of local market dominance, firmographic data (e.g., region, size, Medicare, Medicaid and commercial product mix) are generally not predictive of MCO decision making. Stratification is not a substitute for probability sampling and may yield results that are not generalizable. Bigger Is Better Fallacy We often find some smaller payers behaving in nearly identical ways of much larger payers. Some relatively smaller payers are dominant in local markets. Smaller payers then do not only represent a particular number of covered lives, they can represent a style of decision making. Including smaller MCOs may actually enhance our insights into larger MCOs. Targeting Most manufacturers have target accounts and these target lists are remarkably similar and typically include the largest MCOs. As a result, decision makers representing these MCOs are oversampled and their nonresponse rates are typically higher. Targeting larger MCOs makes sense from a business perspective, but limiting research to those same accounts is not advisable unless their awareness and perceptions are paramount. Again, if behaviors are most important, then nontarget MCOs can provide an excellent lens through which to study target MCOs. Double Counting Many regional MCOs are part of national parent organizations. Most of these follow a national formulary or medical policy and allow regional plans to deviate from the national formulary policy to a greater or lesser extent in response to local market conditions. Interviewing or surveying multiple respondents at a regional level may, in essence, be double-counting the same organizations. It is critically important to know whether a regional MCO is part of a national organization and the extent to which decisions are made at a national level. Comingling MCOs and PBMs Another potential source of double-counting is the comingling of MCOs and pharmacy benefit managers. Whereas MCOs manage medical benefits and sometimes pharmacy benefits, PBMs manage pharmacy benefits or simply process and pay drug claims (or other administrative functions such as contracting with pharmacies and negotiating discounts and rebates with drug manufacturers). As a result, the same individual may be served by an MCO and a PBM. Without scratching beneath the surface, summary PBM stats frequently overstate their influence (i.e., they may make management decisions for only a smaller portion of the lives they serve). Panels Most payer tracking research relies on panels. Despite best efforts, these designs encounter panel attrition (sometimes referred to as experimental mortality or loss to follow-up). The organizations represented at each point in time differ because an individual participant is not available (e.g., busy, refuses, changes roles, changes jobs, retires) or an organization no longer exists (as a result of mergers and acquisitions). Panel attrition threatens the validity of comparisons over time and injects uncertainty into the interpretations of longitudinal data. In contrast, fresh representative samples are comparable over time. As the graphic below illustrates, MCO panels are inferior to fresh representative samples when conducting tracking research.  Sampling MCOs is a blend of art and science. The future of more rigorous, more scientific MCO sampling is bright. |
Want to learn more on this topic? Please contact:  |