Developing Core Outcome Sets: The Importance of Common Standards for Clinical Studies of Specific Diseases and Conditions

Variations in the outcomes reported in clinical studies can prevent effective meta-analyses from answering important clinical questions. The development of core outcome sets has proven to be an effective solution to this problem in many areas of medicine. Practical Dermatology® spoke exclusively with Murad Alam, MD, MBA, MSCI, Vice Chair of the Department of Dermatology at Northwestern University’s Feinberg School of Medicine, and Daniel Schlessinger, MD, a Mohs Micrographic Surgery and Cosmetic Dermatology Fellow at Northwestern University, about their development and publication of core outcome sets for actinic keratosis, basal cell carcinoma, squamous cell carcinoma, melasma, facial aging, and, most recently, rosacea.

Why is the development of core outcome sets in dermatology so important?

Dr. Murad Alam: We perform clinical trials to better understand what kinds of medications, surgeries, or other therapies are effective. When a new treatment is available, we compare it to the old treatment or to doing nothing, and we try to see how beneficial it is. However, what comes out of the clinical trials are outcomes; we measure the results using particular rulers. For example, for an acne treatment, we might count how many pimples people have after using a medication, how red their faces are, how good they feel afterward, or how willing they are to go to a social engagement. There are so many ways to measure the success of a medication, and different study teams have used different measures to study the same disease. Some look at patient-reported outcomes, and some ask doctors about the results. The problem is that all of that information cannot be combined effectively to better understand the treatment options. If 20 studies evaluated 10 patients each, combining them for a more powerful study of 200 patients is not possible if they all studied different factors. You have little bits of information, but you can’t combine them into a powerful whole to make a strong statement. Core outcome sets are a way of overcoming this problem. A core outcome set is a message to investigators: When you study this disease or condition and when you do clinical trials to evaluate drugs, devices, therapies, or surgeries for it, there are a certain number of assessments that we definitely want you to do. You can do other ones also, but we want everyone to do these. For acne, for example, we may want everyone to count the number of pimples, measure the average diameter of the pimples, and ask the patient whether they feel good, bad, or medium at the end of the treatment. In that way, we would create a core group of outcome measures. In 5, 10, or maybe 15 years, we will have large pools of data for every disease type that we can use to answer very complicated questions, and we will minimize research waste. 

How does a lack of core outcome sets impact the everyday practitioner?

Dr. Daniel Schlessinger: It’s hard to trust the data; not all data sets are created equally. We rely on scientific inquiry to make determinations about what is best for patients, but if we are not measuring things in standard ways, then it is hard to know what the actual effect was. That was what we had in mind when we selected the conditions within dermatology that would be most relevant and impactful to the everyday practitioner.

What goes into developing core outcome sets?

Dr. Alam: The process is time- and resource-intensive because we need to agree on those three, four, or five outcomes that everybody in the whole world doing research on that condition should collect. Obviously, people have differing opinions. You look at the literature and you talk to a large number of people. You ask patients, you ask physicians, you ask industry scientists, you ask regulators, and then you put together a large group of patients and physicians around the globe to keep talking about it and keep having consensus meetings until you agree on a set of measures. That can take several years, and then you need to convince all the researchers to actually use them, and you need to convince the regulatory bodies to insist on them.

Did you model this project on any specific core outcome set from another segment of medicine?

Dr. Schlessinger: Core outcome sets have been created in some other fields of medicine more extensively than they have been done in dermatology. The fields of rheumatology, allergy, immunology and a few other subspecialties were used as guidance.

How many people were involved?

Dr. Schlessinger: It is hundreds of people for each one of these, especially when you take into account the fact that we are gathering groups of different types of stakeholders. We are interested not just in the opinions of physicians or researchers, but also in the opinions of patients, stakeholders at the regulatory level, stakeholders from insurers, policymakers, and anyone else who could possibly have a role in studying a particular condition. It is a large effort. 

What are the next steps? 

Dr. Schlessinger: We have figured out the most important things to measure for several different conditions—most recently, rosacea. We have also studied basal cell carcinoma, squamous cell carcinoma, actinic keratosis, facial aging, and melasma. The next step is to figure out not just what the most important outcomes to measure are, but also how to measure them. Through this first process, for example, we may have determined that quality of life is an important thing to measure in a trial involving a particular skin condition, but there may be eight different ways to measure quality of life. So, one of the next steps is translating the core outcome set into core outcome measures.

Do you expect the entities performing these studies to be receptive? 

Dr. Schlessinger: I think it will be a process because it represents change. We are basically telling some groups of people, “You may be studying this condition in a particular way, but we have come to consensus that there is a better way to do it.” Any time you are doing something like that, there will be a bit of resistance. However, what we have seen from other fields that have core outcome sets is that, once you get buy-in from a lot of people in the field, they see the benefits immediately. If you can harmonize the way that people measure things, you can combine your data and more easily compare one clinical trial to another clinical trial.

Do you anticipate challenges in obtaining buy-in because core outcome sets, by nature, would make it more difficult for large numbers of companies to all claim they are the best based on different data sets?

Dr. Alam: That is a good point. People have different interests. Companies have different interests. Even individual researchers not affiliated with companies can be resistant to doing things the same way. Part of the process of developing core outcome sets is to include a lot of these constituencies from the outset. If you are involved in developing something, you are much more likely to think it is a good outcome. It can be helpful to companies, too, because some diseases really do not have good therapies due to challenges in studying them. Manufacturers and drug developers can better demonstrate when their products make people better based on these new scales that we have developed and validated.

How important is getting the word out as much as possible right now?

Dr. Alam: That is probably the No. 1 thing. Everyone is busy. Everyone has their own objectives. It is about uptake. Just as a consultant does not want to write a report only for it to sit on a shelf, we want implementation. If no one uses a core outcome set, then it is not helping anyone. So, we need to involve people. We need people to understand what a core outcome set is. It is nothing mysterious. It is a sensible, basic, common-sense thing that, ideally, does not harm anybody. It helps everybody avoid research waste, and it really helps patients. Hopefully, over time, we can get more groups on the same page, such as regulatory agencies requesting its use, journal editors requiring it for publication, and patients or patient advocacy groups insisting on it. A number of authorities are increasingly aware that this is something that really helps everybody and improves the health of the country and the world. There is no good reason not to do it. Change is hard, but if we can convince some key policymakers and key gatekeepers to insist upon it, then it will gradually change behavior.

Can everyday practitioners get involved?

Dr. Alam: If a dermatologist or patient is interested in being involved in the development of core outcome sets, there are literally dozens of core outcome set groups that are disease-specific; you can easily look up these groups and contact them to participate.

Until these core outcome sets are widely utilized, is there anything you would recommend in the meantime to doctors when evaluating studies? 

Dr. Schlessinger: Pay attention to the methodology. We are trained on this in medical school and residency, but it is always important to dig deeper into the data and to look at how people are measuring improvements.