## CMA Software - Comparisons

#### How does CMA compare to Revman?

#### Data entry is much easier with CMA.

The mechanics of data entry are much simpler in CMA – you work with a spreadsheet interface, and can copy-and paste-data as easily as you could in Excel. By contrast, the data entry process in Revman requires the user to set up tables and comparisons before starting data entry.

#### CMA will compute the effect sizes automatically.

In every meta-analysis you start with the published summary data for each study and compute the treatment effect (or effect size). For example, if a study reports the number of events in each group you might compute the odds ratio. Or, if a study reports means and standard deviations you might compute the standardized mean difference.

Revman will accept summary data in only two formats – events and sample size, or means and standard deviations. If any studies provide data in another format (such as odds ratio and confidence intervals) you would need to compute the effect sizes and variances manually for those studies. By contrast, CMA allows you to enter data in more than 100 formats, and will compute the effect size and variance for all of these formats. Equally important, Revman requires that data for all studies be entered using the same format. By contrast, with CMA you can enter data for each study in its own format, and use as many formats as needed in the same analysis. CMA also supports a much wider range of effect sizes than Revman.

#### CMA is able to create a customized, high-resolution forest plot.

The forest plot in Revman offers few options for customization. By contrast, CMA allows the user full control over all elements in the forest plot, will create scalable plots (that print at the highest resolution possible for the printer or journal), and allows the user to control the color for every element on the plot. CMA also allows one-click export to other programs such as PowerPoint™ and Word™.

#### Reports

CMA will create a two-page text that reports all the statistics and can serve as a template for publication. It also offers the option to annotate the report with a discussion of what each statistic means, what assumptions are made in computing the statistic, and how to interpret that statistic properly. The program will insert references and you can export the report to Word with one click. Click here to see a sample report.

#### Prediction Intervals

All software will compute the confidence interval. This tells us how precisely we have estimated the mean effect size but says nothing about how widely the effect size varies across studies. This is the domain of the prediction interval, and most publication guidelines for meta-analysis now encourage researchers to report the prediction interval. CMA allows you to include the prediction interval on the forest plot. Additionally, it is the only program that will create a plot that displays the entire distribution of true effects. This plot can be exported to Word or PowerPoint.

#### Video Tutorials

CMA includes video tutorials. These tutorials use case studies that show how to perform an analysis from start to finish and how to report the results. Importantly, the videos explain the logic of each step in the context of the larger analysis. You can watch a video from start to finish, to learn about the entire process. Or, you can jump to the part of the video that is relevant to a specific part of the program.

#### Avoiding Common Mistakes

The vast majority of meta-analyses submitted for publication include mistakes in interpreting the statistics. CMA includes a link to PDFs that discuss these common (and sometimes serious) mistakes in detail. And, the PDFs explain how to avoid these mistakes in your analysis.

#### Advanced Functions

CMA allows you to assess the impact of moderator variables. Use analysis of variance to compare the treatment effect across groups (“Is the treatment more effective for acute patients than for chronic patients?”). Use meta-regression to assess the impact of continuous moderators (“Does the treatment effect increase with dosage?”). To assess the potential impact of publication bias CMA includes an array of functions including a funnel plot, where Revman includes only the funnel plot. CMA will run a cumulative meta-analysis to show how the evidence has shifted over time. It will also run a one-study removed analysis to show the impact of each study on the combined effect.

#### Does CMA offer the same formulas as Revman?

Yes. The development team for CMA includes some of the same people responsible for the development of Revman. CMA includes all of the same computational formulas, has been validated against Revman and provides exactly the same results (see documentation). CMA offers additional options as well, but includes a button to "Use the same options as Revman," which sets all options to match Revman.

##### Comprehensive Meta-Analysis

"Although my centre develops the statistical software the whole Cochrane Collaboration uses for its systematic reviews and meta‐analyses (Review Manager), I have often used Comprehensive Meta‐ Analysis for my own research projects, as it is easy to import data from Excel, to use effect modifiers, and also because it allows meta‐regression and produces bubble graphs that are easy to work with in Word or PowerPoint."

**Peter C. Gøtzsche - Professor, Director, MD, DrMedSci, MSc, Nordic Cochrane Centre, Copenhagen, Denmark**

"Your software is very convenient and useful for systematic reviewers. I used this software to conduct Cochrane systematic review and also helped several postgraduates at University of Ottawa for their research on environmental toxicology and epidemiology. Their manuscripts were all accepted by the internal peer‐reviewed journals."

**Senior Epidemiologist - Blood Safety Surveillance and Health Care, Acquired Infection Division Centre for Communicable Disease and Infection Control, Public Health Agency of Canada, Ottawa, ON**

"I have used Comprehensive Meta‐Analysis software within a course on research methods for doctoral students in nursing. With only a 90‐minute introduction, students were ready to begin data entry and to conduct basic analyses. Both they and I were pleased with its intuitive interface and with the clarity of its output. Additionally, Biostat, Inc. has provided excellent support in our efforts to employ the software in a classroom setting."

**Stephen J. Walsh, Sc.D. - Associate Professor School of Nursing, University of Connecticut**

#### How does CMA compare to R?

CMA is a program developed specifically for meta-analysis. As such, it includes functions to automatically compute effect sizes, to perform basic and advanced meta-analyses, and to create publication quality graphics.

R is a general purpose statistical package. While R has no intrinsic support for meta-analysis, various experts have written procedures for basic analysis, for cumulative analysis, for meta-regression, for publication bias, and more. Since the functionality of R and CMA are comparable, the main difference is in ease of use and in the options for customizing the output.

R is a command-driven language, which means that you type commands, or use a dialog box to create commands, which are then submitted to the program. CMA is a menu-driven program, similar to Excel™.

#### Computing effect sizes

In R, you need to compute the effect size and variance for each study and then build a database of these effects prior to running the meta-analysis. If different studies provide data in different formats you would need to compute the effect sizes and variances using various functions. By contrast, CMA allows you to enter data in more than 100 formats, and will compute the effect size and variance for all of these formats. With CMA you can enter data for each study in its own format, and use as many formats as needed in the same analysis.

#### Analyses

R computes values and sends these to a DOS-like window for viewing. In CMA the analysis screen is interactive – you can use the screen interactively to explore the impact of different studies, the effect of alternate weighting schemes, and so on.

#### Forest plots

To create a forest plot in R you need to work with code that is not terribly intuitive. By contrast, the forest plot in CMA can be customized extensively.

#### Reports

CMA will create a two-page text that reports all the statistics and can serve as a template for publication. It also offers the option to annotate the report with a discussion of what each statistic means, what assumptions are made in computing the statistic, and how to interpret that statistic properly. The program will insert references and you can export the report to Word with one click. Click here to see a sample report.

#### Prediction Intervals

All software will compute the confidence interval. This tells us how precisely we have estimated the mean effect size but says nothing about how widely the effect size varies across studies. This is the domain of the prediction interval, and most publication guidelines for meta-analysis now encourage researchers to report the prediction interval. CMA allows you to include the prediction interval on the forest plot. Additionally, it is the only program that will create a plot that displays the entire distribution of true effects. This plot can be exported to Word or PowerPoint.

#### Video Tutorials

CMA includes video tutorials. These tutorials use case studies that show how to perform an analysis from start to finish and how to report the results. Importantly, the videos explain the logic of each step in the context of the larger analysis. You can watch a video from start to finish, to learn about the entire process. Or, you can jump to the part of the video that is relevant to a specific part of the program.

#### Avoiding Common Mistakes

The vast majority of meta-analyses submitted for publication include mistakes in interpreting the statistics. CMA includes a link to PDFs that discuss these common (and sometimes serious) mistakes in detail. And, the PDFs explain how to avoid these mistakes in your analysis.

#### Does CMA offer the same formulas as R?

Yes. CMA includes all of the same computational formulas, was validated against R and provides exactly the same results (see documentation).

#### Can CMA import data from R?

Yes. You can open the data sheet in R, copy the data onto the Windows clipboard, and then paste it into CMA. Then you tell CMA what kind of data is located in each column. The process takes only moments to complete.

##### Comprehensive Meta-Analysis

"I have used Comprehensive Meta‐Analysis II for the last 6 years to assist with a number of published meta‐analytic studies in the behavioral sciences. I have found the program to be flexible and powerful. I have also used CMA II in two graduate seminars focused on meta‐analysis. The students are able to learn the program quickly and appreciate how it structures their data sets and prevents some common mistakes made in meta‐analysis."

**Alan J. Hawkins, Ph.D. - Professor of Family Life, Brigham Young University, Provo, UT**

"Have had a great experience with CMA. Not only an intuitively simple‐to‐use, yet powerful, research tool, but also a great way to teach and introduce students to meta‐analysis."

**Charles DiMaggio, PhD - Associate Clinical Professor, Columbia University, Departments of Anesthesiology and Epidemiology, New York, NY**

#### How does CMA compare to Stata?

CMA is a program developed specifically for meta-analysis. As such, it includes functions to automatically compute effect sizes, to perform basic and advanced meta-analyses, and to create publication quality graphics.

Stata is a general purpose statistical package. The functionality of Stata and CMA are comparable. The main difference is in ease of use and in the options for customizing the output. Stata is a command-driven language, which means that you type commands, or use a dialog box to create commands, which are then submitted to the program. CMA is a menu-driven program, similar to Excel™.

#### Computing effect sizes

Stata will accept summary data in only three formats – events and sample size, means and standard deviations, or (in some cases) point estimate and confidence interval. If any studies provide data in another format you would need to compute the effect sizes and variances manually or by writing code. By contrast, CMA allows you to enter data in more than 100 formats, and will compute the effect size and variance for all of these formats. Equally important, Stata requires that data for all studies be entered using the same format. By contrast, with CMA you can enter data for each study in its own format, and use as many formats as needed in the same analysis.

#### Analyses

Stata computes values and sends these to a DOS-like window for viewing. In CMA the analysis screen is interactive – you can use the screen interactively to explore the impact of different studies, the effect of alternate weighting schemes, and so on.

#### Forest plots

Stata’s forest plot offers few options for customization. It includes a column for the study name and a symbol representing the point estimate and confidence interval. By contrast, the forest plot in CMA can be customized extensively.

#### Reports

CMA will create a two-page text that reports all the statistics and can serve as a template for publication. It also offers the option to annotate the report with a discussion of what each statistic means, what assumptions are made in computing the statistic, and how to interpret that statistic properly. The program will insert references and you can export the report to Word with one click. Click here to see a sample report.

#### Prediction Intervals

All software will compute the confidence interval. This tells us how precisely we have estimated the mean effect size but says nothing about how widely the effect size varies across studies. This is the domain of the prediction interval, and most publication guidelines for meta-analysis now encourage researchers to report the prediction interval. CMA allows you to include the prediction interval on the forest plot. Additionally, it is the only program that will create a plot that displays the entire distribution of true effects. This plot can be exported to Word or PowerPoint.

#### Video Tutorials

CMA includes video tutorials. These tutorials use case studies that show how to perform an analysis from start to finish and how to report the results. Importantly, the videos explain the logic of each step in the context of the larger analysis. You can watch a video from start to finish, to learn about the entire process. Or, you can jump to the part of the video that is relevant to a specific part of the program.

#### Avoiding Common Mistakes

The vast majority of meta-analyses submitted for publication include mistakes in interpreting the statistics. CMA includes a link to PDFs that discuss these common (and sometimes serious) mistakes in detail. And, the PDFs explain how to avoid these mistakes in your analysis.

#### Does CMA offer the same formulas as Stata?

Yes. The development team for CMA includes some of the same people who developed the Stata macros. CMA includes all of the same computational formulas, was validated against Stata and provides exactly the same results (see documentation). CMA offers additional options as well, but includes a button to "Use the same options as Stata," which sets all options to match Stata.

#### Can CMA import data from Stata?

Yes. You can open the data sheet in Stata, copy the data onto the Windows clipboard, and then paste it into CMA. Then you tell CMA what kind of data is located in each column. The process takes only moments to complete.

##### Comprehensive Meta-Analysis

"Thank you for developing Comprehensive Meta‐Analysis. It is by far the best tool available for today’s meta‐analytic researcher. It provides flexibility for all possible types of data, it produces great colorful graphical displays, and the product support is unmatched! I have used a number of different programs for conducting meta‐analytic research over the last ten years, and CMA is the only product I recommend to my students and colleagues."

**Karen Larwin, Ph.D. - Educational Foundations, Research Technology and Leadership, Beeghly College of Education, Youngstown State University, Youngstown, OH**

"Our group has recently begun to conduct meta‐analyses within our area of research, i.e. psychooncology and health psychology, and we have found CMA extremely useful. CMA distinguishes itself from other available meta‐analysis software by the various options to explore and adjust for possible publication bias, as well as by providing several options to explore possible moderators, not only categorical but also continuous. We encourage our PhD students to conduct quantitative systematic reviews as a part of their dissertation whenever possible. The clear menu‐driven approach of CMA makes it easy to use for beginners, so that they can focus their energy on the analytical aspects of metaanalysis, rather than on the technical issues of using the software."

**Bobby (Robert) Zachariae - Professor, dr.med., Psycho‐oncology Research Unit, Aarhus University Hospital, Denmark**

#### How does CMA compare to SPSS?

CMA is a program developed specifically for meta-analysis. As such, it includes functions to automatically compute effect sizes, to perform basic and advanced meta-analyses, and to create publication quality graphics.

SPSS is a general purpose statistical package with no intrinsic support for meta-analysis. However, David Wilson has written macros that can be incorporated into SPSS and will run a basic meta-analysis, an analysis of variance, and meta-regression.

#### Computing effect sizes

In every meta-analysis you start with the published summary data for each study and compute the treatment effect (or effect size). For example, if a study reports the number of events in each group you might compute the odds ratio. Or, if a study reports means and standard deviations you might compute the standardized mean difference. Additionally, you need to compute the variance for each effect size.

Wilson’s macros require the user to compute an effect size and variance for each study, and then provide these values to the program. Therefore, the user must either compute these values separately and then enter them as data, or write code to compute these values within SPSS.

By contrast, with CMA you enter the data directly in almost any format(s), and the program computes the effect size and variance automatically.

#### Analyses and forest plots

Wilson’s macros will report all relevant statistics, but will not create graphics such as a forest plot. CMA is able to create a forest plot, which can play a key role in helping the researcher to interpret the data and to convey it to others.

#### Reports

#### Prediction Intervals

#### Video Tutorials

#### Avoiding Common Mistakes

#### Does CMA offer the same formulas as SPSS?

SPSS itself does not include any support for meta-analysis. Wilson’s macros use the same formulas as CMA (CMA offers additional options as well), and so will yield identical results. This assumes, of course, that the user has used the same formulas to compute effect sizes and variance for each study.

#### Can CMA import data from SPSS?

Yes. You can open the data sheet in SPSS, copy the data onto the Windows clipboard, and then paste it into CMA. Then you tell CMA what kind of data is located in each column.

##### Comprehensive Meta-Analysis

"Comprehensive Meta‐Analysis is, literally speaking, the most comprehensive and user friendly metaanalysis “package” program. It excellently accommodates both advanced and beginning users’ needs at the same time and different levels of needs over time. Among numerous laudable features, this package program provides the most comprehensive and advanced solution for testing publication bias. Using Excel‐like interfaces, moderator analyses are very easy to do! It would be a great loss for any competent researcher and practitioner to overlook this impressive package program."

**In‐Sue Oh, PhD - Department of Management, Virginia Commonwealth University**

"Comprehensive Meta‐Analysis has been the perfect tool for all of my meta‐analysis needs. From calculating effect sizes to examining moderator effects to testing publication bias to building visually appealing forest plots, this program has it all. Its simplicity and straightforward design makes it easy to use for those just learning this data analysis technique. Additionally, with all that is included, this program is also great for those wishing to run more sophisticated analyses."

**Joshua Swift, PhD - University of Alaska, Anchorage**

#### How does CMA compare to SAS?

SAS is a general purpose statistical package with no intrinsic support for meta-analysis. David Wilson has written macros that can be incorporated into SAS and will run a basic meta-analysis, an analysis of variance, and meta-regression.

#### Computing effect sizes

In every meta-analysis you start with the published summary data for each study and compute the treatment effect (or effect size). For example, if a study reports the number of events in each group you might compute the odds ratio. Or, if a study reports means and standard deviations you might compute the standardized mean difference. Additionally, you need to compute the variance for each effect size.

Wilson’s macros require the user to compute an effect size and variance for each study, and then provide these values to the program. Therefore, the user must either compute these values separately and then enter them as data, or write code to compute these values within SAS.

By contrast, with CMA you enter the data directly in almost any format(s), and the program computes the effect size and variance automatically.

#### Analyses and forest plots

Wilson’s macros will report all relevant statistics, but will not create graphics such as a forest plot. CMA is able to create a forest plot, which can play a key role in helping the researcher to interpret the data and to convey it to others.

#### Reports

#### Prediction Intervals

#### Video Tutorials

#### Avoiding Common Mistakes

#### Does CMA offer the same formulas as SAS?

SAS itself does not include any support for meta-analysis. Wilson’s macros use the same formulas as CMA (CMA offers additional options as well), and so will yield identical results. This assumes, of course, that the user has used the same formulas to compute effect sizes and variance for each study.

#### Can CMA import data from SAS?

Yes. Since SAS does not support copy-and-paste, it is not possible to simply copy the data to CMA. However, SAS is able to export data to a file, which can then be imported to CMA.

##### Comprehensive Meta-Analysis

"While I was looking for a computer program for meta‐analysis, I did a thorough search and compared all available programs I located. I decided to use CMA because of its versatility and authoritativeness. It allows me to code data of various formats and conduct all kinds of analysis – Q tests, meta‐regression, sensitivity analysis, etc. It also provides results associated with both the fixed‐effects and randomeffects models. Furthermore, the program is developed by a team of leading scholars in meta‐analysis. I feel lucky to be able to find a program through which I can draw on the expertise of so many outstanding experts."

**Shaofeng Li, Ph.D. - Candidate in Second Language Studies, Michigan State University, East Lansing, MI**

"(1) Working with your staff and the software itself made computer installation and start up incredibly easy. (2) Instructing my students (seniors in a capstone research intensive course) on how to use the software was also remarkably easy. They did far more with the software than I had expected and made some elegant looking figures for their senior presentations. (3) Meta‐analysis was a new technique for our whole department (PUI institution), and the oral presentations by my students using the CMA software and graphics impressed faculty immensely."

**Susan B. Chaplin - Department of Biology, University of St. Thomas, St. Paul, MN**

#### How does CMA compare to Excel?

Excel is a spreadsheet program with no intrinsic support for meta-analysis. While it is possible to program all meta-analysis formulas in Excel, this requires knowledge of the formulas and a substantial investment of time for development and testing. Also, there is no mechanism in Excel that can be used to create a forest plot.

By contrast, all of this functionality is built into CMA.

##### Comprehensive Meta-Analysis

#### How does CMA compare to Metawin?

CMA and Metawin are both dedicated meta-analysis programs but CMA incorporates a much wider and more fully developed set of options.

#### Computing effect sizes

In every meta-analysis you start with the published summary data for each study and compute the treatment effect (or effect size). For example, if a study reports the number of events in each group you might compute the odds ratio. Or, if a study reports means and standard deviations you might compute the standardized mean difference. Additionally, you need to compute the variance for each effect size.

Both Metawin and CMA will allow you to enter summary data and will compute the effect size from that data. However, CMA will work with a much wider array of data formats. Metawin will accept data in a few formats where CMA can accept more than 100. Metawin requires that all studies provide data in the same format while CMA allows you to enter data for each study in its own format. Metawin can work with a few indices of treatment effect (or effect size) where CMA includes more than 15.

#### Analyses

Metawin will run the analysis and show the computed values. CMA displays all values as part of a scrollable grid which makes the analysis transparent – you can see which studies are included in the analysis, how the studies were weighted, and so on. CMA also includes a much more extensive set of computational options.

#### Forest plots

In CMA you can fully customize the plot, to ensure that each study stands out clearly, so that the plot is proportioned properly on the page, include all relevant columns, and so on. In Metawin the plot is very basic, allowing one column for study names but little control over formatting.

#### Reports

#### Prediction Intervals

#### Video Tutorials

#### Avoiding Common Mistakes

##### Comprehensive Meta-Analysis

"I have used Comprehensive Meta‐Analysis II for the last 6 years to assist with a number of published meta‐analytic studies in the behavioral sciences. I have found the program to be flexible and powerful. I have also used CMA II in two graduate seminars focused on meta‐analysis. The students are able to learn the program quickly and appreciate how it structures their data sets and prevents some common mistakes made in meta‐analysis."

**Alan J. Hawkins, Ph.D. - Professor of Family Life, Brigham Young University, Provo, UT**

"Have had a great experience with CMA. Not only an intuitively simple‐to‐use, yet powerful, research tool, but also a great way to teach and introduce students to meta‐analysis."

**Charles DiMaggio, PhD - Associate Clinical Professor, Columbia University, Departments of Anesthesiology and Epidemiology, New York, NY**

## Comprehensive **Meta-Analysis**

Comprehensive Meta-Analysis (CMA) is a powerful computer program for meta-analysis. The program combines ease of use with a wide array of computational options and sophisticated graphics.