## Risky Betting for Rating Mild Psoriasis Therapeutic Outcomes: Intra-Class Correlation Coefficients Confirm Low PASI Score Inter-rater Unreliability

GUEST BLOGGERS:
Robert P. Dellavalle and Julie Green

In 1946, the mathematician Stanisław Ulam wondered how many times a 52 card deck would play out successfully in a game of solitaire. Instead of mathematically calculating the solution Ulam empirically played the solitaire one hundred times and observed the results. And thus the Monte Carlo simulation (named after the popular European casino where Stanislaw’s uncle frequently gambled) was born.

Monte Carlo simulations used in space and oil exploration, and many other risky endeavors, predict cost and schedule overruns. So a gambling moniker is appropriate. Increased computer power has made Monte Carlo simulations easier over the years since Ulam used them to help produce nuclear bombs during the Manhattan project. Monte Carlo simulations now model a boggling array of phenomena ranging from predicting the weather, determining the lifetime energy output of wind farms, forecasting the impact of pollution, optimizing winning strategies for games like Battleship and Go, modeling virtual 3D images, and valuing a company’s assets, to optimizing the design of wireless telephone networks.

And why did we spend so much time on calculus in high school?

Monte Carlo simulations are comprised of computational algorithms performed on multiple random samplings. In the recent JID paper by Gourraud and colleagues (Gourraud et al., 2012) the authors examined samplings of theoretical Psoriasis Area Severity Index (PASI) scores.

The PASI was developed to standardize assessment of the severity of psoriasis and changes in severity over time in individual patients. The PASI reflects a mathematical equation that incorporates scaled measurements of psoriasis-related erythema, induration, and scaling on the head, arms, trunk, and legs, which results in a score that ranges from 0 (no disease) to 72 (maximal disease). Because two clinicians evaluating the same patient rarely calculate the same PASI scores, these measurements are at least partially subjective. The PASI scoring system is also too cumbersome for many clinicians to incorporate into daily practice.

Many clinical research protocols have used the PASI scoring system to monitor responses of patients to experimental psoriasis treatments, and determination of efficacy is generally based upon the FDA guideline of at least a 75% improvement in PASI score. Using the Intra-Class Correlation Coefficient (ICC) Gourraud and colleagues confirmed what has long been suspected but hidden by less appropriate statistical methods—that for patients with limited psoriasis (evidenced PASI scores below 20) PASI scores are not reliable measures of therapeutic outcome. These results confirm the need for better therapeutic outcome measures of psoriasis for patients with limited psoriasis (Jensen et al. 2010).

References:

Jensen JD, Fujita M, Dellavalle RP. Validation of Psoriasis Clinical Severity and Outcome Measures: Searching for a Gold Standard. Commentary on: How Good Are Clinical Severity and Outcome Measures for Psoriasis?: Quantitative Evaluation in a Systematic Review by Spuls PI, et al. Arch Dermatol, 2011 Jan;147(1):95-8. PMID: 20855674.

Gourraud, P-A, Le Gall C, Puzenat E (2012) Why Statistics Matter: Limited Inter-Rater Agreement Prevents Using Psoriasis Area and Severity Index as a Unique Determinant of Therapeutic Decision in Psoriasis. J Invest Dermatol epublished 17 May 2012.

Wikipedia, Stanisław Ulam Accessed June 2012

This image was obtained from Flickr, and it is by sskennel.