Markov Model

A Markov model is a mathematical framework used primarily in health economics and decision analysis to simulate transitions between a set of discrete health states over time. Transitions occur in fixed cycles (such as months or years) according to probabilities that depend only on the current state, not on the path that led to it, a property known as the Markov assumption or memorylessness.

Markov models are among the most widely used model types for economic evaluation in health care, particularly for chronic diseases where patients move through different health states over extended periods. They allow researchers and decision makers to project long term costs and health outcomes (typically measured in QALYs) for competing interventions, even when clinical trial follow up is limited to a few years. HTA agencies such as NICE in England, Canada's Drug Agency (formerly CADTH), and PBAC in Australia routinely receive cost effectiveness analyses built on Markov models, although they ask for the model structure to be justified rather than requiring one model type, and published guidelines (such as the ISPOR-SMDM modeling good research practices reports of 2012) provide detailed recommendations for their construction, validation, and reporting.

A Markov model begins by defining a set of mutually exclusive and exhaustive health states that describe the possible conditions a patient can be in at any point. For a cancer model, these might include "healthy," "disease free after treatment," "recurrence," "second line treatment," and "death." Each cycle, a cohort of patients transitions between states according to a matrix of transition probabilities. For example, a patient in the "disease free" state might have a 5% probability of transitioning to "recurrence," a 2% probability of transitioning to "death," and a 93% probability of remaining "disease free" during each annual cycle.

Each health state is assigned a cost (reflecting the health care resources consumed while in that state) and a utility weight (reflecting the health related quality of life associated with that state). The model runs through many cycles (often until nearly all of the cohort has reached the absorbing state, typically death, or until a fixed time horizon is reached). At each cycle, the proportion of the cohort in each state is calculated, and the costs and utilities are accumulated. The total discounted costs and QALYs for each treatment arm are computed, and the incremental cost effectiveness ratio (ICER) is calculated.

Transition probabilities are derived from clinical trial data, observational studies, meta analyses, and expert opinion. Time dependent transition probabilities (where the probability changes as time passes) can be incorporated in two ways: probabilities that depend on time since the start of the model, such as mortality rising with age, can simply change from cycle to cycle, while probabilities that depend on time spent in a state are handled with tunnel states, temporary states that track how many cycles a patient has spent there. Half cycle corrections adjust for the fact that the model moves patients only at cycle boundaries, while real transitions happen throughout a cycle (on average about halfway through), so counting state membership only at the start or only at the end of each cycle overstates or understates the time spent in each state.

Sensitivity analyses explore the impact of uncertainty in model inputs. One way sensitivity analysis varies a single parameter while holding others constant. Probabilistic sensitivity analysis (PSA) simultaneously varies all uncertain parameters according to their distributions over thousands of iterations, producing a distribution of cost effectiveness results.

Use a Markov model when the disease or condition involves transitions between health states over time, when the time horizon extends beyond the available clinical trial data, and when costs and health outcomes need to be projected over a long period. Markov models are most appropriate for chronic or recurring conditions such as cancer, cardiovascular disease, diabetes, and HIV. For acute decisions with a short time horizon and few possible outcomes, decision trees may be simpler and more appropriate. For conditions where patient history matters (violating the Markov assumption), individual patient simulation models may be preferred.

A common error is defining health states that are not mutually exclusive, allowing a patient to be in more than one state simultaneously. Another mistake is using a cycle length that is too long for the disease being modeled, which can miss important transitions that occur within the cycle. Failing to apply a half cycle correction (or another within cycle correction) introduces systematic bias in cost and QALY estimates, and the bias grows with the cycle length. Some modelers neglect time dependent transition probabilities, using a constant probability of recurrence when the actual risk decreases over time after treatment. Finally, failing to validate the model against external data (such as observed survival curves from independent studies) undermines confidence in its predictions.

In a hypothetical example, a health economics team builds a Markov model to evaluate a new disease modifying therapy for multiple sclerosis compared to standard interferon therapy over a 30 year time horizon. The model includes five health states: relapsing remitting MS with an Expanded Disability Status Scale (EDSS) score of 0 to 3, relapsing remitting MS with EDSS 4 to 6, relapsing remitting MS with EDSS 7 to 9, secondary progressive MS, and death. Baseline one year transition probabilities come from natural history data, and the treatment effects from a network meta analysis of 18 clinical trials. The new therapy reduces the annual probability of transitioning from EDSS 0 to 3 to EDSS 4 to 6 from 8% to 5%. Over the model horizon, the new therapy produces 1.2 additional discounted QALYs at an incremental discounted cost of $45,000, yielding an ICER of $37,500 per QALY.