Markov Models in Health Economics: Transition Probabilities

Published June 20, 2026 | Last updated October 31, 2026

Quick Answer: Markov models in health economics follow a cohort through a few health states in fixed time steps called cycles. A transition-probability matrix moves the cohort each cycle, every row must sum to 1 and death is an absorbing state.¹^,² Costs and QALYs are summed cycle by cycle with a within-cycle correction and discounting. CHEERS 2022 asks you to report the time horizon (item 9), the discount rate (item 10) and your analytic methods (item 17), where, in our reading, the correction method belongs.³^,⁴ Convert rates with p = 1 − exp(−rt), not p = r.⁵ Case: we re-ran in R the four-state teaching model of Kouame and colleagues (PLOS One 2026, CC BY 4.0; inputs from its Tables 1 and 2, ten yearly cycles, no patient data).⁵ The printed costs, incremental cost and ICER (€16,065 per QALY) reproduce to the cent, and the QALYs to the printed digits, when remission is modelled with tunnel states and the within-cycle weights follow the order in the tutorial code rather than textbook Simpson order.

How Markov Models in Health Economics Work

A Markov model assumes that a patient is always in one of a finite number of discrete health states and treats every event as a transition from one state to another. Sonnenberg and Beck describe these models as useful when risk is continuous over time, when the timing of events matters and when important events may happen more than once. They can be evaluated by matrix algebra, as a cohort simulation or as a Monte Carlo simulation.¹ The ISPOR-SMDM task force report on state-transition modelling covers both the Markov cohort simulation and individual-based microsimulation.² This page is about the cohort version. A cohort model needs the parts below.

1States. A finite set of discrete health states, with each patient in one state at a time.¹ The case model has four: Cancer, Remission, Relapse and Death.
2Cycle length and horizon. The time step and the number of steps. The case model runs 10 yearly cycles.
3Starting distribution. Where the cohort begins. In the case model the whole cohort starts in Cancer.
4Transition matrix. The probability of moving from each state to each other state in one cycle.
5Rewards. The cost and the utility attached to each state per cycle.
6Correction and discounting. The within-cycle weights and the discount factors applied before the cycles are summed.

The Transition Probability Matrix

The matrix has one row per state the cohort leaves and one column per state it can enter. The case paper states the two rules: each entry is between 0 and 1 and each probability row must sum to 1.⁵ Death has a 1 on the diagonal because nobody leaves it. Below is the standard-of-care matrix as printed in the paper’s Table 1, with its last row of year-specific remission-to-death probabilities.⁵

Swipe sideways to see every column.

From \ to	Cancer	Remission	Relapse	Death
Cancer	0.45	0.40	0	0.15
Remission	0	0.50	1 − 0.50 − p(R→D)	p(R→D), by year
Relapse	0	0	0.70	0.30
Death	0	0	0	1

Swipe sideways to see every column.

Year	p(remission to death)	Implied p(remission to relapse)
1	0.05	0.45
2	0.06	0.44
3	0.06	0.44
4	0.07	0.43
5	0.09	0.41
6	0.11	0.39
7	0.14	0.36
8	0.12	0.38
9	0.12	0.38
10	0.11	0.39

The remission row shows why the matrix is built from the other entries. Staying in remission is fixed at 0.50, remission to death changes each year, and remission to relapse is whatever is left, 1 − 0.50 − p(R→D). In year 1 that is 0.45 and in year 7, when death risk peaks at 0.14, it is 0.36 (computed by Editverse). Because the death probability changes by cycle, the model needs a different matrix in every cycle. The authors call this simulation-time dependence, and a companion tutorial explains how to implement it.⁵^,⁶ The paper presents two versions of the model. In the first, the year-specific vector is indexed by model cycle. In the second, remission is split into tunnel states, so the probability depends on the years already spent in remission; we label these states R1 to R10. The costs and ICER below come from the tunnel-state version, because that is the one on which the printed Table 3 reproduces.⁵

Converting Rates to Transition Probabilities

Trial and registry results are often reported as rates, such as events per person-year, but the matrix needs a probability for one cycle. The case paper gives the conversion as p = 1 − exp(−rt), where p is the transition probability, r the rate of the event and t the time period.⁵ The introductory tutorial from Alarid-Escudero and colleagues applies the same transformation in code to constant annual rates, scaled by the cycle length.⁷

Line chart of probability against rate times time: the curve 1 minus exp of minus r t bends below the straight shortcut r t, giving 0.632 instead of 1.000 when rate times time equals 1 — The exponential conversion against the p = rt shortcut. Formula from Kouame et al. (CC BY 4.0); curves and worked values computed in R by Editverse.

Swipe sideways to see every column.

Case	Rate times time	Probability, 1 − exp(−rt)	Shortcut, p = rt
Annual rate 0.15, yearly cycle	0.15	0.1393	0.1500
Same rate, monthly cycle (t = 1/12)	0.0125	0.01242	0.01250
Rate times time equal to 1	1.00	0.632	1.000

The shortcut is close when rt is small and wrong when it is not. At rt = 1 it gives a probability of 1, a certain event, where the exponential gives 0.632. Changing the cycle length is the other common slip: an annual probability of 0.1393 re-expressed as a monthly probability with 1 − (1 − p)^1/12 gives 0.01242, the same as converting the underlying rate directly. That equivalence holds only when the rate is constant within the year.

Running the Cohort Trace

The cohort trace is the share of the cohort in each state at the end of each cycle. It starts as a row vector with everyone in Cancer and is multiplied by that cycle’s matrix to give the next row. We rebuilt the case paper’s trace for both strategies from its printed inputs, using the tunnel-state version with the ten remission states added together.⁵

Line charts of the 10-year cohort trace for standard of care and innovative therapy in a four-state cancer Markov model: cancer falls from 100 percent, remission peaks at 40 and 50 percent in year 1, relapse peaks near year 4, and death reaches about 91 percent by year 10 — Cohort trace of the tunnel-state version for each strategy, recomputed in R by Editverse from the inputs in Kouame et al. (CC BY 4.0), Tables 1 and 2.

Computed by Editverse in R

The trace sums to 1 in every cycle for both strategies (checked), so no part of the cohort is lost or created.
Remission (R1 to R10 combined) peaks at 40.0% in year 1 under standard of care and 50.0% under the innovative therapy.
Relapse peaks at 32.7% in year 4 under standard of care and 38.5% in year 4 under the innovative therapy.
By year 10, 90.9% of the standard-of-care cohort and 90.4% of the innovative-therapy cohort are dead.

Costs, QALYs, Within-Cycle Correction and Discounting

Expected cost and QALYs come from multiplying each trace row by the state reward vector, weighting by a within-cycle weight and a discount factor, and summing: total = sum over cycles of w_t × d_t × (trace row × reward vector). The discount factor is d_t = 1/(1 + r)^t. CHEERS 2022 asks authors to report the discount rate and why it was chosen (item 10).³

Swipe sideways to see every column.

Year	0	1	5	10
Discount factor at 2.5%	1.0000	0.9756	0.8839	0.7812

The within-cycle weight w_t corrects for counting a whole cycle at one time point. Elbasha and Chhatwal show that the standard half-cycle method gives the same cumulative result as the trapezoidal rule and the life-table method, that Simpson’s one-third rule is more accurate, and that the errors need not cancel in incremental results. They recommend Simpson’s one-third rule in the base case with other methods in sensitivity analysis.⁴^,⁸ The introductory tutorial describes weights of 1/3 at the first and last time points with 2/3 and 4/3 between, and its code gives 2/3 to year 1, 4/3 to year 2 and so on: 1/3, 2/3, 4/3, 2/3, …, 2/3, 1/3, which sum to 9.33.⁷ Textbook Simpson order starts 1/3, 4/3, 2/3 and the weights over the 11 time points sum to 10. The two orders are different choices, and the table shows what each does to the tunnel-state model of the case paper.

Swipe sideways to see every column.

Method	Weights on the 11 time points (years 0 to 10)	Standard of care: cost (€), QALYs	ICER (€ per QALY)
No correction	1 at every point	16,949, 2.253	23,637
Standard half-cycle (trapezoidal)	0.5 at years 0 and 10, 1 elsewhere	14,779, 2.040	18,080
Simpson one-third, textbook order	1/3, 4/3, 2/3, 4/3, …, 4/3, 1/3 (sum 10)	14,510, 2.052	16,360
Simpson one-third, tutorial code order	1/3, 2/3, 4/3, 2/3, …, 2/3, 1/3 (sum 9.33)	13,602, 1.886	16,065

For these inputs the ICER moves from €23,637 with no correction to €18,080 with the half-cycle method, €16,360 with Simpson weights in textbook order and €16,065 with the tutorial code order. This was computed by Editverse on the tunnel-state model rebuilt from the printed inputs, so treat it as an illustration of sensitivity to the correction method.

Real Case: Re-Running a Published Markov Model in R

To see whether a published Markov model can be rebuilt from what it prints, we re-ran the teaching model of Kouame and colleagues, published in PLOS One in June 2026 under CC BY 4.0. It compares standard of care with an innovative therapy for breast cancer in four states over 10 yearly cycles, and its data statement says no data were used.⁵ It is a worked example, not a clinical study. The inputs we used are printed in its Tables 1 and 2.

Swipe sideways to see every column.

Input (Kouame Table 2 and Table 1)	Standard of care	Innovative therapy
Cancer to remission	0.40	0.50
Cancer to death	0.15	0.10
Stay in remission	0.50	0.50
Remission to death	0.05 to 0.14 by year (Table 1); we index it by years spent in remission, R1 to R10	same vector (our assumption)
Relapse to death	0.30	0.30
Annual cost: cancer, remission, relapse	€4,000, €1,000, €5,000	€6,000, €1,000, €5,000
Utility: cancer, remission, relapse	0.40, 0.70, 0.35	0.40, 0.70, 0.35
Cycles and discount	10 yearly cycles; 2.5% for costs and QALYs	same
Within-cycle weights	Tutorial code order, 1/3, 2/3, 4/3, …, 1/3	same

The paper says the innovative-therapy matrix starts as a copy of the standard-of-care matrix, so we used the same remission-to-death vector in both arms. We took death as costing nothing and carrying no utility, because neither is printed, applied the 2.5% discounting the paper states (it also states that it applied the Simpson one-third rule), and counted rewards for the state occupied at the end of each cycle from year 0 to year 10. We modelled remission with tunnel states and applied the within-cycle weights in the order given in the tutorial code. The paper’s R code is hosted on GitHub, which our tools could not open, so the weight order is inferred from the tutorial and supported by the numeric match below, not read from the authors’ code.⁵

Bar chart of the ICER of the tunnel-state model under four within-cycle weightings: 23,637 euros per QALY with no correction, 18,080 with half-cycle, 16,360 with textbook Simpson order and 16,065 with the tutorial code order, which matches the printed 16,065.39 — ICER of the tunnel-state model under four sets of within-cycle weights, against the ICER printed in Table 3 of Kouame et al. (CC BY 4.0). Re-run computed in R by Editverse.

Swipe sideways to see every column.

Check	Result	Detail
Every row of the matrix sums to 1, every entry between 0 and 1	Reproduces	Passed for both arms in all 10 cycles of the tunnel-state version.
Standard of care: cost	Reproduces to the cent	Printed €13,601.93; re-run €13,601.93.
Standard of care: QALYs	Reproduces to the printed digits	Printed 1.88; re-run 1.886, which truncates to 1.88 (rounding would give 1.89).
Innovative therapy: cost	Reproduces to the cent	Printed €16,354.72; re-run €16,354.72.
Innovative therapy: QALYs	Reproduces to the printed digits	Printed 2.05; re-run 2.058, which truncates to 2.05.
Incremental cost	Reproduces to the cent	Printed €2,752.78; re-run €2,752.78. The two printed costs differ by €2,752.79, a rounding effect in the paper’s own table.
ICER	Reproduces to the cent	Printed €16,065.39 per QALY; re-run €16,065.39. Below the €30,000 per QALY threshold the paper uses.
Incremental QALYs	Consistent with the printed 0.17	Re-run 0.1713. Dividing 2,752.78 by the printed 0.17 gives €16,193, which is why the printed ICER implies an unrounded gain of about 0.171.
Probabilistic results: 85% cost-effective, NMB €45,375.74, EVPI €6,200	Not re-run	These come from random sampling of 1,000 parameter sets, and Table 2 prints one distribution parameter pair as “471,5”, so we did not attempt it.

Two further notes. The paper’s results paragraph pairs 1.88 QALYs with €16,354.72 for the innovative therapy, while its Table 3 assigns 1.88 to standard of care and 2.05 to the innovative therapy; we used the table.⁵ The match depends on the model version and on the weight order: with the same inputs, textbook Simpson order gives an ICER of €16,360, the half-cycle method €18,080, and the first (cycle-indexed) version of the model with the tutorial weights €16,079, close to but not equal to the printed value.

What this case shows: printed costs, incremental cost and ICER can be reproduced to the cent from a four-state matrix, tunnel states for remission, 2.5% discounting and the tutorial weight order, and the choice of within-cycle weights alone moves the ICER between €16,065 and €23,637. The tutorial order is the tutorial’s choice and is not textbook Simpson order, whose weights sum to 10 rather than 9.33. What it does not show: which code the authors ran, because we could not open it; that the tutorial order is the better method, since Elbasha and Chhatwal recommend the Simpson one-third rule;⁴ or anything about real patients, because the inputs are illustrative. The practical lesson is to publish code and every input, including the weights, as CHEERS 2022 items 16 and 22 ask.³

Reporting a Markov Model: CHEERS 2022 and the ISPOR-SMDM Task Forces

CHEERS 2022 replaced the 2013 statement with a 28-item checklist for health economic evaluations.³ These are the items that bear most on a Markov model, with the checklist wording and our suggestion for what to show.

Swipe sideways to see every column.

CHEERS 2022 item	What it asks (as worded)	What to show for a Markov model
9 Time horizon	State the time horizon for the study and why appropriate.	Number of cycles times cycle length, and why that horizon fits the disease. The case used 10 yearly cycles.
10 Discount rate	Report the discount rate(s) and reason chosen.	Rates for costs and QALYs. The case used 2.5% for both.
16 Rationale and description of model	If modelling is used, describe in detail and why used. Report if the model is publicly available and where it can be accessed.	States, cycle length, structure diagram, where the code sits.
17 Analytics and assumptions	Describe any methods for analysing or statistically transforming data, any extrapolation methods, and approaches for validating any model used.	Rate-to-probability conversion, within-cycle correction method, validation checks.
20 Characterising uncertainty	Describe methods to characterise any sources of uncertainty in the analysis.	Deterministic and probabilistic sensitivity analysis.
22 Study parameters	Report all analytic inputs (such as values, ranges, references) including uncertainty or distributional assumptions.	The full matrix, costs and utilities with their distributions.
24 Effect of uncertainty	Describe how uncertainty about analytic judgments, inputs, or projections affect findings. Report the effect of choice of discount rate and time horizon, if applicable.	Results under other discount rates, horizons and correction methods.

The ISPOR-SMDM task force report on state-transition modelling gives consensus guidance organised as choice of model type (cohort or individual-level), model structure, model parameters, analysis, reporting and communication.² Its companion on parameter estimation and uncertainty separates stochastic (first-order) uncertainty, parameter (second-order) uncertainty, heterogeneity and structural uncertainty, and recommends reporting both deterministic and probabilistic sensitivity analysis.⁹ We read both reports as PubMed abstracts only, because the publisher pages were not reachable from our tools on 31 October 2026, so check the full text for item-level wording.

Editverse checks before you trust any Markov model

1Rows and entries. Every row sums to 1, no entry is negative or above 1, and death stays at 1.
2Trace. The trace sums to 1 in every cycle.
3Units. Each rate was converted to a probability for your cycle length, not copied across.
4Correction. The within-cycle method is named and its effect on the ICER is shown.
5Discounting. Rates for costs and QALYs are stated, with the reason.
6Reproducibility. Code and every input, including distributions, can be found by a reader.

Need the Analysis Run or Checked? Consult Editverse

Editverse subject-matter experts work to COPE, ICMJE and GPP 2022 guidance, with institutional and country-specific ethics followed.

Editverse

Statistical Analysis

Need the analysis behind your model inputs run in R?

You send: your data, your research questions and your preferred software.

You get: analysis files with annotated code or syntax, complete output files, an interpretation guide and publication-ready tables.

When: You need transition rates, costs or utilities estimated from your own dataset.

SPSS/R/STATA Statistical Analysis · $400

Analyse my data →

Editverse

Manuscript Writing

Model built and the economic evaluation still unwritten?

You send: your data, outline, references and target journal.

You get: a manuscript written by PhD experts, with editing, statistical review, journal selection and submission support.

When: Your model is finished and the paper is not.

Manuscript Writing Services · $1,500

Write my paper →

Editverse

Journal Selection

Not sure which journal takes health economic evaluations?

You send: your manuscript, field and publication goals.

You get: a ranked list of 5-10 journals with scope fit, impact factor and acceptance rate estimates, open access and APC information, and the rationale for each.

When: You need a target journal before you format the paper.

Scientific Journal Selection Services · $115

Get my journal shortlist →

Free Research Tools from Editverse

Free Research Tools for Medical Researchers: the full set of browser-based tools.
Statistical Test Selector: pick the test and the items to report when you estimate model inputs from data.
Sample Size Calculator: power analysis for the study that supplies your inputs.

Frequently Asked Questions

What is a Markov model in health economics?

It is a model in which a hypothetical cohort is always in one of a small number of health states and moves between them in fixed time steps called cycles. Every event is a transition from one state to another. Sonnenberg and Beck describe three ways to evaluate such a model: matrix algebra, a cohort simulation or a Monte Carlo simulation. Costs and quality-adjusted life years are added up from the time the cohort spends in each state.

What is a transition probability matrix?

It is a square table in which each row is the state the cohort leaves and each column is the state it enters in the next cycle. Every entry lies between 0 and 1 and every row sums to 1. Death is an absorbing state, so its row has a 1 on the diagonal. In the Kouame teaching model, for example, the Relapse row is 0.70 for staying in relapse and 0.30 for moving to death.

How do I convert a rate into a transition probability?

Use p = 1 minus exp(minus r times t), where r is the rate and t is the cycle length in the same time unit as the rate. Computed by Editverse in R, an annual rate of 0.15 gives a yearly probability of 0.1393 and a monthly probability of 0.01242, whereas dividing the rate by 12 gives 0.01250. The shortcut p = r overstates the probability and breaks down as r times t grows.

What is half-cycle correction and which method should I use?

It is a within-cycle correction that reduces the error from counting a whole cycle of cost and QALYs at one time point. The standard half-cycle method gives the same cumulative result as the trapezoidal rule. Elbasha and Chhatwal recommend the Simpson one-third rule in the base case, with other methods shown in sensitivity analysis. On the tunnel-state model built from the Kouame inputs, Editverse computed an ICER of 23,637 with no correction, 18,080 with the half-cycle method, 16,360 with Simpson weights in textbook order and 16,065 with the weight order used in the tutorial code.

What discount rate should a Markov model use?

CHEERS 2022 does not set a rate. Item 10 asks you to report the discount rate or rates and the reason you chose them, so follow the guideline of the jurisdiction or the journal you are writing for and state it. The Kouame teaching model discounted both costs and QALYs at 2.5 percent a year over 10 yearly cycles.

How long should the cycle be and how many cycles do I need?

The number of cycles times the cycle length is the time horizon, and CHEERS 2022 item 9 asks you to state it and why it is appropriate. Elbasha and Chhatwal report that the performance of every correction method improved when a shorter cycle was used.

What does CHEERS 2022 ask for in a Markov model paper?

The 28-item checklist asks you to describe the model and say whether it is publicly available (item 16), to describe analytic methods and model validation (item 17), to report all analytic inputs with their uncertainty or distributions (item 22), and to show how uncertainty, the discount rate and the time horizon affect the findings (item 24).

Can I trust a published Markov model without re-running it?

Re-running it is the check. Editverse re-ran an open-access teaching model in R from its printed inputs. The printed costs, incremental cost and ICER reproduced to the cent, and the QALYs to the printed digits, when remission was modelled with tunnel states and the within-cycle weights followed the order used in the tutorial code. With textbook Simpson weights the ICER was 16,360 per QALY against 16,065 printed. Ask for the code and inputs, which CHEERS 2022 item 16 and item 22 are designed to make available.

References

Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Med Decis Making. 1993;13(4):322-338. doi:10.1177/0272989X9301300409. Abstract read; full text not accessed. pubmed.ncbi.nlm.nih.gov/8246705/
Siebert U, Alagoz O, Bayoumi AM, et al. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value Health. 2012;15(6):812-820. doi:10.1016/j.jval.2012.06.014. Abstract read; the publisher page was not reachable from our tools on 31 October 2026. pubmed.ncbi.nlm.nih.gov/22999130/
Husereau D, Drummond M, Augustovski F, et al. Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations. BMJ. 2022;376:e067975. doi:10.1136/bmj-2021-067975. Open access, CC BY 4.0. pubmed.ncbi.nlm.nih.gov/35017145/
Elbasha EH, Chhatwal J. Theoretical foundations and practical applications of within-cycle correction methods. Med Decis Making. 2016;36(1):115-131. doi:10.1177/0272989X15585121. Abstract read; full text not accessed. pubmed.ncbi.nlm.nih.gov/26092831/
Kouame JM, Kouakou C, Gnanou S, Yacouba B, Siani C. Markov modeling in R: advanced method using a cost-effectiveness analysis. PLoS One. 2026;21(6):e0350698. doi:10.1371/journal.pone.0350698. Open access, CC BY 4.0; inputs and printed results reused with attribution, re-run and derived numbers computed by Editverse in R. pubmed.ncbi.nlm.nih.gov/42302041/
Alarid-Escudero F, Krijkamp E, Enns EA, et al. A tutorial on time-dependent cohort state-transition models in R using a cost-effectiveness analysis example. Med Decis Making. 2023;43(1):21-41. doi:10.1177/0272989X221121747. Abstract read; cited for method only. pubmed.ncbi.nlm.nih.gov/36112849/
Alarid-Escudero F, Krijkamp E, Enns EA, et al. An introductory tutorial on cohort state-transition models in R using a cost-effectiveness analysis example. Med Decis Making. 2023;43(1):3-20. doi:10.1177/0272989X221103163. Full text read (PubMed Central author manuscript); cited for method only. pubmed.ncbi.nlm.nih.gov/35770931/
Elbasha EH, Chhatwal J. Myths and misconceptions of within-cycle correction: a guide for modelers and decision makers. Pharmacoeconomics. 2016;34(1):13-22. doi:10.1007/s40273-015-0337-0. Abstract read; full text not accessed. pubmed.ncbi.nlm.nih.gov/26643402/
Briggs AH, Weinstein MC, Fenwick EAL, et al. Model parameter estimation and uncertainty: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-6. Value Health. 2012;15(6):835-842. doi:10.1016/j.jval.2012.04.014. Abstract read; the publisher page was not reachable from our tools on 31 October 2026. pubmed.ncbi.nlm.nih.gov/22999133/