Markov Chain Utilities

CTMC and DTMC analysis tools.

The mc module provides general tools for analyzing Markov chains, including steady-state and transient analysis for both continuous-time and discrete-time Markov chains.

Key function categories:

  • Steady-state analysis: ctmc_solve_reducible(), dtmc_solve_reducible(), ctmc_stochcomp()

  • Transient analysis: ctmc_transient(), ctmc_uniformization()

  • Simulation: ctmc_simulate(), ctmc_rand()

  • Aggregation methods: ctmc_courtois(), ctmc_takahashi(), ctmc_kms()

  • State-space generation: ctmc_ssg(), ctmc_ssg_reachability()

  • Generator matrices: ctmc_makeinfgen(), ctmc_multi()

Markov Chain Analysis (line_solver.api.mc)

Markov Chain analysis algorithms.

Native Python implementations for continuous-time and discrete-time Markov chain analysis.

Key algorithms:

ctmc_solve: CTMC steady-state distribution ctmc_sens: Steady-state sensitivity to a scalar parameter ctmc_transient_sens: Transient sensitivity to a scalar parameter ctmc_transient: CTMC transient analysis ctmc_uniformization: Transient distribution by Jensen uniformization ctmc_randomization: Randomized (uniformized) DTMC P = I + Q/q ctmc_foxglynn: Fox-Glynn uniformization for transient analysis ctmc_fau: Fast adaptive uniformization for transient analysis ctmc_gmres: Restarted GMRES with ILUT preconditioning ctmc_bicgstab: BiCGSTAB with the same ILUT preconditioner ctmc_saddlepoint: Pr{N(t)=k} of a MAP counting process by saddlepoint ctmc_stochcomp: Stochastic complementation dtmc_solve: DTMC steady-state distribution smp_stage_cif / smp_stage_quad: cumulative incidence, jump chain and mean holding time of a semi-Markov stage

ctmc_solve(Q, method=None)[source]

Solve for steady-state probabilities of a CTMC.

Computes the stationary distribution π by solving πQ = 0 with normalization constraint Σπ = 1.

Handles reducible CTMCs by decomposing into strongly connected components and solving each separately.

Parameters:

  • Q (ndarray) – Infinitesimal generator matrix (row sums should be zero)

  • method (str | None) – ‘gmres’ or ‘direct’ to force a solution method; None or ‘default’ selects by size, GMRES above GMRES_MIN_STATES states

Returns:

Steady-state probability distribution (1D array)

Return type:

ndarray

ctmc_makeinfgen(Q)[source]

Convert a matrix into a valid infinitesimal generator for a CTMC.

An infinitesimal generator has: - Row sums equal to zero - Non-positive diagonal elements - Non-negative off-diagonal elements

Parameters:

Q – Candidate infinitesimal generator matrix

Returns:

Valid infinitesimal generator matrix with corrected diagonal

The ctmc_* and dtmc_* families (steady-state solvers, transient analysis, uniformization) live in line_solver.api.mc and are listed above.