Package jline.lang.processes

Class MarkovProcess

java.lang.Object
jline.lang.processes.Process
jline.lang.processes.MarkovProcess
All Implemented Interfaces:
Serializable
Direct Known Subclasses:
MarkedMarkovProcess
public class MarkovProcess extends Process
A class for a continuous time Markov chain
See Also:

  • Field Details

    • infGen

      protected Matrix infGen

    • stateSpace

      protected Matrix stateSpace

    • isfinite

      protected boolean isfinite

  • Constructor Details

    • MarkovProcess

      public MarkovProcess(Matrix infGen)
      Creates a CTMC with the specified infinitesimal generator
      Parameters:
      infGen - the infinitesimal generator matrix

    • MarkovProcess

      public MarkovProcess(Matrix infGen, boolean isFinite)
      Creates a CTMC with the specified infinitesimal generator and finite flag
      Parameters:
      infGen - the infinitesimal generator matrix
      isFinite - whether the CTMC is finite

    • MarkovProcess

      public MarkovProcess(Matrix infGen, boolean isFinite, Matrix stateSpace)
      Creates a CTMC with the specified infinitesimal generator, finite flag, and state space
      Parameters:
      infGen - the infinitesimal generator matrix
      isFinite - whether the CTMC is finite
      stateSpace - the state space representation

  • Method Details

    • toDTMC

      public MarkovChain toDTMC()
      Convert to DTMC using uniformization
      Returns:
      the equivalent DTMC

    • toDTMC

      public MarkovChain toDTMC(double q)

    • toTimeReversed

      public MarkovProcess toTimeReversed()
      Get time-reversed CTMC
      Returns:
      time-reversed CTMC

    • setStateSpace

      public void setStateSpace(Matrix stateSpace)
      Set the state space
      Parameters:
      stateSpace - the state space matrix

    • getGenerator

      public Matrix getGenerator()
      Get the infinitesimal generator matrix
      Returns:
      the generator matrix

    • getProbState

      public double getProbState(int state)
      Get probability of a specific state using steady-state analysis
      Parameters:
      state - the state index (0-based)
      Returns:
      the probability of the specified state

    • getProbState

      public Matrix getProbState(Matrix state)
      Get probability of a specific state using Cramer's rule
      Parameters:
      state - the state vector (if state space is defined) or single state index
      Returns:
      the probability matrix containing the state probability

    • solve

      public Matrix solve()
      Solve the CTMC for steady-state probabilities
      Returns:
      the steady-state probability vector

    • transientProb

      public Matrix transientProb(Matrix pi0, double t)
      Distribution at time t from pi0, by Jensen uniformization.
      Parameters:
      pi0 - initial distribution, uniform when null
      t - time point
      Returns:
      the distribution at time t

    • transientProb

      public Matrix transientProb(Matrix pi0, double t, String method)
      Distribution at time t from pi0.
      Parameters:
      pi0 - initial distribution, uniform when null
      t - time point
      method - "unif" for Jensen uniformization, "foxglynn" for the Fox-Glynn weights, which avoid evaluating the Poisson terms directly
      Returns:
      the distribution at time t

    • solveRelative

      public Matrix solveRelative(int refstate)
      Equilibrium distribution relative to a reference state, i.e. with p(refstate) = 1. Unnormalized by construction, so it is defined even where the normalizing constant is not.
      Parameters:
      refstate - 0-based index of the reference state
      Returns:
      the relative equilibrium distribution

    • aggregate

      public Triple<Matrix,Double,Double> aggregate(List<List<Integer>> MS, String method, Object param)
      Aggregation-disaggregation over a macrostate partition.
      Parameters:
      MS - macrostates, each a list of 0-based state indices
      method - "courtois" (param is the randomization rate q), "kms" or "takahashi" (param is the iteration count, default 10), or "multi" (param is the second-level partition, a List<List<Integer>>)
      param - the per-method parameter described above, may be null
      Returns:
      the approximate stationary vector, the nearly-complete- decomposability index of the partition, and the largest index for which the approximation is meant to hold

    • timeAverage

      public Matrix timeAverage(Matrix pi0, double t)
      Time-averaged distribution over [0,t].
      Parameters:
      pi0 - initial distribution, uniform when null
      t - horizon
      Returns:
      the time-averaged distribution over [0,t]

    • sens

      public Matrix sens(Matrix dQ)
      Sensitivity of the stationary distribution to a scalar parameter.
      Parameters:
      dQ - derivative of the generator with respect to the parameter
      Returns:
      the derivative of the stationary distribution

    • stochComp

      public Matrix stochComp(List<Double> I)
      Stochastic complement of a subset of states. Use stochCompFull to also obtain the partitioned blocks.
      Parameters:
      I - 0-based indices of the states to retain
      Returns:
      the generator of the complement on those states

    • stochCompFull

      public SolverCTMC.StochCompResult stochCompFull(List<Double> I)
      Stochastic complement of a subset of states together with the blocks of the generator partitioned by I and its complement, and the return-path term T = Q12*inv(-Q22)*Q21, so that S = Q11 + T. Twin of the MATLAB [S,Q11,Q12,Q21,Q22,T] = ctmc.stochCompFull(I) and of the Python stochCompFull, which return the same six matrices.
      Parameters:
      I - 0-based indices of the states to retain
      Returns:
      the complement and the blocks it was built from

    • isFeasible

      public boolean isFeasible()
      Returns:
      true when the generator is a valid one

    • toEmbedded

      public MarkovChain toEmbedded()
      Embedded jump chain, i.e. the DTMC of the states visited at transition epochs. Unlike toDTMC (uniformization) it does not preserve the stationary distribution, since it drops the holding times; an absorbing state stays absorbing.
      Returns:
      the embedded DTMC

    • sample

      public Matrix sample()
      Sample from the CTMC
      Specified by:
      sample in class Process
      Returns:
      sampled value

    • sample

      public Matrix sample(int n)
      Sample n state transitions from the CTMC starting from steady-state
      Parameters:
      n - number of state transitions to sample
      Returns:
      matrix containing sampled states (column vector)

    • sample

      public Matrix sample(int n, Matrix pi0, Random random)
      Sample n state transitions from the CTMC
      Parameters:
      n - number of state transitions to sample
      pi0 - initial state distribution (if null, uses steady-state)
      random - random number generator
      Returns:
      matrix containing sampled states (column vector)

    • getStateSpace

      public Matrix getStateSpace()
      Get the state space
      Returns:
      the state space matrix

    • isFinite

      public boolean isFinite()
      Check if the CTMC is finite
      Returns:
      true if finite

    • rand

      public static MarkovProcess rand(int nStates)
      Create a random CTMC
      Parameters:
      nStates - number of states
      Returns:
      random CTMC

    • fromSampleSysAggr

      public static MarkovProcess fromSampleSysAggr(Matrix samples, Matrix sojournTimes)
      Create CTMC from sample system aggregation
      Parameters:
      samples - matrix where each row is a sample trajectory, each column is a time step
      sojournTimes - matrix of sojourn times corresponding to each state in samples
      Returns:
      CTMC constructed from samples by estimating transition rates

    • fromSampleSysAggr

      public static MarkovProcess fromSampleSysAggr(Matrix samples, Matrix sojournTimes, int numStates)
      Create CTMC from sample system aggregation
      Parameters:
      samples - matrix where each row is a sample trajectory, each column is a time step
      sojournTimes - matrix of sojourn times corresponding to each state in samples
      numStates - number of states (if -1, infer from data)
      Returns:
      CTMC constructed from samples by estimating transition rates

    • fromSampleSysAggr

      public static MarkovProcess fromSampleSysAggr(Matrix samples)
      Create CTMC from sample system aggregation (assuming unit sojourn times)
      Parameters:
      samples - matrix where each row is a sample trajectory, each column is a time step
      Returns:
      CTMC constructed from samples with unit time steps

    • fromSampleSysAggr

      public static MarkovProcess fromSampleSysAggr(Ret.SampleResult sa)
      Create CTMC from a SampleResult containing multi-node state trajectories. Matches MATLAB MarkovProcess.fromSampleSysAggr(sa) where sa has .state{} and .t fields.
      Parameters:
      sa - the SampleResult from sampleSysAggr()
      Returns:
      CTMC constructed from sampled state trajectories

    • fromSampleSysAggr

      public static MarkovProcess fromSampleSysAggr(Object sa)
      Create CTMC from sample system aggregation (dispatches based on input type).
      Parameters:
      sa - the sample aggregation object (Matrix or SampleResult)
      Returns:
      CTMC constructed from samples