Package jline.lang.processes

Class MarkovChain

java.lang.Object
jline.lang.processes.Process
jline.lang.processes.MarkovChain
All Implemented Interfaces:
Serializable
public class MarkovChain extends Process
A class for a discrete time Markov chain
See Also:

  • Field Details

    • transMat

      protected Matrix transMat

    • stateSpace

      protected Matrix stateSpace

    • isfinite

      protected boolean isfinite

  • Constructor Details

    • MarkovChain

      public MarkovChain(Matrix transMat)
      Creates a DTMC with the specified transition matrix
      Parameters:
      transMat - the transition matrix

    • MarkovChain

      public MarkovChain(Matrix transMat, boolean isFinite)
      Creates a DTMC with the specified transition matrix and finite flag
      Parameters:
      transMat - the transition matrix
      isFinite - whether the DTMC is finite

  • Method Details

    • toCTMC

      public MarkovProcess toCTMC()
      Convert to CTMC by subtracting identity matrix
      Returns:
      the equivalent CTMC

    • toTimeReversed

      public MarkovChain toTimeReversed()
      Get time-reversed DTMC
      Returns:
      time-reversed DTMC

    • getTransMat

      public Matrix getTransMat()
      Get the transition matrix
      Returns:
      the transition matrix

    • solve

      public Matrix solve()
      Solve the DTMC for steady-state probabilities. Twin of MarkovProcess.solve.
      Returns:
      the steady-state probability vector

    • transientProb

      public Matrix transientProb(Matrix pi0, int steps)
      Distribution at each step 0,...,steps.
      Parameters:
      pi0 - initial distribution, uniform when null
      steps - number of steps
      Returns:
      matrix with one row per step

    • hittingTime

      public Matrix hittingTime(int[] targetStates)
      Mean number of steps to reach any target state.
      Parameters:
      targetStates - 0-based indices of the target states
      Returns:
      column vector of mean hitting times

    • stochComp

      public Matrix stochComp(List<Integer> I)
      Stochastic complement of a subset of states.
      Parameters:
      I - 0-based indices of the states to retain
      Returns:
      the transition matrix of the complement on those states

    • stochCompFull

      public Dtmc_stochcomp.DtmcStochCompResult stochCompFull(List<Integer> I)
      Stochastic complement of a subset of states together with the four blocks of the transition matrix partitioned by I and its complement. Twin of the MATLAB [S,P11,P12,P21,P22] = dtmc.stochCompFull(I) and of the Python stochCompFull.
      Parameters:
      I - 0-based indices of the states to retain
      Returns:
      the complement and the blocks it was built from

    • transientUnif

      public Matrix transientUnif(Matrix pi0, double t)
      Distribution at time t of the DTMC seen through uniformization. The chain is read as the randomized image of a CTMC, so t is continuous here, unlike the step count taken by transientProb.
      Parameters:
      pi0 - initial distribution, uniform when null
      t - time point
      Returns:
      the distribution at time t

    • isFeasible

      public boolean isFeasible()
      Returns:
      true when the transition matrix is stochastic

    • setStateSpace

      public void setStateSpace(Matrix stateSpace)
      Set the state space
      Parameters:
      stateSpace - the state space matrix

    • getStateSpace

      public Matrix getStateSpace()
      Get the state space
      Returns:
      the state space matrix

    • isFinite

      public boolean isFinite()
      Check if the DTMC is finite
      Returns:
      true if finite

    • sample

      public Matrix sample()
      Sample from the DTMC
      Specified by:
      sample in class Process
      Returns:
      sampled value

    • sample

      public Matrix sample(int n)
      Sample n steps from the DTMC starting from initial distribution
      Parameters:
      n - number of steps to sample
      Returns:
      matrix containing sampled states (column vector)

    • sample

      public Matrix sample(int n, Matrix pi0, Random random)
      Sample n steps from the DTMC
      Parameters:
      n - number of steps to sample
      pi0 - initial state distribution (if null, uses uniform distribution)
      random - random number generator
      Returns:
      matrix containing sampled states (column vector)

    • rand

      public static MarkovChain rand(int nStates)
      Create a random DTMC
      Parameters:
      nStates - number of states
      Returns:
      random DTMC

    • fromSampleSysAggr

      public static MarkovChain fromSampleSysAggr(Matrix samples)
      Create DTMC from sample system aggregation
      Parameters:
      samples - matrix where each row is a sample trajectory, each column is a time step
      Returns:
      DTMC constructed from samples by estimating transition probabilities

    • fromSampleSysAggr

      public static MarkovChain fromSampleSysAggr(Matrix samples, int numStates)
      Create DTMC from sample system aggregation
      Parameters:
      samples - matrix where each row is a sample trajectory, each column is a time step
      numStates - number of states (if -1, infer from data)
      Returns:
      DTMC constructed from samples by estimating transition probabilities

    • fromSampleSysAggr

      public static MarkovChain fromSampleSysAggr(Object sa)
      Create DTMC from sample system aggregation (legacy interface)
      Parameters:
      sa - the sample aggregation object
      Returns:
      DTMC constructed from samples