Package jline.api.rl

Class RlEnv

  • All Implemented Interfaces:
    
public final class RlEnv

    RL environment for queueing network routing decisions.

    This environment wraps a queueing network model and provides methods for:

    • Checking if the current state is within defined state/action spaces

    • Sampling the next event using SSA simulation

    • Updating the model state after an action

    • Resetting the environment to its initial state

    The state is defined by queue lengths at each queue node. The action space corresponds to routing decisions (one per queue).

    • Constructor Detail

      • RlEnv

        RlEnv(Network model, IntArray idxOfQueueInNodes, IntArray idxOfSourceInNodes, Integer stateSize, Double gamma)
        Parameters:
        model - the queueing network model (a copy is stored internally)
        idxOfQueueInNodes - indices of queue nodes in model.
        idxOfSourceInNodes - indices of source nodes in model.
        stateSize - maximum number of jobs per queue to consider in the state space
        gamma - discount factor for future rewards (0 < gamma <= 1)

    • Method Detail

      • getActionSize

         final Integer getActionSize()

        Number of possible actions (equal to the number of queues).

      • isInStateSpace

         final Boolean isInStateSpace(List<Node> nodes)

        Checks if the current model state is within the defined state space.

        A state is valid if the total number of jobs at each queue node does not exceed stateSize.

        Parameters:
        nodes - the list of nodes from the model (model.
        Returns:

        true if all queue lengths are within bounds

      • isInActionSpace

         final Boolean isInActionSpace(List<Node> nodes)

        Checks if actions can be taken from the current model state.

        An action is valid if each queue can accept at least one more job, i.e., the total number of jobs at each queue is strictly less than stateSize.

        Parameters:
        nodes - the list of nodes from the model (model.
        Returns:

        true if all queues can accept new jobs

      • sample

         final SampleEvent sample()

        Samples the next event from the environment using the SSA solver.

        Generates a single system event and returns the elapsed time and the index of the node where a departure occurred.

        Returns:

        a SampleEvent containing the time delta and departure node index

      • update

         final Unit update(IntArray newState)

        Updates the model state after an event.

        Sets the state of each queue node based on the new marginal queue lengths.

        Parameters:
        newState - array of new queue lengths, one per queue (indexed by position in idxOfQueueInNodes)

      • reset

         final Unit reset()

        Resets the environment to its initial state.

        Calls model.reset() and model.initDefault() to restore the model to its default configuration.