Package jline.api.rl

Class RlEnvGeneral

  • All Implemented Interfaces:
    
public final class RlEnvGeneral

    General RL environment for queueing network control decisions.

    This environment supports arbitrary dispatch/routing decisions at designated action nodes. The action space at each action node is determined by the network's connection matrix (the set of downstream nodes reachable from that node).

    The state is defined by queue lengths at each queue node. Events are sampled using the SSA solver and the agent can override routing decisions at action nodes.

    • Constructor Detail

      • RlEnvGeneral

        RlEnvGeneral(Network model, IntArray idxOfQueueInNodes, IntArray idxOfActionNodes, Integer stateSize, Double gamma)
        Parameters:
        model - the queueing network model (a copy is stored internally)
        idxOfQueueInNodes - indices of queue nodes in model.
        idxOfActionNodes - indices of nodes where routing actions are needed
        stateSize - maximum number of jobs per queue to consider in the state space
        gamma - discount factor for future rewards (0 < gamma <= 1)

    • Method Detail

      • getActionSpace

         final HashMap<Integer, IntArray> getActionSpace()

        Action space map: for each action node index, stores the array of downstream node indices reachable from that node (based on the network's connection matrix).

      • isInStateSpace

         final Boolean isInStateSpace(IntArray state)

        Checks if the given state vector is within the defined state space.

        A state is valid if the number of jobs at each queue does not exceed stateSize.

        Parameters:
        state - array of queue lengths (one per queue, indexed by position in idxOfQueueInNodes)
        Returns:

        true if all queue lengths are within bounds

      • isInActionSpace

         final Boolean isInActionSpace(IntArray state)

        Checks if actions can be taken from the given state.

        An action is valid if each queue can accept at least one more job, i.e., each queue length is strictly less than stateSize.

        Parameters:
        state - array of queue lengths (one per queue)
        Returns:

        true if all queues can accept new jobs

      • sample

         final GeneralSampleEvent sample()

        Samples the next event from the environment using the SSA solver.

        Generates a single system event and returns the elapsed time, the departure node, the arrival node, and the raw sample result for state updates.

        Returns:

        a GeneralSampleEvent containing the time delta, departure node index, arrival node index, and the raw SSA sample result

      • update

         final Unit update(Ret.SampleResult sampleResult)

        Updates the model state after an event using the SSA sample result.

        Applies State.afterEvent for each event in the sample to advance the model's internal state consistently with the SSA simulation.

        Parameters:
        sampleResult - the raw SSA sample result from sample

      • reset

         final Unit reset()

        Resets the environment to its initial state.

        Calls model.reset() and model.initDefault() to restore the model to its default configuration.