Action

• Numeric hyperparameter \(\iff\) Continuous action \([0, 1]\)

• Categorical hyperparameter \(\iff\) Discrete action \(0, 1, 2, \ldots\)

Action Configuration

Before creating Action or BatchedAction instances, you need to specify the model family and numeric hyperparameter bounds by setting up the abstract class ActionConfig.

from linguaml.rl.action import ActionConfig
from linguaml.tolearn.family import Family
from linguaml.tolearn.hp.bounds import NumericHPBounds

ActionConfig.family = Family.SVC
ActionConfig.numeric_hp_bounds = NumericHPBounds.from_dict({
    "C": (0.1, 100),
    "gamma": (0.1, 100),
    "tol": (1e-5, 1e-3)
})

ActionConfig is shared by all Action and BatchedAction instances.

Single Action

Direct Construction and Conversion to Hyperparameter Configuration

Construct an action from a dictionary:

from linguaml.rl.action import Action

action = Action({
    "C": 0.1,
    "gamma": 0.5,
    "tol": 0.1,
    "kernel": 0,
    "decision_function_shape": 1
})

action

{'C': 0.1, 'gamma': 0.5, 'tol': 0.1, 'kernel': 0, 'decision_function_shape': 1}

print(action.family)
print(action.numeric_hp_bounds)

Family.SVC
name2bounds={'C': Bounds(min=0.1, max=100.0), 'gamma': Bounds(min=0.1, max=100.0), 'tol': Bounds(min=1e-05, max=0.001)}

Use to_hp_config method to create the corresponding hyperparameter configuration.

action.to_hp_config()

SVCConfig(C=10.090000000000002, kernel='linear', gamma=50.050000000000004, tol=0.000109, decision_function_shape='ovr')

Construction from Hyperparameter Configuration

Conversely, suppose you have a hyperparameter configuration setting:

hp_config = Family.SVC.hp().model_validate({
    "C": 10,
    "gamma": 0.01,
    "tol":1e-3,
    "kernel": "rbf",
    "decision_function_shape": "ovr"
})

hp_config

SVCConfig(C=10.0, kernel='rbf', gamma=0.01, tol=0.001, decision_function_shape='ovr')

To construct an action from this hyperparameter configuration setting, you can use the following code:

action = Action.from_hp_config(hp_config)

action

{'C': 0.0990990990990991,
 'gamma': -0.0009009009009009009,
 'tol': 1.0,
 'kernel': 2,
 'decision_function_shape': 1}

print(f"original:\t{hp_config}")
print(f"recovered:\t{action.to_hp_config()}")

original:	C=10.0 kernel='rbf' gamma=0.01 tol=0.001 decision_function_shape='ovr'
recovered:	C=10.0 kernel='rbf' gamma=0.009999999999999995 tol=0.001 decision_function_shape='ovr'

Close enough.

To PyTorch Tensors

Convert the action to a dictionary that maps hyperparameter names to tensors so that you can feed it to PyTorch’s neural network:

action.to_tensor_dict()

{'C': tensor(0.0991),
 'gamma': tensor(-0.0009),
 'tol': tensor(1.),
 'kernel': tensor(2.),
 'decision_function_shape': tensor(1.)}

Batched Actions

Construct batched actions from a dictionary with values of NumPy arrays:

from linguaml.rl.action import BatchedActions
import numpy as np

batched_actions = BatchedActions({
    "C": np.array([0.5, 0.1]),
    "gamma": np.array([0.1, 0.1]),
    "tol": np.array([0.1, 0.01]),
    "kernel": np.array([0, 1]),
    "decision_function_shape": np.array([1, 0])
})

batched_actions

{'C': array([0.5, 0.1]),
 'gamma': array([0.1, 0.1]),
 'tol': array([0.1 , 0.01]),
 'kernel': array([0, 1]),
 'decision_function_shape': array([1, 0])}

Convert to a list of Action instances:

batched_actions.to_actions()

[{'C': 0.5,
  'gamma': 0.1,
  'tol': 0.1,
  'kernel': 0,
  'decision_function_shape': 1},
 {'C': 0.1,
  'gamma': 0.1,
  'tol': 0.01,
  'kernel': 1,
  'decision_function_shape': 0}]

Convert to a list of hyperparameter configurations:

batched_actions.to_hp_configs()

[SVCConfig(C=50.050000000000004, kernel='linear', gamma=10.090000000000002, tol=0.000109, decision_function_shape='ovr'),
 SVCConfig(C=10.090000000000002, kernel='poly', gamma=10.090000000000002, tol=1.9900000000000003e-05, decision_function_shape='ovo')]

Convert to tensor dictionary:

batched_actions.to_tensor_dict()

{'C': tensor([0.5000, 0.1000]),
 'gamma': tensor([0.1000, 0.1000]),
 'tol': tensor([0.1000, 0.0100]),
 'kernel': tensor([0., 1.]),
 'decision_function_shape': tensor([1., 0.])}