rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy Class Reference

This class is a PyTorch Model implementing the MLP based Actor-Critic Policy. More...

Inheritance diagram for rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy:

Collaboration diagram for rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy:

Public Member Functions
def	__init__ (self, int sequence_length, List[int] hidden_sizes, Union[int, List[Union[int, List[int]]]] action_space, Union[Activation, List[Activation]] activation=pytorch.nn.ReLU(), float dropout=0.5)
	Initialize ActorCriticMlpPolicy model. More...
def	forward (self, x)
	The forwards method of the nn.Module. More...

Data Fields
	actor_activation
	actor_head
	The final head for actor; creates logits for actions. More...
	critic_head
	The final head for critic; creates the state value. More...
	flatten
	The object to flatten the output fo feature extractor. More...
	mlp_feature_extractor
	The feature extractor instance of rlpack.models._mlp_feature_extractor._MlpFeatureExtractor. More...
	value_activation

Static Private Member Functions
int	_process_action_space (Union[int, List[Union[int, List[int]]]] action_space)
	Processes action_space for use by the model. More...
List[Activation]	_process_activation (Union[Activation, List[Activation]] activation)
	Processes activation for use by the model. More...

Private Attributes
	_apply_actor_activation
	FLag indicating whether to apply activation to output of actor head or not. More...
	_apply_critic_activation
	FLag indicating whether to apply activation to output of critic head or not. More...

Detailed Description

This class is a PyTorch Model implementing the MLP based Actor-Critic Policy.

Constructor & Destructor Documentation

__init__()

def rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.__init__	(		self,
		int	sequence_length,
		List[int]	hidden_sizes,
		Union[int, List[Union[int, List[int]]]]	action_space,
		Union[Activation, List[Activation]]	activation = pytorch.nn.ReLU(),
		float	dropout = 0.5
	)

Initialize ActorCriticMlpPolicy model.

Parameters

sequence_length	int: The sequence length of the expected tensor.
hidden_sizes	List[int]: The list of hidden sizes for each layer.
action_space	Union[int, List[Union[int, List[int]]]]: The action space of the environment. If discrete action set is used, number of actions can be passed. If continuous action space is used, a list must be passed with first element representing the output features from model, second representing the shape of action to be sampled.
activation	Union[Activation, List[Activation]]: The activation function class(es) for the model. Must be an initialized activation object from PyTorch's nn (torch.nn) module. If a list is passed, List must be of length [1, 3], first activation for feature extractor, second for actor head and third for critic head.
dropout	float: The dropout to be used in the final Linear (FC) layer.

Member Function Documentation

_process_action_space()

int rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy._process_action_space ( Union[int, List[Union[int, List[int]]]]  action_space )

staticprivate

Processes action_space for use by the model.

If checks are passed, returns the output features for actor head.

Parameters

action_space

Union[int, List[Union[int, List[int]]]]: The action space of the environment. If discrete action set is used, number of actions can be passed. If continuous action space is used, a list must be passed with first element representing the output features from model, second representing the shape of action to be sampled.

Returns

: Tuple[Union[int, List[Union[int, List[int]]]], Union[Activation, List[Activation]]]: The corrected values for action_space and activation if required.

_process_activation()

List[Activation] rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy._process_activation ( Union[Activation, List[Activation]]   activation )

staticprivate

Processes activation for use by the model.

Parameters

activation

Union[Activation, List[Activation]]: The activation function class(es) for the model. Must be an initialized activation object from PyTorch's nn (torch.nn) module. If a list is passed, List must be of length [1, 3], first activation for feature extractor, second for actor head and third for critic head.

forward()

def rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.forward	(		self,
			x
	)

The forwards method of the nn.Module.

Parameters

x	pytorch.Tensor: The model input.

Returns

pytorch.Tensor: The model output (logits).

Field Documentation

_apply_actor_activation

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy._apply_actor_activation

private

FLag indicating whether to apply activation to output of actor head or not.

_apply_critic_activation

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy._apply_critic_activation

private

FLag indicating whether to apply activation to output of critic head or not.

actor_activation

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.actor_activation

actor_head

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.actor_head

The final head for actor; creates logits for actions.

critic_head

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.critic_head

The final head for critic; creates the state value.

flatten

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.flatten

The object to flatten the output fo feature extractor.

mlp_feature_extractor

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.mlp_feature_extractor

The feature extractor instance of rlpack.models._mlp_feature_extractor._MlpFeatureExtractor.

value_activation

rlpack.models.actor_critic_mlp_policy.ActorCriticMlpPolicy.value_activation