#include <classicreinforce.h>
Inherits AbstractController.
Inheritance diagram for ClassicReinforce:
Public Member Functions | |
| ClassicReinforce (const ClassicReinforceConf &conf=getDefaultConf()) | |
| virtual void | init (int sensornumber, int motornumber, RandGen *randGen=0) |
| initialisation of the controller with the given sensor/ motornumber Must be called before use. | |
| virtual | ~ClassicReinforce () |
| virtual int | getSensorNumber () const |
| returns the number of sensors the controller was initialised with or 0 if not initialised | |
| virtual int | getMotorNumber () const |
| returns the mumber of motors the controller was initialised with or 0 if not initialised | |
| virtual void | step (const sensor *, int number_sensors, motor *, int number_motors) |
| performs one step (includes learning). | |
| virtual void | stepNoLearning (const sensor *, int number_sensors, motor *, int number_motors) |
| performs one step without learning. Calulates motor commands from sensor inputs. | |
| void | setManualControl (bool mControl, int action_=0) |
| enables/disables manual control, action_ is the sat network number to be used if mControl is false, action is ignored | |
| virtual paramval | getParam (const paramkey &key) const |
| virtual bool | setParam (const paramkey &key, paramval val) |
| virtual paramlist | getParamList () const |
| The list of all parameters with there value as allocated lists. | |
| virtual bool | store (FILE *f) const |
| stores the controller values to a given file. | |
| virtual bool | restore (FILE *f) |
| loads the controller values from a given file. | |
| virtual std::list< iparamkey > | getInternalParamNames () const |
| The list of the names of all internal parameters given by getInternalParams(). | |
| virtual std::list< iparamval > | getInternalParams () const |
| virtual std::list< ILayer > | getStructuralLayers () const |
| Specifies which parameter vector forms a structural layer (in terms of a neural network) The ordering is important. | |
| virtual std::list< IConnection > | getStructuralConnections () const |
| Specifies which parameter matrix forms a connection between layers (in terms of a neural network) The orderning is not important. | |
Static Public Member Functions | |
| static ClassicReinforceConf | getDefaultConf () |
Protected Member Functions | |
| virtual int | getStateNumber ()=0 |
| returns number of state, to be overwritten | |
| virtual int | calcState ()=0 |
| returns state, to be overwritten | |
| virtual int | getActionNumber ()=0 |
| returns number of actions, to be overwritten | |
| virtual matrix::Matrix | calcMotor (int action)=0 |
| returns action Matrix from discrete actions, to be overwritten | |
| virtual double | calcReinforcement ()=0 |
| returns the reinforcement (reward), to be overwritten | |
| void | putInBuffer (matrix::Matrix *buffer, const matrix::Matrix &vec, int delay=0) |
| virtual void | fillSensorBuffer (const sensor *x_, int number_sensors) |
| puts the sensors in the ringbuffer | |
| virtual void | fillMotorBuffer (const motor *y_, int number_motors) |
| puts the motors in the ringbuffer | |
| virtual void | management () |
| handles inhibition damping etc. | |
Protected Attributes | |
| unsigned short | number_sensors |
| unsigned short | number_motors |
| unsigned short | buffersize |
| matrix::Matrix * | x_buffer |
| matrix::Matrix * | y_buffer |
| matrix::Matrix * | x_context_buffer |
| bool | manualControl |
| True if actions (sats) are selected manually. | |
| int | action |
| action | |
| int | oldaction |
| old action | |
| int | state |
| current state | |
| double | reward |
| current reward | |
| double | oldreward |
| old reward (nicer for plotting) | |
| ClassicReinforceConf | conf |
| bool | initialised |
| int | t |
| int | managementInterval |
| interval between subsequent management calls | |
| ClassicReinforce | ( | const ClassicReinforceConf & | conf = getDefaultConf() |
) |
| ~ClassicReinforce | ( | ) | [virtual] |
| virtual matrix::Matrix calcMotor | ( | int | action | ) | [protected, pure virtual] |
returns action Matrix from discrete actions, to be overwritten
| virtual double calcReinforcement | ( | ) | [protected, pure virtual] |
returns the reinforcement (reward), to be overwritten
| virtual int calcState | ( | ) | [protected, pure virtual] |
returns state, to be overwritten
| void fillMotorBuffer | ( | const motor * | y_, | |
| int | number_motors | |||
| ) | [protected, virtual] |
puts the motors in the ringbuffer
| void fillSensorBuffer | ( | const sensor * | x_, | |
| int | number_sensors | |||
| ) | [protected, virtual] |
puts the sensors in the ringbuffer
| virtual int getActionNumber | ( | ) | [protected, pure virtual] |
returns number of actions, to be overwritten
| static ClassicReinforceConf getDefaultConf | ( | ) | [inline, static] |
| list< Inspectable::iparamkey > getInternalParamNames | ( | ) | const [virtual] |
The list of the names of all internal parameters given by getInternalParams().
The naming convention is "v[i]" for vectors and "A[i][j]" for matrices, where i, j start at 0.
Reimplemented from Inspectable.
| list< Inspectable::iparamval > getInternalParams | ( | ) | const [virtual] |
| virtual int getMotorNumber | ( | ) | const [inline, virtual] |
returns the mumber of motors the controller was initialised with or 0 if not initialised
Implements AbstractController.
| Configurable::paramval getParam | ( | const paramkey & | key | ) | const [virtual] |
| Configurable::paramlist getParamList | ( | ) | const [virtual] |
The list of all parameters with there value as allocated lists.
Note that these are only parameters that are managed manually (with setParam, getParam)
Reimplemented from Configurable.
| virtual int getSensorNumber | ( | ) | const [inline, virtual] |
returns the number of sensors the controller was initialised with or 0 if not initialised
Implements AbstractController.
| virtual int getStateNumber | ( | ) | [protected, pure virtual] |
returns number of state, to be overwritten
| list< Inspectable::IConnection > getStructuralConnections | ( | ) | const [virtual] |
Specifies which parameter matrix forms a connection between layers (in terms of a neural network) The orderning is not important.
Reimplemented from Inspectable.
| list< Inspectable::ILayer > getStructuralLayers | ( | ) | const [virtual] |
Specifies which parameter vector forms a structural layer (in terms of a neural network) The ordering is important.
The first entry is the input layer and so on.
Reimplemented from Inspectable.
| void init | ( | int | sensornumber, | |
| int | motornumber, | |||
| RandGen * | randGen = 0 | |||
| ) | [virtual] |
initialisation of the controller with the given sensor/ motornumber Must be called before use.
The random generator is optional.
Implements AbstractController.
| void management | ( | ) | [protected, virtual] |
handles inhibition damping etc.
| void putInBuffer | ( | matrix::Matrix * | buffer, | |
| const matrix::Matrix & | vec, | |||
| int | delay = 0 | |||
| ) | [protected] |
| bool restore | ( | FILE * | f | ) | [virtual] |
| void setManualControl | ( | bool | mControl, | |
| int | action_ = 0 | |||
| ) |
enables/disables manual control, action_ is the sat network number to be used if mControl is false, action is ignored
performs one step (includes learning).
Calulates motor commands from sensor inputs.
Implements AbstractController.
performs one step without learning. Calulates motor commands from sensor inputs.
Implements AbstractController.
| bool store | ( | FILE * | f | ) | const [virtual] |
int action [protected] |
action
unsigned short buffersize [protected] |
ClassicReinforceConf conf [protected] |
bool initialised [protected] |
int managementInterval [protected] |
interval between subsequent management calls
bool manualControl [protected] |
True if actions (sats) are selected manually.
unsigned short number_motors [protected] |
unsigned short number_sensors [protected] |
int oldaction [protected] |
old action
double oldreward [protected] |
old reward (nicer for plotting)
double reward [protected] |
current reward
int state [protected] |
current state
int t [protected] |
matrix::Matrix* x_buffer [protected] |
matrix::Matrix* x_context_buffer [protected] |
matrix::Matrix* y_buffer [protected] |
1.4.7