Robot Simulator of the Robotics Group for Self-Organization of Control: regularisation.h File Reference


Functions
double	sqr (double x)
double	g (double z)
	neuron transfer function
double	g_s (double z)
	first dervative
double	g_derivative (double z)
	first dervative with smoothing for large z
double	g_s_inv (double z)
	inverse of the first derivative
double	g_s (double z, double xsi)
	$g'(z+xsi) = 1-(tanh(z+xsi))^2$ with additional clipping
double	g_s_soft (double z, double xsi)
	soft version: $g'(z+xsi) = 1/(1+(z+xsi)^2$ with additional clipping
double	g_ss_div_s (double z, double xsi)
	an exact formula for g''/g'= -2g(Z), with clipped Z = z+xsi
double	g_ss_div_s_soft (double z, double xsi)
	an soft formula for g''/g' = -2Z, with clipped Z = z+xsi
double	derive_g_s_inv_exact_clip (double z, double xsi)
	with $g'(z) = 1-(g(z+\xi))^2$ we get $\frac{\partial}{\partial z} \frac{1}{g'(Z)} = \frac{g''}{g'^2}$ again with clipped Z
double	g_s_expand2 (double z, double xsi)
	$g'(z) = 1-(z+\xi)^2$ which is the series expansion to the second order
double	g_s_inv_expand2 (double z, double xsi)
	$\frac{1}{g'(z)} \approx 1+(z+\xi)^2$ with geometric series approximation
double	g_ss_div_s_expand2 (double z, double xsi)
	$\frac{g''(z)}{g'(z)} \approx 2(z+\xi)(1+(z+\xi)^2)$ with geometric series approximation
double	squash (double z)
	squashing function (-0.1 to 0.1), to protect against to large weight updates
double	squash (void *d, double z)
	squashing function with adjustable clipping size, to protect against too large weight updates

Function Documentation

double derive_g_s_inv_exact_clip	(	double	z,
		double	xsi
	)			`[inline]`

with

$g'(z) = 1-(g(z+\xi))^2$

we get

$\frac{\partial}{\partial z} \frac{1}{g'(Z)} = \frac{g''}{g'^2}$

again with clipped Z

double g ( double z ) [inline]

neuron transfer function

double g_derivative ( double z ) [inline]

first dervative with smoothing for large z

$g'(z+xsi) = 1-(tanh(z+xsi))^2$

with additional clipping

double g_s ( double z ) [inline]

first dervative

$g'(z) = 1-(z+\xi)^2$

which is the series expansion to the second order

double g_s_inv ( double z ) [inline]

inverse of the first derivative

$\frac{1}{g'(z)} \approx 1+(z+\xi)^2$

with geometric series approximation

soft version:

$g'(z+xsi) = 1/(1+(z+xsi)^2$

with additional clipping

an exact formula for g''/g'= -2g(Z), with clipped Z = z+xsi

$\frac{g''(z)}{g'(z)} \approx 2(z+\xi)(1+(z+\xi)^2)$

with geometric series approximation

an soft formula for g''/g' = -2Z, with clipped Z = z+xsi

double sqr ( double x ) [inline]

squashing function with adjustable clipping size, to protect against too large weight updates

double squash ( double z ) [inline]

squashing function (-0.1 to 0.1), to protect against to large weight updates

regularisation.h File Reference