Derivatives

The derivative of the identity function is just 1. That is, if f(x) is the identity then

The derivative of the step function is not defined which is exactly why it isn't used.

The nice feature of sigmoids is that their derivatives are easy to compute. If f(x) is the logistic function above then .

> diff(f3(x,a),x);

> simplify(f3(x,a)*(1-f3(x,a)));

This is also true of hyperbolic tangent . If f(x) is tanh the

> diff(tanh(x),x);

>