Softmax

The softmax function takes as input a tuple $z$ of $K$ real numbers, and normalizes it into a probability distribution consisting of $K$ probabilities proportional to the exponentials of the input numbers.

\[ \sigma(z)_i = \frac{e^{z_i}}{\sum_{j=1}^K e^{z_j}} \]