Normalized Linear Combination of Activations

Introduced by Pishchik in Trainable Activations for Image Classification

The Normalized Linear Combination of Activations, or NormLinComb, is a type of activation function that has trainable parameters and uses the normalized linear combination of other activation functions.

$$NormLinComb(x) = \frac{\sum\limits_{i=0}^{n} w_i \mathcal{F}_i(x)}{\mid \mid W \mid \mid}$$

Source: Trainable Activations for Image Classification

Read Paper See Code

Papers

Paper	Code	Results	Date	Stars

Tasks

Task	Papers	Share
Image Classification	1	100.00%

Usage Over Time

This feature is experimental; we are continuously improving our matching algorithm.

Components

Component	Type	Add Remove
ReLU	Activation Functions	(optional)
Sigmoid Activation	Activation Functions	(optional)
Softsign Activation	Activation Functions	(optional)
Tanh Activation	Activation Functions	(optional)

Categories

Add Remove

Activation Functions

Adaptive Activation Functions