TY  - JOUR
T1  - Performance Improvement Method of Fully Connected Neural Network
Using Combined Parametric Activation Functions
AU  - Ko, Young Min 
AU  - Li, Peng Hang 
AU  - Ko, Sun Woo 
JO  - The Transactions of the Korea Information Processing Society
PY  - 2022
DA  - 2022/1/30
DO  - https://doi.org/10.3745/KTSDE.2022.11.1.1
KW  - Fully Connected Neural Network
KW  - Nonlinear Activation Function
KW  - Combined Parametric Activation Function
KW  - Learning
AB  - Deep neural networks are widely used to solve various problems. In a fully connected neural network, the nonlinear activation function
is a function that nonlinearly transforms the input value and outputs it. The nonlinear activation function plays an important role in solving
the nonlinear problem, and various nonlinear activation functions have been studied. In this study, we propose a combined parametric
activation function that can improve the performance of a fully connected neural network. Combined parametric activation functions can
be created by simply adding parametric activation functions. The parametric activation function is a function that can be optimized in
the direction of minimizing the loss function by applying a parameter that converts the scale and location of the activation function according
to the input data. By combining the parametric activation functions, more diverse nonlinear intervals can be created, and the parameters
of the parametric activation functions can be optimized in the direction of minimizing the loss function. The performance of the combined
parametric activation function was tested through the MNIST classification problem and the Fashion MNIST classification problem, and
as a result, it was confirmed that it has better performance than the existing nonlinear activation function and parametric activation function.