TY  - JOUR
AB  - Neural networks based on high-dimensional random feature generation have become popular under the notions extreme learning machine (ELM) and reservoir computing (RC). We provide an in-depth analysis of such networks with respect to feature selection, model complexity, and regularization. Starting from an ELM, we show how recurrent connections increase the effective complexity leading to reservoir networks. On the contrary, intrinsic plasticity (IP), a biologically inspired, unsupervised learning rule, acts as a task-specific feature regularizer, which tunes the effective model complexity. Combing both mechanisms in the framework of static reservoir computing, we achieve an excellent balance of feature complexity and regularization, which provides an impressive robustness to other model selection parameters like network size, initialization ranges, or the regularization parameter of the output learning. We demonstrate the advantages on several synthetic data as well as on benchmark tasks from the UCI repository providing practical insights how to use high-dimensional random networks for data processing
DA  - 2012
DO  - 10.4236/jilsa.2012.43024
KW  - Extreme Learning Machine
KW  - Reservoir Computing
KW  - Model Selection
KW  - Feature Selection
KW  - Model Complexity
KW  - Intrinsic Plasticity
KW  - Regularization
LA  - eng
IS  - 3
M2  - 230
PY  - 2012
SN  - 2150-8402
SP  - 230-246
T2  - Journal of Intelligent Learning Systems and Applications
TI  - Regularization by Intrinsic Plasticity and its Synergies with Recurrence for Random Projection Methods
UR  - https://nbn-resolving.org/urn:nbn:de:0070-pub-25086144
Y2  - 2024-11-22T05:09:30
ER  -