S. Raudys
TECHNICAL REPORT ON RESEARCH ACTIVITES IN LAFORIA-IBP, University PARIS VI in a period 15 February - 15 August 1994.The technical report also contains three articles:
Unexpected small sample properties of the linear perceptrons
S. Raudys
(16p anglais)
Abstract: An analytical expression for the generalization error
of a zero empirical error classifier is derived. It shown that
small training set behavior of the linear perceptron essentially
differs from that of conventional parametric classification
rules: for parametric linear and quadratic discriminant functions
the dimensionality/samples size ratio is p/N and p2/N, for the
zero empirical error classifier we have significantly better ra
tio p/N2. Asymptotic formula for the generalization error explain
margin A influence on the generalization error. Equation obtained
shows that the training process first starts with a decreasing
the generalization error, but can later increase it.The linear
perceptron trained by a proposed "targets 0.4 & 0.0001 " strategy
can design complex nonlinear decision boundaries and can be use
ful in solving nonlinear classification problems.
Optimal regularization of linear and nonlinear perceptrons
S. Raudys, M. Skurichina,T. Cibas,P. Gallinari
(10p anglais)
Abstract. We derive an analytical formula for the generalization
error of linear adaptive classifiers trained with weight decay.
Analytical and experimental results are then presented to analyze
the optimal value of regularization parameters as a function of
the training set size.
Generalization errors of Adaptive Linear Classifiers
S. Raudys
(26p anglais)
The cost function of the perceptron which minimizes a squared
loss function with nonlinearity depends upon target values: in
one extreme case it is close to ADALINE and a standard Fisher
linear classifier and in another one to a minimum empirical er
ror classifier. An analytical equation for a generalization error
of the "zero empirical error classifier" is derived for a case
when the distributions of pattern classes share a common covari
ance matrix. It is shown that for small training set, the behav
ior of this classifier deeply differs from that of conventional
parametric classifiers: for Fisher classifier the dimensionali
ty/sample size ratio is ,p/N, for the zero empirical error clas
sifier in spherical pattern distribution case we have signifi
cantly better ratio - p/N2. A successful initialization of the
linear perceptron contains a large amount of information which
can dramatically reduce the generalization error. The asymptotic
formula obtained explains the influence of targets on the gener
alization error: the training process first decreases the gener
alization error, but can later increase it. A new training strat
egy with varying targets, called "targets 0.4 and 0.0001" tech
nique is proposed to improve generalization.