Nonlinear Modelling of Technical Processes by Support Vector Machines and Neural Networks
One of the most important tasks in automatic control is the modeling of technical processes, which is necessary for almost all further applications. Whereas theoretical modeling describes technical processes using the relevant first principles, also experimental modeling (identification) is more and more employed. This technique builds models based on measured signals. A particularly challenging field is the identification of nonlinear dynamic processes.
For both classical modelling techniques and neural networks (which have been frequently employed in the last several years), the main problems are the determination of the model structure and the nonlinear optimization of the model. The results are always dependent of the current data set and the chosen model structure. This particularly implies the danger of »overfitting«.
Starting in the middle nineties, support vector machines (SVMs) have seen a growing popularity. These approximators have their roots in the statistical lerning theory developed by Vapnik and Chervonenkis in the sixties, which treats a measured data set as a sample drawn from an unknown main unit. The goal is to find a model that is not optimal on the data set but on the main unit.
This is achieved by the concept illustrated by the figure on the right: SVMs optimize the model's »flatness« whereas the model error must be smaller than a given threshold. Sample data that do not fulfill these requirements or lie on the edge of the error tube, are support vectors and become therefore a part of the model. The larger the acceptable error is, the less support vectors are present and the simpler is the model.
This concept leads to a QP problem – a standard problem in nonlinear optimization with a single optimum. For this reason, the computation of SVMs can be done by ordinary QP algorithms in the case of small and medium size data sets.
The objective of the research project is to investigate the possibility of employing support vector machines as nonlinear process models in the field of automatic control. The main topics of interest are the problems that typically arise with this application, e.g.:
- Development of algorithms for large data sets (since the computational effort for SVMs is dependent on the data set size).
- Methods for extracting characteristic support vectors, and therefore obtaining a compact representation of support vector machines.
First, new methods are investigated theoretically, implemented, and tested in simulations. Then, the resulting SVMs are compared with existing models for real test processes. Additional material on SVMs, tutorials etc. can be found at Kernel-Machines.Org.