Rank constraints and error quantification in restricted complexity problems

This thesis addresses two issues that arise in restricted complexity estimation problems: The first is estimation subject to rank constraints. The second corresponds to uncertainty quantification when the amount of data available is small relative to the number of variables to be estimated. In many practical problems one wishes to choose a simple solution from a set of possible solutions. The reasons for this can be many fold. For example, in design problems, one may know that a simple solution is possible. However, one does not know how to obtain such a simple solution from a large set of possible alternatives. In estimation problems, one may deliberately restrict the set of possible solutions to avoid over-fitting of noisy data. We term the class of problems having simple solutions restricted complexity problems. The first part of the thesis address restricted problems where the restriction on complexity can be related to constraining the rank of a particular matrix. This leads us to address rank-constrained optimization problems. The second part of the thesis focuses on quantification of estimation-error. It is well known that, when the amount of data available for estimation is small, the variance error could be significantly large. In these circumstances it is beneficial to, not only, have an estimated value for the parameters but also to be able to quantify the associated error. However, most of the existing methods for error quantification rely upon asymptotic results with large data. We focus on parametric uncertainty quantification for finite data estimation, with an extension to the related problem of moving horizon estimation. The third part of the thesis, focuses on quantification of estimation errors when the complexity of the model is deliberately chosen to be smaller than the complexity of the "true" model. This has motivated a novel approach, commonly known in the literature by the generic title "model error modelling", to uncertainty quantification. This has been a central theme in several areas including statistics, time series analysis, econometrics and system identification. In the third part of the thesis we address the problem of model error modelling for dynamic system identification.