Skip to main content
Department of Information Technology

This page is a copy of research/systems_and_control/signalproc/topics/order (Wed, 31 Aug 2022 15:09:07)

Model selection

The parametric (or model-based) methods of signal processing require often not only the estimation of a vector of real-valued parameters but also the selection of one or several integer-valued parameters that are equally important for the specification of a data model. Examples of these integer-valued parameters of the model include the orders of an autoregressive moving average model, the number of sinusoidal components in a sinusoids-in-noise signal, and the number of source signals impinging on a sensor array. In each of these cases, the integer-valued parameters determine the dimension of the parameter vector of the data model, and they must be estimated from the data.

The task of selecting the above mentioned integer parameters is known as model (order) selection. Our research group has in the last few years published several papers on this topic in well-known journals. The papers concern the following:

Information Criteria
A review on order selection algorithms like Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC). [1], [3]
Cross Validation Rules for Model Order Selection
Two previously presented order selection rules based on cross-validation are revisited. [2]
The Multi-model Approach
By considering several models instead of only one and weighing them in a specific manner, better performance for e.g. prediction can be obtained. [4], [5]
Estimation of sparse models
A sparse model, by definition, comprises many parameters equal to zero. We consider methods to select which parameters of a model should be included (i.e. given non-zero values), and which should not. See Figure 1. [6], [7]

sparse_channel2.GIF

Figure 1: Example of a sparse channel with L = 17 taps where we want to estimate n = 8 taps. A non-sparse model would consist of the 8 initial taps, whereas a sparse model could consist, e.g., of the 8 largest taps.

Empirical Bayesian model selection
Following a Bayesian framework it is possible to satisfy certain optimality criteria, such as selecting the model with the highest posterior probability, and so on. However, the use of Bayesian methods requires the specification of certain a priori parameters, which may not be available. We have studied empirical Bayesian solutions to the model selection problem: In this type of approach the values of the a priori parameters are first estimated from the data, and then used as if they were the true parameter values.

Selected publications

[[OPUS style=numbered;pubids=87401,87400,81128,73905,73906,73902,73908]]

Updated  2022-08-31 15:09:07 by Victor Kuismin.