This page contains links to several software tools/packages I have written. This code is provided as-is, with no guarantees. If you do happen to find any bugs, please let me know!
High-Dimensional Bayesian Regularised Regression with the BayesReg Package
MATLAB and R code to implement efficient, high dimensional Bayesian regression with continuous shrinkage priors. The package is very flexible, fast and highly numerically stable, particularly in the case of the horseshoe/horseshoe+, for which the heavy tails of the prior distributions cause problems for most other implementations. We have just released Version 1.3 with the following additional features:
- Improved summary display
- Support for MATLAB tables has been added
- Support for handling categorical predictors has been added
- A prediction function that also provides prediction statistics has been added
- Example scripts have been updated and improved
- Minor bug in the computation of the R2 statistic has been fixed
BayesReg supports the following data models:
- Gaussian (“L2 errors”)
- Laplace (“L1 errors”)
- Student-t (very heavy tails)
- Logistic regression (binary data)
It also supports a range of state-of-the-art continuous shrinkage priors to handle different underlying regression model structures:
- Ridge regression (“L2″ shrinkage/regularisation)
- LASSO regression (“L1″ shrinkage/regularisation)
- Horseshoe regression (global-local shrinkage for sparse models)
- Horseshoe+ regression (global-local shrinkage for ultra-sparse models)
The MATLAB code for Version 1.2 of the package can be downloaded here, and the R code can be obtained from CRAN under the package name “bayesreg”. This R package can also be installed from within R by using the command “install.packages(“bayesreg”)”.
If you use the package, and wish to cite it in your work, please use the reference below. If you find this package useful, it would be great if you could leave a comment or rating on the MathWorks File Exchange page.
- “High-Dimensional Bayesian Regularised Regression with the BayesReg Package”, E. Makalic and D. F. Schmidt, arXiv:1611.06649 [stat.CO], 2016
Robust lasso regression with Student-t residuals
This MATLAB code implements lasso based estimation of linear models in which the residuals follow a Student-t distibution using the expectation-maximisation algorithm. By varying the degrees-of-freedom parameter of the Student-t likelihood, the model can be made more resistant to outlying observations.
The software has the following features:
- Automatically generate complete lasso regularisation paths for a given degrees-of-freedom
- Selection of lasso regularisation parameter and degrees-of-freedom using either cross-validation or information criteria.
The code is straightforward to run, efficient and comes with several examples that recreate the analyses from the paper below. To cite this toolbox, please use the reference below:
The code can be obtained from MathWorks File Exchange.
- “Robust Lasso Regression with Student-t Residuals”, D. F. Schmidt and E. Makalic, Lecture Notes in Artificial Intelligence, to appear, 2016
Bayesian LASSO for estimation of stationary autoregressive models
This MATLAB code implements the Bayesian LASSO sampling hierarchy for inference of autoregressive models from an observed time series. The idea behind the approach is place Laplace prior distributions over the partial autocorrelations of an AR(k) model, which leads to a relatively simple Gibbs’ sampling scheme, and guarantees stationarity. Both empirical Bayes and fully Bayesian estimation of the shrinkage hyperparameter is available.
Once downloaded and extracted, all 3 folders/subfolders should be added to the MATLAB path. [code]
- "Estimation of Stationary Autoregressive Models with the Bayesian LASSO", D. F. Schmidt and E. Makalic, Journal of Time Series Analysis, Vol. 34, No. 5, pp. 517--531, 2013