Least-Squares Independence Regression (LSIR)

Main Idea

LSIR learns the additive noise model through minimization of an estimator of the squaredloss mutual information between inputs and residuals:

{mathbf alpha}^* := mathop{textnormal{argmin}}_{{mathbf alpha}} widehat{textnormal{SMI}}(X, Y - f_{{mathbf alpha}}(X)).

A notable advantage of LSIR over existing approaches is that tuning parameters such as the kernel width and the regularization parameter can be naturally optimized by cross-validation, allowing us to avoid overfitting in a data-dependent fashion.

Features

  • All the model parameters are automatically tuned by cross-validation.

  • Can be used for causal direction inference.

Download

Usage

  • Download & compile the source code.

  • Run the script.

Causal direction inference

  • Toy data

Acknowledgement

I am grateful to Prof. Masashi Sugiyama for his support in developing this software.

Contact

I am happy to have any kind of feedbacks. E-mail: textnormal{yamada@sg.cs.titech.ac.jp}

Reference

Yamada, M., & Sugiyama, M.
Dependence minimizing regression with model selection for non-linear causal inference under non-Gaussian noise.
In Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI2010), pp.643-648, Atlanta, Georgia, USA, Jul. 11-15, 2010.
[paper]