General software
Tensorlab Version 2.02 (May 2014)
Contributed by Lieven De Lathauwer
Download the software
Tensorlab is a MATLAB toolbox that offers algorithms for:
- structured data fusion: define your own (coupled) matrix and tensor factorizations with structured factors and support for dense, sparse and incomplete data sets,
- tensor decompositions: canonical polyadic decomposition (CPD), multilinear singular value decomposition (MLSVD), block term decompositions (BTD) and low multilinear rank approximation (LMLRA),
- complex optimization: quasi-Newton and nonlinear-least squares optimization with complex variables including numerical complex differentiation,
- global minimization of bivariate polynomials and rational functions: both real and complex exact line search (LS) and real exact plane search (PS) for tensor optimization,
- and much more: cumulants, tensor visualization, estimating a tensor’s rank or multilinear rank, …
Information Theoretical Estimators Toolbox (ITE, 2012-)
Author: Zoltán Szabó.
Environment: Matlab, Octave.
License: GPLv3(>=).
Homepage of ITE
We present ITE (Information Theoretical Estimators) a free and open source, multi-platform, Matlab/Octave toolbox that is capable of estimating many different variants of entropy, mutual information, divergence, association measures, cross quantities and kernels on distributions. Thanks to its highly modular design, ITE supports additionally (i) the combinations of the estimation techniques, (ii) the easy construction and embedding of novel information theoretical estimators, and (iii) their immediate application in information theoretical optimization problems. ITE also includes (i) a prototype application in independent subspace analysis and its extensions, (ii) several consistency tests (analytical vs estimated value), (iii) illustrations for information theoretical image registration, and (iv) distribution regression with applications in supervised entropy learning and aerosol prediction based on multispectral satellite images.
ICA by entropy bound minimization (EBM) and entropy rate bound minimization (ERBM) (2010)
Contributed by MLSP-Lab (Z. Boukouvalas, X.-L. Li,G.-S. Fu, and T. Adali)
Environment: Matlab
Download the software
We present two general purpose algorithms that effectively take higher-order statistics (HOS) into account: EBM that provides a flexible density matching mechanism at a reasonable computational cost, and ERBM that incorporates sample dependence (non-whiteness) as an additional statistical property to the flexible density matching of EBM to further improve the performance.
The algorithms do not assume that the demixing matrix is orthogonal (note that whitening implies orthogonality for the demixing matrix only when sample size tends to infinity). For density matching, an entropy bound estimator based on the maximum entropy principle is used (hence combining advantages of parametric and non-parametric methods). Using four nonlinearities as measuring functions a wide range of distributions, such as symmetric or skewed, heavy-tailed or not heavy-tailed, unimodal and bimodal are effectively approximated. Both algorithms have been shown to lead to improved performance for both simulated and a number of real-world problems.
Efficient version of algorithm FastICA (2009)
Contributed by Zbynek Koldovsky
Programming language: Matlab
Download the software
We have derived an improved version of the FastICA algorithm which is asymptotically efficient, i.e., its accuracy given by the residual error variance attains the Cramerr-Rao? lower bound. The error is thus as small as possible. The algorithm is taylored to achieve the efficiency when the probability distribution of the independent signal components belongs to the class of generalized Gaussian distributions. Nevertheless, this can be enhanced for other distributions via any kind of extension of adaptive choice of nonlinearity used. We name the algorithm Efficient FastICA (EFICA). The algorithm has improved stability (global convergence properties) and the computational complexity is only slightly higher than that of the original symmetric FastICA.
Deterministic algorithms for separation of non-negative sources (2008)
Contributed by Tsung-Han Chan
Programming language: Matlab
Download the software
We provide a set of non-negative blind source separation (nBSS) algorithms, based on convex analysis and optimization, for over-determined instantaneous mixtures. The algorithms are computationally efficient, and do not rely on the assumption of independent (or uncorrelated) sources, the entrenched premise in many existing BSS frameworks. Instead, the algorithms were based on either source local dominance assumption (likely to be satisfied by sparse or high-contrast sources), or maximum/minimum volume (determinant measure) criteria. Applications include biomedical image analysis and hyperspectral image analysis (for remote sensing).
Soft-LOST: algorithm for under-determined instantaneous mixtures (2008)
Contributed by Paul D. O’Grady
Programming language: C Matlab
Download the software
Soft-LOST (Line Orientation Separation Technique) identifies line orientations within a scatter plot using an EM procedure, where the E-step calculates posterior probabilities assigning data points to lines and M-step repositions the lines to match the points assigned to them. This method, combined with a transformation into a sparse domain and an L1-norm optimisation, constitutes a blind source separation algorithm for the under-determined case.
Paul D. O’Grady and Barak A. Pearlmutter: “Soft-LOST: EM on a mixture of oriented lines”, In Fifth International Conference on Independent Component Analysis, LNCS 3195, pages 430-436, Granada, Spain, Sept. 22-24 2004. Springer-Verlag?.
Fast Kernel ICA using an approximate Newton method (2007)
Contributed by Arthur Gretton
Matlab, with some C routines
Download the software
Kernel ICA uses kernel measures of statistical independence to separate linearly mixed sources. We have made this process much faster by using an approximate Newton-like method on the special orthogonal group to perform the optimisation.
Non-parametric ICA algorithm (2006)
Contributed by Erik G. Learned-Miller
Programming language: Matlab
Download the software
My ICA algorithm, RADICAL, has been described in the Journal of Machine Learning Research, Volume 4, 2003, under the article “ICA Using Spacings Estimates of Entropy”. After whitening, our algorithm minimizes the sum of the entropies of the mixture marginals under rotations. It is relatively fast for a non-parametric method. It outperforms many of the parametric methods (FastICA, JADE) and also methods like Bach and Jordan’s kernel ICA. Both a two-dimensional version and an N-dimensional version are provided.
The FastICA software package (2005)
Contributed by Aapo Hyvarinen
Programming language: Matlab
Download the software
The FastICA software package implements the fixed-point algorithm for ICA. Main features:
- computationally very efficient algorithm
- graphical user interface
- one-by-one estimation of the independent components (optionally)
- estimation of both sub- and supergaussian independent components.
Accurate ICA algorithm & reliability tests (2004)
Contributed by Harald Stoegbauer
Programming language: C and Matlab
Download the software
MILCA is an Independent Component Analysis (ICA)-algorithm which uses an accurate Mutual Information (MI) estimator to find the least dependent components under a linear transformation. The MI estimator is data efficient, adaptive and has minimal bias. The direct access to the dependency between the components can be used not only as a contrast function to find the most independent components but also in performance tests, reliability tests and for cluster analysis (when the signals are not independent). We provide here algorithms for performing ICA (or rather LCA) and these tests. Furthermore, we extended all these approaches to use additional time structure information which is often contained in physical data.
BSS in 2 lines of matlab code (2003)
Contributed by Lucas C. Parra
Programming language: Matlab
Download the software
Blind source separation (BSS) can often be formulated as a simple eigenvalue problem to simultaneously diagonalize the covariance matrix (X*X’) and an additional matrix Q leading to the simple two lines of matlab code:
[W,D] = eig(X*X',Q); % compute unmixing matrix W S = W'*X; % compute sources S.
Matrix Q takes on different forms depending on whether one exploits non-stationarity, non-whiteness or non- Gaussianity of the sources, as explained here. The caveat with this simple solution is numerical stability.
TFBSS: algorithm for blind separation of non-stationary sources (2003)
Contributed by Cédric Févotte
Programming language: Matlab
Download the software
TFBSS performs blind separation of (over)determined multiplicative mixtures of non-stationary real-valued sources. TFBSS is based on the joint-diagonalization of whitened and noise-compensated Spatial Time-Frequency? Distribution (STFD) matrices of the observations, corresponding to single auto-terms positions.
Off-line linear ICA algorithm, instantaneous case (2003)
Contributed by Juan José Murillo Fuentes
Programming language: Matlab
Download the software
OGWE is a computationally optimized fourth-order based ICA/BSS algorithm for the instantaneous case. It uses the SICA algorithm, a quite similar method to the ME-ICA by Comon (1994) with lower complexity. Besides it does not have the limitation in the number of sources the JADE suffers from. Other contrast may be also minimized with this algorithm: AML, AEML, EML, MaSSFOC, ML, MK, SKSE, …
Kernel ICA (2003)
Contributed by Francis Bach
Programming language: Matlab
Download the software
The kernel-ica package is a Matlab program that implements the Kernel ICA algorithm for linear independent component analysis (ICA). The Kernel ICA algorithm is based on the minimization of a contrast function based on kernel ideas.
TCA – Tree dependent component analysis (2003)
Contributed by Francis Bach
Programming language: Matlab
Download the software
The tca package is a Matlab program that implements the tree-dependent component analysis (TCA) algorithms that extends the independent component analysis (ICA), where instead of looking for a linear transform that makes the data components independent, we are looking for components that can be best fitted in a tree structured graphical model. The TCA model can be applied in any situation where the data can be assumed to have been transformed by an unknown linear transformation. In addition, the TCA algorithm can be specialized to provide a principled way of finding clusters in ICA, where components in the same cluster are dependent, but independent from components in other clusters.
Algorithm for non-orthogonal approximate joint diagonalization (2002)
Contributed by Arie Yeredor
Programming language: Matlab 5
Download the software
An iterative alternating-directions type algorithm (named “AC-DC”) for the approximate joint diagonalization of a set of matrices. The diagonalizing matrix is a general (not necessarily orthogonal) matrix, and the diagonalization is optimized in the direct Least-Squares? sense. Two different routines are provided: one for the case of Hermitian target matrices and one for the case of symmetric matrices (the distinction is only for the complex-valued case). Also included is an optional initialization function for providing an intelligent initial guess to the AC-DC algorithm in the Hermitian case. Updated on Jun. 20, 2002 : no longer restricted to Hermitian matrices.
Pearson-ICA algorithm for wide class of sources (sub-,super-Gaussian, skewed,…) (2000)
Contributed by Jan Eriksson
Programming language: Matlab
Download the software
The package provides the Matlab (5.x) functions needed for the use of the Pearson-ICA algorithm as described in
Karvanen, J.,Eriksson, J., and Koivunen, V.: “Pearson System Based Method for Blind Separation”, Proceedings of Second International Workshop on Independent Component Analysis and Blind Signal Separation, Helsinki 2000, pp. 585–590.
The algorithm is proposed to solve the standard noiseless linear ICA problem, i.e. the ICA model is Y=AS, where the number of sources s_i is equal to the number of observations y_i.
EGLD-ICA algorithm for wide class of sources (sub-,super-Gaussian, skewed,…) (2000)
Contributed by Jan Eriksson
Programming language: Matlab
Download the software
The package provides the Matlab (5.x) functions needed for the use of the EGLD-ICA algorithm as described in
Eriksson, J., Karvanen, J., and Koivunen, V.: “Source Distribution Adaptive Maximum Likelihood Estimation of ICA Model”, Proceedings of Second International Workshop on Independent Component Analysis and Blind Signal Separation, Helsinki 2000, pp. 227–232.
The algorithm is proposed to solve the standard noiseless linear ICA problem, i.e. the ICA model is Y=AS, where the number of sources s_i is equal to the number of observations y_i.
Two batch cumulant-based ICA algorithms: JADE and SHIBBS (1999)
Contributed by Jean-Francois Cardoso
Programming language: Plain C, Matlab and a Matlab interface to C
Download the software
JADE and SHIBBS are two cumulant-based batch algorithms for source separation. They are described in the paper High-order contrasts for independent component analysis, Neural Computation’ 11(1):157–192, 1999. JADE has proven itself successful in many applications but I have less experience with SHIBBS whose current implementation may require some tuning. Both algorithms use algebraic ideas to optimize a 4th-order measure of independence under the whiteness constraint. This approach makes them “parameter-free”. The SHIBBS algorithm is supposed to lower the memory requirements of JADE (it scales as N^3 rather than N^4 for JADE).
This package contains source code in C and in Matlab as well as a Matlab interface to the C code. Very minor adjustments (read: deletions) may be needed to compile the C code if Matlab is not present. The C code is *very naive* because this is basically my first C program: feedback is welcome. This is distributed under the GPL.
Batch ICA algorithms for more or fewer sensors than sources (1989, 1994, 1998)
Contributed by Pierre Comon
Programming language: Matlab 5.x, C
Download the software
The following 3 packages can be downloaded all together with the file IcaComonMatlab?.tar.gz:
- Closed-form block Source separation codes maximizing the contrast (2,4), the sum of squares of fourth-order marginal cumulants for real data and mixtures in presence of unknown noise. References: Vail’89, Chamrousse’91, Eurasip Signal Processing’94.
- Closed-form block Source separation codes maximizing the contrast (1,4), the sum of fourth-order marginal cumulants, when sources have a kurtosis of given constant sign. References: ICASSP’97, SPAWC’97, ICA’99
- Codes for identifying a complex mixture of 3 sources received on 2 sensors. References: SPIE’98, IEEE TSP’04