Selected bibliography
Tensor decompositions
- A. Anandkumar, R. Ge, D. Hsu, S. M. Kakade, and M. Telgarsky. Tensor decompositions for learning latent variable models, Journal of Machine Learning Research, 2014 (to appear).
- B. Barak, J. Kelner, and D. Steurer. Dictionary learning and tensor decomposition via the sum-of-squares method. 2014.
- A. Bhaskara, M. Charikar, and A. Vijayaraghavan. Uniqueness of tensor decompositions and polynomial identifiability of latent variable models. COLT, 2014.
- P. Comon, X. Luciani, and A.L.F. de Almeida. Tensor decompositions, alternating least squares and other tales. Journal of Chemometrics, 2009.
- L. de Lathauwer, B. de Moor, and J. Vandewalle. On the best rank-1 and rank-(R_1,R_2,...,R_n) approximation and applications of higher-order tensors. SIAM Journal on Matrix Analysis and Applications, 2000.
- R. Harshman. Foundations of the PARAFAC procedure: model and conditions for an 'explanatory' multi-mode factor analysis. UCLA Working Papers in Phonetics, 1970.
- F.L. Hitchcock. The expression of a tensor or a polyadic as a sum of products. Journal of Mathematics and Physics, 1927.
- F.L. Hitchcock. Multiple invariants and generalized rank of a p-way matrix or tensor. Journal of Mathematics and Physics, 1927.
- S. Leurgans, R. Ross, and R. Abel. A decomposition for three-way arrays. SIAM Journal on Matrix Analysis and Applications, 1993.
- T.G. Kolda and B.W. Bader. Tensor decompositions and applications. SIAM Review, 2009.
- T.G. Kolda and J.R. Mayo. Shifted power method for computing tensor eigenpairs. SIAM Journal on Matrix Analysis and Applications, 2011.
- J.B. Kruskal. Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear Algebra and Its Applications, 1977.
- T. Zhang and G. Golub. Rank-one approximation to high order tensors. SIAM Journal on Matrix Analysis and Applications, 2001.
Method-of-moments via tensor decomposition algorithms
- E.S. Allman, C. Matias, J.A. Rhodes, and S. Sullivant. Identifiability of parameters in latent structure models with many observed variables. Annals of Statistics, 2009.
- A. Anandkumar, D.P. Foster, D. Hsu, S.M. Kakade, and Y.K. Liu. A spectral algorithm for latent Dirichlet allocation, NIPS, 2012.
- A. Anandkumar, D. Hsu, and S.M. Kakade. A method of moments for mixture models and hidden Markov models. COLT, 2012.
- J. Anderson, M. Belkin, N. Goyal, L. Rademacher, and J. Voss. The more, the merrier: the blessing of dimensionality for learning large Gaussian mixtures. COLT, 2014.
- A. Bhaskara, M. Charikar, A. Moitra, and A. Vijayaraghavan. Smoothed analysis of tensor decompositions. STOC, 2014.
- A. Chaganty and P. Liang. Estimating latent-variable graphical models using moments and likelihoods. ICML, 2014.
- A. Chaganty and P. Liang. Spectral experts for estimating mixtures of linear regressions. ICML, 2013.
- J.T. Chang. Full reconstruction of Markov models on evolutionary trees: Identifiability and consistency. Mathematical Biosciences, 1996.
- D. Hsu, S.M. Kakade, and P. Liang. Identifiability and unmixing of latent parse trees. NIPS, 2012.
- D. Hsu and S.M. Kakade. Learning mixtures of spherical Gaussians: moment methods and spectral decompositions. ITCS, 2013.
- E. Mossel and S. Roch. Learning nonsingular phylogenies and hidden Markov models. Annals of Applied Probability, 2006.
- J. Zou, D. Hsu, D. Parkes, and R. Adams. Contrastive learning using spectral methods. NIPS, 2013.
Method-of-moments via other algorithms
- S. Arora, R. Ge, Y. Halpern, D. Mimno, A. Moitra, D. Sontag, Y. Wu, M. Zhu. A practical algorithm for topic modeling with provable guarantees. ICML, 2013.
- S. Arora, R. Ge, and A. Moitra. Learning topic models---going beyond SVD. FOCS, 2012.
- M. Belkin and K. Sinha. Polynomial learning of distribution families. FOCS, 2010.
- Y. Halpern and D. Sontag. Unsupervised learning of noisy-or Bayesian networks. UAI, 2013.
- M. Hardt and E. Price. Sharp bounds for learning a mixture of two Gaussians. 2014.
- A.T. Kalai, A. Moitra, and G. Valiant. Efficiently learning mixtures of two Gaussians. STOC, 2010.
- A. Moitra, and G. Valiant. Settling the polynomial learnability of mixtures of Gaussians. FOCS, 2010.
- K. Pearson. Contributions to the mathematical theory of evolution. Philosophical Transactions of the Royal Society, London, A., 1894.
- K. Stratos, D. Kim, M. Collins, and D. Hsu. A spectral algorithm for learning class-based n-gram models of natural language. UAI, 2014.
Dynamical models, subspace identification, observable operator models, predictive state representations
- A. Anandkumar, K. Chaudhuri, D. Hsu, S.M. Kakade, L. Song and T. Zhang, Spectral methods for learning multivariate latent tree structure. NIPS, 2011.
- R. Bailly. Quadratic weighted automata: spectral algorithm and likelihood maximization. ACML, 2011.
- R. Bailly, F. Denis, L. Ralaivola. Grammatical inference as a principal component analysis problem. ICML, 2009.
- R. Bailly, A. Habrard, F. Denis. A spectral approach for probabilistic grammatical inference on trees. ALT, 2010.
- B. Balle, X. Carreras, F. M. Luque, and A. Quattoni. Spectral learning of weighted automata: a forward-backward perspective. Machine Learning, 2013.
- B. Balle and M. Mohri. Spectral learning of general weighted automata via constrained matrix completion. NIPS, 2012.
- B. Balle, A. Quattoni, and X. Carreras. A spectral learning algorithm for finite state transducers. ECML, 2011.
- B. Balle, A. Quattoni, and X. Carreras. Local loss optimization in operator models: A new insight into spectral learning. ICML, 2012.
- B. Boots and G. Gordon. An online spectral learning algorithm for partially observable nonlinear dynamical systems. AAAI, 2011.
- B. Boots and G. Gordon. Predictive state temporal difference learning. NIPS, 2010.
- B. Boots and G. Gordon. Two-manifold problems with applications to nonlinear system identification. ICML, 2012.
- B. Boots, S. Siddiqi and G. Gordon. Closing the learning-planning loop with predictive state representations. Robotics: Science and Systems, 2010.
- S.B. Cohen, K. Stratos, M. Collins, D.P. Foster, and L. Ungar. Spectral learning of latent variable PCFGs. ACL, 2012.
- F. Denis, Y. Esposito, and A. Habrard. Learning rational stochastic languages. COLT, 2006
- P. Dhillon, J. Rodu, M. Collins, D. Foster, and L. Ungar. Spectral dependency parsing with latent variables. EMNLP/CoNLL, 2012.
- D. Hsu, S.M. Kakade, and T. Zhang. A spectral algorithm for learning hidden Markov models. Journal of Computer and System Sciences, 2012.
- H. Jaeger. Observable operator models for discrete stochastic time series. Neural Computation, 2000.
- M. Littman, R. Sutton, and S. Singh. Predictive representations of state. NIPS, 2001.
- F.M. Luque, A. Quattoni, B. Balle, and X. Carreras. Spectral learning in non-deterministic dependency parsing. EACL, 2012.
- P.V. Overschee and B. De Moor. Subspace identification of linear systems. 1996.
- A.P. Parikh, L. Song and E. Xing, A spectral algorithm for latent tree graphical models. ICML, 2011.
- A.P. Parikh, L. Song, M. Ishteva, G. Teodoru, and E.P. Xing. A spectral algorithm for latent junction trees. UAI, 2012.
- S. Siddiqi, B. Boots and G. Gordon, A constraint generation approach to learning stable linear dynamical systems. NIPS, 2007.
- S. Siddiqi, B. Boots and G. Gordon, Reduced rank hidden Markov models. AISTATS, 2010.
- L. Song, B. Boots, S. Siddiqi, G. Gordon and A. Smola, Hilbert space embedding of hidden Markov model. ICML, 2010.
Independent component analysis
- S. Arora, R. Ge, A. Moitra, and S. Sachdeva. Provable ICA with unknown Gaussian noise, and implications for Gaussian mixtures and autoencoders. NIPS, 2012.
- J.-F. Cardoso. Super-symmetric decomposition of the fourth-order cumulant tensor: blind identification of more sources than sensors. ICASSP, 1991.
- J.-F. Cardoso and Pierre Comon. Independent component analysis, a survey of some algebraic methods. IEEE International Symposium on Circuits and Systems, 1996.
- P. Comon. Independent component analysis, a new concept? Signal Processing, 1994.
- P. Comon. and C. Jutten. Handbook of blind source separation: independent component analysis and applications. 2010.
- A.T. Erdogan. On the convergence of ICA algorithms with symmetric orthogonalization. IEEE Transactions on Signal Processing, 2009.
- N. Delfosse and P. Loubaton. Adaptive blind separation of independent sources: a deflation approach. Signal processing, 1995.
- A.M. Frieze, M. Jerrum, and R. Kannan. Learning linear transformations. FOCS, 1996.
- A. Hyvarinen. Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks, 1999.
- A. Hyvarinen and E. Oja. Independent component analysis: algorithms and applications. Neural Networks, 2000.
- P. Q. Nguyen and O. Regev. Learning a parallelepiped: Cryptanalysis of GGH and NTRU signatures. Journal of Cryptology, 2009.