Deep Learning

• Time Period: September 2004 - present.
• Participants: Marc'Aurelio Ranzato, Koray Kavukcuoglu, Karol Gregor, Y-Lan Boureau, Yann LeCun (Courant Institute/CBLL).
• Sponsors: ONR, NSF.
• Description: Animals and humans can learn to see, perceive, act, and communicate with an efficiency that no Machine Learning method can approach. The brains of humans and animals are "deep", in the sense that each action is the result of a long chain of synaptic communications (many layers of processing). We are currently researching efficient learning algorithms for such "deep architectures". We are currently concentrating on unsupervised learning algorithms that can be used to produce deep hierarchies of features for visual recognition. We surmise that understanding deep learning will not only enable us to build more intelligent machines, but will also help us understand human intelligence and the mechanisms of human learning.

  More information can be obtained from Marc'Aurelio Ranzato's page, and Koray Kavukcuoglu's page.

Filters being learned using the Predictive Sparse Decomposition algorithm as described in the 2008 CBLL Tech Report CBLL-TR-2008-12-01: "Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition".

Publications

140.	Kevin Jarrett, Koray Kavukcuoglu, Marc'Aurelio Ranzato and Yann LeCun: What is the Best Multi-Stage Architecture for Object Recognition?, Proc. International Conference on Computer Vision (ICCV'09), IEEE, 2009, \cite{jarrett-iccv-09}.	303KB	DjVu
		769KB	PDF
		834KB	PS.GZ

8.	Koray Kavukcuoglu, Marc'Aurelio Ranzato and Yann LeCun: Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition, Tech Report CBLL-TR-2008-12-01, Computational and Biological Learning Lab, Courant Institute, NYU, 2008, \cite{koray-psd-08}.	99KB	DjVu
		343KB	PDF
		384KB	PS.GZ

123.	Yoshua Bengio and Yann LeCun: Scaling learning algorithms towards AI, in Bottou, L. and Chapelle, O. and DeCoste, D. and Weston, J. (Eds), Large-Scale Kernel Machines, MIT Press, 2007, \cite{bengio-lecun-07}.	427KB	DjVu
		470KB	PDF
		497KB	PS.GZ

137.	Koray Kavukcuoglu, Marc'Aurelio Ranzato, Rob Fergus and Yann LeCun: Learning Invariant Features through Topographic Filter Maps, Proc. International Conference on Computer Vision and Pattern Recognition (CVPR'09), IEEE, 2009, \cite{koray-cvpr-09}.	334KB	DjVu
		864KB	PDF
		1474KB	PS.GZ

135.	Marc'Aurelio Ranzato, Y-Lan Boureau and Yann LeCun: Sparse feature learning for deep belief networks, Advances in Neural Information Processing Systems (NIPS 2007), 2007, \cite{ranzato-nips-07}.	129KB	DjVu
		174KB	PDF
		212KB	PS.GZ

134.	Marc'Aurelio Ranzato and Yann LeCun: A Sparse and Locally Shift Invariant Feature Extractor Applied to Document Images, Proc. International Conference on Document Analysis and Recognition (ICDAR), 2007, \cite{ranzato-icdar-07}.	139KB	DjVu
		205KB	PDF
		228KB	PS.GZ

132.	Yann LeCun, Sumit Chopra, Marc'Aurelio Ranzato and Fu-Jie Huang: Energy-Based Models in Document Recognition and Computer Vision, Proc. International Conference on Document Analysis and Recognition (ICDAR), 2007, \cite{lecun-icdar-keynote-07}.	110KB	DjVu
		355KB	PDF
		551KB	PS.GZ

126.	Marc'Aurelio Ranzato, Fu-Jie Huang, Y-Lan Boureau and Yann LeCun: Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition, Proc. Computer Vision and Pattern Recognition Conference (CVPR'07), IEEE Press, 2007, \cite{ranzato-cvpr-07}.	186KB	DjVu
		330KB	PDF
		344KB	PS.GZ

124.	Marc'Aurelio Ranzato, Y-Lan Boureau, Sumit Chopra and Yann LeCun: A Unified Energy-Based Framework for Unsupervised Learning, Proc. Conference on AI and Statistics (AI-Stats), 2007, \cite{ranzato-unsup-07}.	136KB	DjVu
		241KB	PDF
		413KB	PS.GZ

121.	Ranzato Marc'Aurelio, Christopher Poultney, Sumit Chopra and Yann LeCun: Efficient Learning of Sparse Representations with an Energy-Based Model, in J. Platt et al. (Eds), Advances in Neural Information Processing Systems (NIPS 2006), MIT Press, 2006, \cite{ranzato-06}.	152KB	DjVu
		191KB	PDF
		204KB	PS.GZ