Computer Science Colloquium

A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data

Tong Zhang
IBM TJ Watson Research Center

Wednesday, March 7, 2005 11:15 A.M.
Room 1302 Warren Weaver Hall
251 Mercer Street
New York, NY 10012-1185

Colloquium Information:


Richard Cole, (212) 998-3119


Semisuperised learning, i.e. statistical machine learning methods that use both labeled and unlabeled data, have been actively investigated in recent years. However, at the current stage, we still don't have a complete understanding of their effectiveness. This talk presents a closely related problem, which leads to novel and robust algorithms for semi-supervised learning. Specifically we consider the problem of learning "what good classifiers are like" (i.e. predictive structures) from multiple learning tasks. We call this problem structural learning.

In the Bayesian framework, structural learning can be viewed as hierarchical modeling, and its simplest form covers the well-known Stein's effect. In this talk, I will give a more flexible and realistic formulation of the problem under the standard machine learning framework, in which rigorous theoretical analysis can be carried out. I will then present a specific learning formulation which can be analyzed in our framework. The proposed formulation can be solved by an iterative SVD procedure.

Finally I will illustrate how to apply this algorithm to semi-supervised learning. The idea is to discover the common predictive structure shared by many automatically generated auxiliary classification problems using unlabeled data, and then employ this structure to improve performance on target problems. Experiments will be presented to demonstrate the effectiveness of the proposed method in the setting of semi-supervised learning.

Joint work with Rie Ando.

top | contact