Graham Taylor

Factored Conditional Restricted Boltzmann Machines

In this section we present a new model, based on the CRBM, that explicitly captures contextual information. Our model preserves the CRBM's most important computational properties and includes multiplicative three-way interactions that allow the effective interaction weight between two units to be modulated by the dynamic state of a third unit. We factor the three-way weight tensor implied by the multiplicative model, reducing the number of parameters from O(N3) to O(N2). Again we demonstrate our model's effectiveness by modeling human motion. The three-way interactions greatly improve the model's ability to blend motion styles or to transition smoothly among them.

Return to index

4.6.1 Modeling with discrete style labels

We use a factored CRBM with three-way, multiplicative interactions to synthesize transitions and blends between two styles. Note that the training data does not contain transitions nor blends.

The player will show in this paragraph unless you do not have flash player installed

4.6.2 Modeling with real-valued style parameters

Here the architecture is identical to that in Section 4.6.1, except that instead of gating interactions by real-valued features connected to discrete labels, we gate directly by real-valued style variables. In our experiment, we use two style variables: stride length and speed. This video shows real-time online generation at 30fps (the video itself is only 10fps). We can manipulate the stride length and speed variables during generation, which changes the effective weights in the model. The hidden units and motion respond to the changing of these variables, producing smooth interpolation and extrapolation beyond the 9 discrete settings of the variables with which the model was trained.

The player will show in this paragraph unless you do not have flash player installed