Notes for September 30 class:

Matrices and ray tracing to transformed surfaces

CURSOR

For your fun and convenience, with this week's package we've included information about the mouse cursor. You can use this to create work which responds interactively to the user's mouse.

You can access it from your fragment shader, via uniform vec3 variable `"uCursor"`:

```   uniform vec3 uCursor;

x: -1.0 ≤ uCursor.x ≤ 1.0  // left to right

y: -1.0 ≤ uCursor.y ≤ 1.0  // bottom to top

z: uCursor.z == 0.0 if mouse is up,
1.0 if mouse is down
```

USING MATRICES TO TRANSFORM POINTS

Our major use of 4x4 matrices will be to perform linear transformations. These are any geometric transformations that always keep straight lines straight.

Internally, the 16 numerical values in a matrix can be stored in either column major order, as shown below, or row major order. For this course, you will be storing matrices in column major order, because that is consistent with how matrices are stored in shader programs on the GPU.

The basic operation is multiplying a column vector on the left by a 4x4 matrix, thereby transforming that column vector. The operation itself is essentially four dot products, one for each row of the matrix:

```

1x4       4x4     1x4

x'
y'
z'
w'

⟵

M0
M1
M2
M3

M4
M5
M6
M7

M8
M9
M10
M11

M12
M13
M14
M15

×

xyzw

```

MATRIX PRIMITIVES

There are a few primitive matrix generating functions. The matrices returned by these functions can be used to generate all other matrices. Each of these matrices represents a simple geometric intuition:

```

identity()
translate(x,y,z)
rotateX(θ)
rotateY(θ)
rotateZ(θ)
scale(x,y,z)
perspective(x,y,z,w)

has no effect on point
moves point by a fixed amount
scales point along X,Y,Z axes
create perspective effects

```
The above matrices are internally structured as follows.

In the chart below, "c" represents cos(θ) and "s" represents sin(θ), where θ is angle of rotation.

```

identity()
translate(x,y,z)
rotateX(θ)
rotateY(θ)
rotateZ(θ)
scale(x,y,z)
perspective(x,y,z,w)

1 0 0 00 1 0 00 0 1 00 0 0 1
1 0 0 x0 1 0 y0 0 1 z0 0 0 1
1 0  0 00 c -s 00 s  c 00 0  0 1
c 0 s 0 0 1 0 0-s 0 c 0 0 0 0 1
c -s 0 0s  c 0 00  0 1 00  0 0 1
x 0 0 00 y 0 00 0 z 00 0 0 1
1 0 0 00 1 0 00 0 1 0x y z w

```
Because you are storing matrices in column major order, the array of 16 numerical values of, for example, the output of translate(x,y,z) will be in the following order:
```   [ 1 0 0 0  0 1 0 0  0 0 1 0  x y z 1 ]
```

MATRIX OPERATIONS

There are some basic operations for matrices:

```

multiply(M,M)

M   ⟵ M x M

Multiply two matrices together

multiply(M,V)

V   ⟵ M x V

Use a matrix to transform a point

transpose(M)

MT  ⟵ M

Swap the rows and columns of a matrix

inverse(M)

M-1 ⟵ M

Find the matrix M-1 such that M-1M is Identity

```
Matrix multiplication essentially consists of sixteen dot products. To produce any given row and column of A×B, we take the dot product of that row of A and that column of B:

```

A×B

A

B

# # # ## # # ## # # ## # # #

⟵

# # # ## # # ## # # ## # # #

×

# # # #
# # # #
# # # #
# # # #

```

In the course notes I will provide you with an implementation of `inverse()`.

BUILDING MATRIX HIERARCHIES

Not this week, but soon, you will be using 4x4 matrices to build hierarchical movement models.

Any hierarchical structure with moving parts, such as an electric motor or a human body, is essentially a tree structure. Traversing such a structure therefore requires some sort of internal data stack.

In our case, each element of that stack is a 4x4 matrix:

```   save()        stack[top+1] = copy(stack[top]);
++top;

restore()     --top;
```

SENDING MATRIX DATA FROM THE CPU TO THE GPU

Eventually you will be using matrices on the CPU to model geometry that you create there. But for now, you will just be using them in our ray tracer on the GPU.

In order to do this, you need to be able to send the 16 numerical values of a matrix from the CPU down to the GPU. You can do this as follows:

```   // In the shader program on the GPU:

uniform mat4 uMyMat;

// In the Javascript program on the CPU -- only once:

state.uMyMatLoc = gl.getUniformLocation(program, 'uMyMat');

// In the Javascript program on the CPU -- at every animation frame:

gl.uniformMatrix4fv(state.uMyMatLoc, false, [ 16 numeric values ]);
```

ACCESSING A MATRIX ON THE GPU

You can directly specify the 16 values of a matrix variable in a shader program as follows:

```   mat4 M = mat4(1.,0.,0.,0., 0.,1.,0.,0., 0.,0.,1.,0., 2.,3.,4.,1.);
```
When you do the above, you get the following column major matrix:
```          1 0 0 2
0 1 0 3
0 0 1 4
0 0 0 1
```
You can then access a single column of this matrix as follows:
```   M // == vec4(2.,3.,4.,1.)
```
You can access a single element of this matrix in two different ways:
```   M.y  // == 3.
M // == 3.  (this line and the previous line are equivalent)
```
Accessing a row of this matrix is a little more awkward:
```   vec4(M.y,M.y,M.y,M.y)  // == vec4(0.,1.,0.,3.)

```

RAY TRACING: TRANSFORMING A PLANE SURFACE

To use a transformation matrix for ray tracing, you will really want to use its corresponding inverse matrix.

I suggest that in your definition of `struct Shape` in your shader program, you add fields for `matrix` and its associated inverse `imatrix`:

```   struct Shape {
...
mat4 matrix;
mat4 imatrix; // this is just the inverse of the above
};
```
The associated code in your Javascript will be something like this:
```   gl.uniformMatrix4fv(state.uShapesLoc.matrix , false, someArray);
gl.uniformMatrix4fv(state.uShapesLoc.imatrix, false, inverse(someArray));
```

To use a 4x4 matrix to transform a ray traced polyhedron, in your fragment shader you can simply transform each of the planes that define one of its component halfspaces. For example, to ray trace to a transformed cube, you would transform each of its 6 defining planes as part of your fragment shader computation.

Because the goal is to preserve the relationship between any halfspace S and any point V:

```   S•V ≤ 0
```
we need to transform `S → S'` so that
```   S'•(M•V) ≤ 0
```
produces the same result.

It is easy to see that this will be the case when:

```   S' = S•M-1
```
because:
```   (S•M-1)•(M•V) == S•(M-1•M)•V == S•V
```
So if you have a function that computes ray/halfspace intersection in your fragment shader program, you can just insert the following assignment right at the beginning of the function:
```   S = S * uShapes[i].imatrix;
```

RAY TRACING: TRANSFORMING A QUADRATIC SURFACE

In the following section, we show how to do the same kind of transformation when ray tracing in your fragment shader to second order surfaces, such as spheres.

In the data you pass down to your fragment shader, you can represent any general quadratic surface (eg: sphere, cylindrical tube, cone, parabaloid, etc) as a 4x4 matrix, where you actually use only 10 of the 16 available matrix slots:

```

S  =
a 0 0 0b e 0 0c f h 0d g i j

```
The above matrix could be created in a GLSL shader by, for example, the following assignment:
```   mat4 S = mat4(a,b,c,d, 0.,e,f,g, 0.,0.,h,i, 0.,0.,0.,j);
```
A key observation, which I learned from the great computer graphics pioneer Jim Blinn, is that the above matrix can be used to represent a general quadratic equation in three variables by expanding the following inequality:
```   VT S V ≤ 0
```
Where:
```

V   =
xyz1

```
If you actually multiply this out, you get the general quadratic polynomial in three variables:
```

x y z 1

a 0 0 0b e 0 0c f h 0d g i j

xyz1
⟶
ax2 + bxy + cxz + dx + ey2 + fyz + gy + hz2 + iz + j

```
Suppose you want to transform `V to V'= M•V`.

This means you now need to find a different `S'` such that:

```   (MV)T S' MV == VT S V
```
Note also, by the way, that when you take the transpose, you need to reverse the order of the matrix/vector multiplication:
```   (M V)T == VT MT
```
Following the same approach you used for half spaces, you will want to modify S as follows in your fragment shader:
```   VT MT M-1T S M-1 M V == VT S V
```
```   S ⇐ transpose(M-1) * S * M-1;
```
After doing that, you need to rearrange the values in S to ensure all of the non-zero values end up in just those 10 slots:
```

S
⇐
a b c d
0 e f g
0 0 h i
0 0 0 j
⇐

S0,0
S0,1+S1,0
S0,2+S2,0
S0,3

0
S1,1
S1,2+S2,1
S1,3+S3,1

0
0
S2,2
S2,3+S3,2

0
0
0
S3,3

```
You can complete the code below to implement this in your fragment shader:
```   S = mat4( S.x, S.y+S.x, S.z+S.x, S.w+S.x,
0.,     S.y       , S.z+S.y, S.w+S.y,
...
);
```
You are really trying to intersect S with a ray V+tW, and your goal is to find the value of t where the resulting point is on the surface of S.

Expanding out the expression (V+tW)T S (V+tW):

```

[vx + t wx , vy + t wy , vz + t wz, 1]

a 0 0 0b e 0 0c f h 0d g i j

vx + t wxvy + t wyvz + t wz1

```
When you multiply out the above, you get the below fairly large expression in variable t.

(Note that the six numbers ``` vx, vy, vz, wx, wy, wz``` are all constants):

```   a (vx + t wx) (vx + t wx) +
b (vx + t wx) (vy + t wy) +
c (vx + t wx) (vz + t wz) +
d (vx + t wx) +

e (vy + t wy) (vy + t wy) +
f (vy + t wy) (vz + t wz) +
g (vy + t wy) +

h (vz + t wz) (vz + t wz) +
i (vz + t wz) +

j
```
Your goal is to end up with a quadratic equation in t that you can then solve:
```   A t2 + B t + C ≤ 0
```
You can multiply out the large expression above and then regroup all of the terms to get the three coefficients of the quadratic equation:
```   A = a wx wx + b wx wy + c wx wz +
e wy wy + f wy wz +
h wz wz

B = a (vx wx + vx wx) + b (vx wy + vy wx) + c (vx wz + vz wx) + d wx +
e (vy wy + vy wy) + f (vy wz + vz wy) + g wy +
h (vz wz + vz wz) + i wz

C = a vx vx + b vx vy + c vx vz + d vx +
e vy vy + f vy vz + g vy +
h vz vz + i vz +
j
```
Now that you have A, B and C, you can just solve quadratic equation for t, to find where the ray intersects the surface.

To sum up the above steps -- all of which should be done in your fragment shader:

1. Transform S → (M-1)T S (M-1)
2. Gather the 16 values of the resulting matrix into 10 coefficients
3. Rearrange terms to find A,B and C, where At2 + Bt + C ≤ 0
4. Solve for t using the quadratic equation

FINDING THE SURFACE NORMAL

You can now easily find the surface normal, because it's just the gradient of the polynomial.

That means you can just take the partial derivatives in x,y,z and normalize the result:

```   (x,y,z) = V + t W

normal = normalize( [ 2ax  +  by  +  cz  +  d,
2ey  +  fz  +  g,
2hz  +  i ] )
```
The next few sections show some convenient coefficient values for the original untransformed S, to make some standard useful shapes.

SPHERE COEFFICIENTS

```   xx + yy + zz ≤ 1  ⟶   [ 1,0,0,0, 0,1,0,0, 0,0,1,0, 0,0,0,-1 ]
```

CYLINDRICAL TUBE COEFFICIENTS

```   xx + yy ≤ 1  ⟶   [ 1,0,0,0, 0,1,0,0, 0,0,0,0, 0,0,0,-1 ]
```

INTERSECT YOUR CYLINDRICAL TUBE WITH TWO HALFSPACES TO MAKE A UNIT CYLINDER

```   z ≥ -1  ⟶  z + 1 ≥ 0  ⟶  -z - 1 ≤ 0  ⟶  [ 0, 0,-1,-1 ]
z ≤  1                 ⟶   z - 1 ≤ 0  ⟶  [ 0, 0, 1,-1 ]
```

COOL VIDEO WE WATCHED

At the end of class we watched World Builder.

HOMEWORK

For your homework, which is due by class time on Monday October 7, do the following:

• Implement the matrix primitives in Javascript.

• Implement matrix multiplication in Javascript.

• Create some examples of shape transformation and animation by multiplying together some time-varying matrix primitives. This should be implemented in your Javascript code.

• Using the Javascript implementation of inverse() that I provide in this week's software package, pass down the matrix inverse to your fragment shader on the GPU as the value of a field in your Shape struct.

An updated library package that you can use, which also contains a working `uCursor` variable for your shader program, is in hw4.zip.
 ``` let setMatrix = (loc, mat) => { gl.uniformMatrix4fv(loc['matrix' ], false, mat); gl.uniformMatrix4fv(loc['imatrix'], false, inverse(mat)); } let M0 = multiply(translate( .4,-.4,-.4), multiply(rotateX(-.5), scale(.3,.2,.1))); let M1 = multiply(translate(-.4, .4,-.4), multiply(rotateY(2 ), scale(.2,.3,.2))); let M2 = multiply(translate(-.4,-.4, .4), multiply(rotateZ(1 ), scale(.3,.2,.1))); let M3 = multiply(translate( .4, .4,-.4), multiply(multiply(rotateY(1), rotateZ(1)), scale(.3,.1,.2))); gl.uniform1i(state.uShapesLoc.type, CYLINDER); gl.uniform1i(state.uShapesLoc.type, SPHERE); gl.uniform1i(state.uShapesLoc.type, OCTAHEDRON); gl.uniform1i(state.uShapesLoc.type, CUBE); setMatrix(state.uShapesLoc, M0); setMatrix(state.uShapesLoc, M1); setMatrix(state.uShapesLoc, M2); setMatrix(state.uShapesLoc, M3); ``` 