Volume Visualization at NYU

Visualization is the study of data presentation into visual form. Its ultimate goal is to convey the information effectively to help the people understand the data, although not necessarily in terms of pretty pictures. The visualization methods should be based on an understanding of human perception and of the particular tasks to be accomplished using the visualization. Visualization is inevitably a psycho-physiological phenomenon. One striking fact is its use of foveated images in which the resolution is higher at the fovea than at the periphery. This characteristic is currently exploited in the Active Visualization Project at NYU, directed by Prof. Chee Yap. Click here for some foveated image demos. We believe that this foveation idea can also provide a superior model for volume visualization, as it did in serving large images over a thinwire.

About Volume Visualization

Visualization systems are ultimately based on computer graphics technology. Volume graphics is a powerful technique to support the visualization of inner structures, amorphous phenomena, and intermixing of models with sampled and/or computed datasets. Unlike the conventional surface-based computer graphics, the volume graphics is concerned with 3D objects represented as volumes, and can show the characteristics of the interior of a solid 3D object in a 2D image (view plane). The complete volume visualization pipeline includes the data representation, modeling, manipulation and rendering stages.

Typically, the volumetric dataset is represented as a 3D discrete regular grid of volume elements (called voxels), stored in a discrete regular volume buffer V(x,y,z). A collection of all the values (attributes) associated with each point in a volume is called a scalar field on the volume. A continuous function f(x,y,z) can be defined on R^3 by applying interpolation over the discrete volume V(x,y,z).

Volume Rendering is the process to display the scalar fields. These fields reflect and can be mapped to the RGB and opacity values. Various volume visualization methods have been developed. Generally there are two major categories: (direct) volume rendering, and level surface (isosurface) rendering. The direct volume rendering methods can be further divided into two sub-categories: image-space based (like ray tracing) and object-space based (like splatting).

Volume Graphics vs. Surface Graphics:
Features	Volume Graphics	Surface Graphics
Input data	sampled data of real object (e.g. MRI and CT data), computed data from simulation or a geometric model	geometric primitives
Data representation	3D discrete regular volume buffer, insensitive to both scene and object complexity	a list of geometric objects, sensitive to both scene and object complexity
Viewpoint dependence	independent, all viewpoint independent data could be pre-computed and stored.	dependent
Texture	insensitive to texture; texturing only needs to be performed once during voxelization.	needs to be repeated for each rendering, complexity proportional to object complexity.
Show internal info	capable	not capable
Hardware requirements	high (esp. in memory size, CPU speed and I/O structures and devices)	relatively low
Discrete model	Yes, difficult to manipulate and transform, sometimes cause aliasing artifact. No geometric information.	No.

Our Particular Interest

We are especially interested in exploiting foveation in the volume visualization process, with the goal of achieving near real-time performance for handling 3D static datasets on the ordinary workstations, and also, over the Internet. There are several techniques to investigate.

Foveation and Multi-Resolution in Volume Rendering:
We need utilize the benefits of foveation throughout the visualization pipeline, from representation to rendering. We also need a multiple resolution model for fast visualization under suitable error bound.
Hierarchical Data Representation:
Wavelet, octree, mip-map techniques are effective approaches for multiple resolutions in the active visualization. It greatly reduces the amount of storage required in the rendering. But how to establish a volume model over these representations is still a problem. For example, interpolation efficiently is a problem to be addressed. Moreover, the manipulation and rendering times can be further reduced by developing corresponding techniques directly on these domains (such as wavelet).
Progressive Refinement:
Given the high cost of executing the direct volume rendering, it is make sense to approximate the rendering result incrementally, i.e. we produce a useful, although perhaps inaccurate, image early on, which can be successively refined to greater accuracy as more time (or other constraint resources) allocated. This task could, to some extent, work under the user guidance and control.
Fast Volume Rendering techniques, especially on the hierarchy representation domain (like wavelet).

An Example Application - Medical Imaging

Magnetic resonance imaging (MRI) is an imaging technique used primarily in medical settings to produce high quality images of the inside of the human body. MRI started out as a tomographic imaging technique, that is, it produced an image of the NMR signal in a thin slice through the human body. MRI has advanced beyond tomographic imaging to volume imaging.

The MRI dataset consists of a stack of continuous 3D slices. We can see them as samples of a function defined on the continuous R^3 spatial domain. Volume visualization is the appropriate way to process these sampled datasets. But the first step is to re-sample it while preserving information and voxelize it. Here we need to notice that the original MRI dataset may be not perfect, and it could already produce the aliasing artifacts because of undersampling. Then we need to take into consideration the assignment of a color map which would be important for the final visual effects.

Useful Links:

Visualization and Volume Graphics:

Applications and Datesets:

Virtual Reality:

Programming Languages and Tools: