FOM: validation of simulations

[Joe Shipman]

Stevenson:

>>I believe the foundational issues are related to the problem of
validating models to reality and that in turn is related to reasoning
when simulations are part of the problem. Suppose a physicist gives
you an analytical statement of a system to solve. These are usually
very complex etc etc. During the computation, you regularly violate
something inviolable in the model. Conservation of energy is a common
problem due to cancellation. You now finish and the physicist now
starts drawing conclusions from the output of the simulation.

Are these viable conclusions?

This is the problem facing the simulation community right now and it's
called "validation of simulations" rather than "verification of
simulations".<<

Obviously the problem is that you are using floating-point arithmetic
rather than some sort of exact or interval arithmetic.  Plenty of work
has been done on this, but I guess the interval arithmetic isn't
implemented in a fast enough way to satisfy your physicist friend.  Why
shouldn't this be the place to focus your research as a numerical
analyst?  We all know that physicists and numerical analysts are just
pretending that floating-point arithmetic has stability properties it
does not in fact have; so why aren't computers and compilers based on
interval arithmetic used more widely?

The other alternative is to make some assumptions about the randomness
of the rounding error and prove some theorems about the probable
reliability of the final result, dealing with the "cancellation" problem
algebraically to the extent possible (I am sure you already know how to
avoid ill-conditioned matrices and the like; if you know cancellation
must be occurring you can rearrange your calculation to minimize the
effect).

There are additional issues related to reversibility here - can the
simulation be organized in such a way that running it backwards is
possible and returns to the original state?  Theoretically we know this
can be done, but what is the situation in practice?

-- Joe Shipman