[SMT-COMP] benchmark data

David Cok dcok at grammatech.com
Mon Jun 9 10:27:19 EDT 2014


SMTCOMP competitors:

The current data on benchmarks for SMT-COMP 2014 has been posted to
http://smtcomp.sourceforge.net/2014/difficulties.shtml
Note the caveats about this data - we are still correcting errors in the 
benchmarks, running trial runs to determine difficulty ratings, and 
trial runs to determine sat/unsat status of (some of the) currently 
unknown benchmarks. When benchmarks are updated the benchmark ids change 
- to be using current benchmarks, you need to recopy benchmarks into 
your trial spaces.

Thanks for email pointing out mal-formed benchmarks. Those are being 
corrected, primarily by Clark Barrett and Morgan Deters. Let us know of 
others you find.

Our initial capacity estimates indicate that for most of the divisions, 
we will be able to run all of the eligible benchmarks (those that are 
non-trivial and have a known status).
(With the reorganization of benchmarks, there are now many divisions 
with <1000 benchmarks).

- David Cok, for the organizers




More information about the SMT-COMP mailing list