[SMT-COMP] SMT-COMP 2016: Incorrect Answers (Unsat-Core Track)
Tjark Weber
tjark.weber at it.uu.se
Wed Jul 6 13:56:22 EDT 2016
In the unsat-core track, 17295 answers were classified as incorrect:
17097 sat answers and 198 cores that were found to be satisfiable by at
least one (and usually several) of our validation solvers.
The following list shows how often each solver gave an incorrect answer
(again, no shaming intended):
190 mathsat-5.3.11-linux-x86_64-Unsat-core
26 veriT-dev
17079 z3-4.4.1
A more detailed list that includes benchmark information is available at
http://smtcomp.sourceforge.net/2016/Unsat_Core_Track.incorrect.csv
If you find that an incorrect answer was caused by a bug in the
competition tools or by an incorrectly classified benchmark (rather
than by a bug in the solver), kindly inform us!
Best,
Tjark
More information about the SMT-COMP
mailing list