[SMT-COMP] SMT-COMP 2016: Incorrect Answers (Unsat-Core Track)

Tjark Weber tjark.weber at it.uu.se
Wed Jul 6 13:56:22 EDT 2016


In the unsat-core track, 17295 answers were classified as incorrect:
17097 sat answers and 198 cores that were found to be satisfiable by at
least one (and usually several) of our validation solvers.

The following list shows how often each solver gave an incorrect answer
(again, no shaming intended):

    190 mathsat-5.3.11-linux-x86_64-Unsat-core
     26 veriT-dev
  17079 z3-4.4.1

A more detailed list that includes benchmark information is available at
http://smtcomp.sourceforge.net/2016/Unsat_Core_Track.incorrect.csv

If you find that an incorrect answer was caused by a bug in the
competition tools or by an incorrectly classified benchmark (rather
than by a bug in the solver), kindly inform us!

Best,
Tjark




More information about the SMT-COMP mailing list