[SMT-COMP] SMT-COMP 2018: Main Track Test Job

[Andres Nötzli]

Hi Tjark,

Thanks for the test job! May we assume that the final version of the post-processor will ignore all output on stderr? The rules don’t seem to explicitly say anything about stderr.

Thanks,
Andres

> On Jun 5, 2018, at 1:16 PM, Tjark Weber <tjark.weber at it.uu.se> wrote:
> 
> Dear SMT-COMP participants,
> 
> I have run a small test job on StarExec to exercise all main track
> solvers on a small number of benchmarks from each division. The results
> are available at
> 
> https://www.starexec.org/starexec/secure/details/job.jsp?id=28495
> 
> I would suggest that you investigate results flagged as wrong. Please
> keep in mind that there are likely false positives:
> 
> i) the StarExec UI flags any result as wrong if the benchmark's status
>   is unknown--for the competition, we will of course use our own
>   analysis instead; and
> 
> ii) the competition will use a more liberal post-processor that
>    attempts to ignore malformed solver output.
> 
> Nevertheless, you might still want to ensure that your solver is
> compliant with SMT-LIB (or notify us if there are ill-formed
> benchmarks).
> 
> Please let us know if your solver was tested in divisions where it
> should not be competing, and vice versa.
> 
> If you upload a new solver version to StarExec before the final solver
> deadline (June 10), please also notify us by email to make sure we are
> aware of the new version and will use it for the competition.
> 
> Best,
> Tjark
> 
> 
> 
> 
> 
> 
> När du har kontakt med oss på Uppsala universitet med e-post så innebär det att vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du läsa här: https://mp.uu.se/web/info/stod/dataskyddsforordningen.
> 
> E-mailing Uppsala University means that we will process your personal data. For more information on how this is performed, please read here: https://mp.uu.se/web/info/stod/dataskyddsforordningen
> _______________________________________________
> SMT-COMP mailing list
> SMT-COMP at cs.nyu.edu
> https://cs.nyu.edu/mailman/listinfo/smt-comp