Updated Winograd Schema Challenge Contest Rules

Registration: Contestants should email Charles Ortiz charles.ortiz@nuance.com stating their intent to enter the contest no later than July 1, 2016. The contest itself will be held at IJCAI 2016 in New York City, July 12, 2016.
Input format: Contestant programs will receive their input in the form of an .xml file. An example file may be found at http://www.cs.nyu.edu/faculty/davise/papers/WSCExample.xml. The structure of the .xml should be self-explanatory on inspection of this file.
Problems: All problems have the following form. There is a text of a single sentence or a few sentences that contains one or more pronouns with ambiguous referents. Each problem asks about one such pronoun. The pronoun that is the subject of the problem is demarcated in the XML input with the tag < pron > < /pron >. (In viewing the XML file in a web browser, the pronoun appears in boldface.) After the text, there is a short excerpt from the text containing the pronoun that is the subject of the problem and a few words that occur on one side or the other; this is for the benefit human viewers. Finally, a list of possible referents is given, labelled "A", "B", "C" ... For example:
Babar wonders how he can get new clothing. Luckily, a very rich old man who has always been fond of little elephants understands right away that he is longing for a fine suit. As he likes to make people happy, he gives him his wallet.
he is longing for a fine suit
1. Babar
2. old man
There may be multiple problems that use the same text but ask about different pronouns, as with problems 2-5 in the example file.
Number of rounds and questions: There will be one or two rounds in the contest. Each round will consist of at least 60 questions.
The two rounds differ in the source of the texts. In the first round, the texts are "Pronoun Disambiguation Problems"; that is, they are drawn from actual texts, possibly with some editing. In the second round, each text is one half of a Winograd schema. A detailed discussion and justification is given in (Morgenstern, Davis, and Ortiz, 2016).
Only contestants who achieve at least 90% in the first round will be allowed to compete in the second round. If no contestants qualify, then the second round will not be given.
Competing: Each contestant must be represented by an individual who is present. The representative must bring a laptop on which the entry will run. Any commercially sold portable computer is acceptable.
Internet access: Limited access to the Internet will be permitted, under certain circumstances. Please contact the organizers for details.
Output format: The output of your program should be a plain text file named TeamName-output.txt. The format to be followed is illustrated in the file http://www.cs.nyu.edu/faculty/davise/papers/WSOutput.txt.
For each problem, there will be four lines in the output, separated by line breaks:
Line 1: Problem number, and echo of text of problem.
Line 2: Echo of the excerpt for the problem.
Line 3: "Answer" problemNumber.answerNumber answer
Line 4: Blank, as a separator between problems
For example, if the problem above is problem 2 in the input, then the corresponding four lines of the output file would be as follows:
```
Babar wonders how he can get new clothing. Luckily, a very rich old man who has always been fond of little elephants understands right away that he is longing for a fine suit.  As he likes to make people happy, he gives him his wallet. 
he is longing for a fine suit
Answer 2.A Babar
```
At the end of the file, there should be a comma-separated list of all the answers in order. E.g.
```
A, A, B, B, A, A, B
```
The submission will be graded on the final list of answers. The remaining material is there for human inspection.
No problems should be omitted. Any problem that is omitted will be marked as wrong, so it always pays to guess.
Publication: In the three weeks following the competition, researchers with winning or potentially winning entries will be expected to submit to WSC organizers a paper explaining the algorithms, knowledge sources, and knowledge structures used. These papers will be posted on the commonsensereasoning.org website. Publication on the commonsensereasoning.org website does not preclude any other publication. Entries not submitting such a paper will be disqualified.
If, in the judgement of the contest committee, the description of the program in this paper is entirely inadequate or implausible as an explanation of the success of the program, then the team involved will be asked to demonstrate in detail that the behavior of the program is in fact that described in the paper.
The aim of this contest is to advance science; all results obtained must be reproducible, and communicable to the public. As such, any winning entry is encouraged to furnish to the organizers of the Winograd Schema Challenge Competition its source code and executable code, and to use open source databases or knowledge bases or make its databases and knowledge structures available for independent verification of results.
Prizes: The grand prize of $25,000 will be awarded to the first team to achieve a score of 90% in both rounds of the contest. If more than one team accomplishes this, the prize will be awarded to the team with the higher score in the second round. If tied in the second round, the prize will go to the team with the higher score in the first round. If both rounds are ties, the prize will be split.
At IJCAI-2016 three smaller prizes, of $1000, $750, and $500 will be awarded to the top three programs that score over 65% on the first round of the contest.
Contest commitee: The contest has been designed and will be administered by a committee consisting of: Leora Morgenstern leora.morgenstern@leidos.com; Ernest Davis davise@cs.nyu.edu; and Charles Ortiz charles.ortiz@nuance.com. The committee's decisions on all matters relating to the contest is final. Any questions should be addressed to them. We gratefully acknowledge the assistance of Hector Levesque and of Gary Marcus.