Answers: The astronomers/the galaxies.
Comment:
Problem: "Galaxies are spread all over the universe" may be Googlable.
Though strangers, Joe and Sue ended up sharing a table at the
restaurant. It was [really busy/the only one left/unavoidable/meant to
be]
What was [really busy/the only one left/unavoidable/meant to be]?
Answers:
The restaurant/the table/sharing the table/Joe and Sue
meeting each other.
Comment: Due to David Bender. This example is charming,
but unfortunately almost all the wrong assignments can be
excluded by selectional restrictions; e.g. the table cannot be "meant to
be"; Joe and Sue sharing a table cannot be "the only one left."
Admittedly, formulating a system of selectional
restrictions capable of handling all of these is itself no walk in the park.
The scientists have been studying three species of deep water fish.
They were not [known/?] until a couple of years ago.
What was not [known/?]
Answers:
The fish/the scientists.
Comment:
I can't find a verb that indicates scientists but
is not a selectional restriction violation for "species"
Tom's books are full of mistakes. Some of them are quite [foolish/worthless].
What are [foolish/worthless]?
Answers:
The mistakes/the books.
Comment: "Foolish mistakes" is Googlable.
I tried to keep the dogs out of the kitchen by putting a chair in the middle
of the doorway, but it was too [wide/small].
What was too [wide/small]?
Answers:
The doorway/the chair.
Comment:
There are selectional preferences for associating "wide" with "doorway" rather
with "chair" and for associating "narrow" with "doorway" rather than "chair"
that suffice to give a cheap method for solving this. These are not strict
selectional constraints; one can speak of a "wide chair" or a "small
doorway" without seeming anomalous. But they are strong enough preferences
that, if there is an ambiguity where this preference points in one direction
while a spatial or physical inference points in the opposite direction,
the resulting text is confusing. For example, in the sentence
I tried to keep the dogs out of the kitchen by putting a chair in the middle
of the doorway, but it was too narrow
the reader may have difficulty realizing that, geometrically, "it" must
refer to the chair, because "narrow" is a more suitable descriptor for a
doorway than for a chair. See
"Qualitative spatial reasoning in interpreting text and narrative,"
E. Davis, Spatial Cognition and Computation, to appear.
I brought the flower pot out to the garage. It is full of [dirt/tools].
What is full of [dirt/tools]?
Answers:
The flower pot/the garage.
Comment:
This was actually included as an example in the collection of Winograd
schemas for a while. However, a number of readers opined that the version
with "tools" was unclear or confusing, because it is not that much more
plausible that a garage should be full of tools than that a flower pot
should be full of tools.
Three examples of difficult reference resolution from natural text.
The following three examples are not Winograd schemas, but they serve to
illustrate the very considerable problems of reference resolution in
high-quality, naturally occurring text.
The affection of the whole family, the warm attachment of Miss Campbell
in particular, was the more honorable to each party from the circumstances
of Jane's decided superiority both in beauty and acquirements. That nature
had given it in feature could not be unseen by the young woman, nor could
her higher powers of mind be unfelt by the parents.
Does "the young woman" refer to Jane or Miss Campbell?
Answer: Miss Campbell.
Comment:
From Jane Austen, Emma, part II, chapter 2. Presumably both
Miss Campbell and Jane were aware of Jane's superior beauty, but considerations
of textual coherence
--- not at all easy to characterize --- require that this refers
to Miss Campbell. I do not feel confident that an overwhelming majority of
human subjects would get this one right.
Note also that the quoted reference to the text in the question --- a
form that we avoid in the real Winograd schemas --- is difficult to avoid
here. Asking about content gets pretty stilted: "Which of the young women could
not help seeing that nature had given Jane superior features?"
Late in 2009, the novelist Jim Powell found a cache of letter written by
George Meredith to his great-great-grandmother Susan Mary Neill.
Whose great-great-grandmother?
Answers:
Powell's.
Comment:
This is from the article "A peacock in the attic" by Nicholas Joukovsky
and Jim Powell, Times Literary Supplement, July 22, 2011.
I myself (Ernie Davis) misinterpreted this on the first reading, apparently
letting the rule that people write to their own relatives override the rules
that few people have living great-great-grandmothers and that people
do not write to deceased relatives.
One could easily turn this into Winograd schema format:
Late in 2009, the novelist Jim Powell found a cache of letter written by
the 19th century novelist George Meredith to his
[great-great-grandmother/friend] Susan Mary Neill
except that the sentence with "great-great-grandmother" is too easily
misunderstood.
Almanzo turned to Mr. Thompson and asked, "Did you lose a pocketbook?"
Mr. Thompson jumped. He slapped a hand to his pocket and fairly shouted.
"Yes, I have! Fifteen hundred dollars in it, too! What about it? What do you
know about it?"
"Is this it?" Alamanzo asked.
From Farmer Boy by Laura Ingalls Wilder. Note that in Almanzo's
question, "Is this it?", "this" refers to the wallet that Almanzo is holding
out (implicit in the text), whereas "it" refers to Mr. Thompson's wallet.
The example is particularly tricky since the two are in fact co-extensional.
This example was noticed by Gary Marcus.