Score ACE Relations

RelationScorer implements a simple scoring metric for ACE 2004 and 2005 relation mentions. A relation mention is considered correct if its two arguments align with those of a relation in the key and the relation has the correct type (subtypes are ignored). An entity in the key is aligned with an entity in the response if the ends of the heads of the entitities are the same. If the particular relation is symmetric, a relation mention in the key will match one with reversed arguments in the system response.

It is invoked by

xjet AceJet.RelationScorer docList textDirectory textExtension responseDirectory responseExtension keyDirectory keyExtension [year]");

where
docList
file containing list of document file names, one per line
textDirectory
directory containing text files for documents
textExtension
file extension for text files
responseDirectory
directory containing APF files to be scored
responseExtension
file extension for APF files to be scored
keyDirectory
directory containing APF keys
keyExtension
file extension for APF keys
year
2004 or 2005, indicating which APF file format and relation inventory is used (default is 2005)
If one line in the docList file is d, the program will compare the APF files responseDirectory/d.responseExtension and keyDirectory/d.keyExtension.