Score ACE Relations
RelationScorer implements a simple scoring metric for ACE 2004 and 2005
relation mentions. A relation mention is considered correct if
its two arguments align with those of a relation in the key and the
relation has the correct type (subtypes are ignored). An entity in
the key is aligned with an entity in the response if the ends of
the heads of the entitities are the same. If the particular relation
is symmetric, a relation mention in the key will match one with
reversed arguments in the system response.
It is invoked by
xjet
AceJet.RelationScorer
docList textDirectory textExtension
responseDirectory responseExtension keyDirectory keyExtension [year]");
where
- docList
- file containing list of document file names, one per line
- textDirectory
- directory containing text files for documents
- textExtension
- file extension for text files
- responseDirectory
- directory containing APF files to be scored
- responseExtension
- file extension for APF files to be scored
- keyDirectory
- directory containing APF keys
- keyExtension
- file extension for APF keys
- year
- 2004 or 2005, indicating which APF file format and relation inventory is used (default is 2005)
If one line in the docList file is d, the program will
compare the APF files
responseDirectory/d.responseExtension and
keyDirectory/d.keyExtension.