(file name: score-summary.TE.04may95) SUMMARY SCORE REPORTS FOR MUC-6 DRY RUN OF THE TEMPLATE ELEMENT SUBTASK OF THE INFORMATION EXTRACTION TASK If you are unfamiliar with the MUC metrics and the format of the score reports, you can get a sense of the performance if you look in the Recall (REC) and Precision (PRE) columns. They measure the extent to which the system got all the expected info (Recall) and only the expected info (Precision). For overall ("bottom-line") results, look at Recall and Precision in the ALL OBJECTS row. ================================================================================ System Pseudonym: Anybody * * * TOTAL SLOT SCORES * * * --------------------+-----------+-----------+-----------+----------------------- SLOT POS ACT|COR PAR INC|XCR XPA XIC|SPU MIS NON|REC PRE UND OVG ERR SUB --------------------+-----------+-----------+-----------+----------------------- 234 244|215 0 10| 0 0 0| 19 9 0| 92 88 4 8 15 4 name 234 244|210 0 15| 0 0 0| 19 9 0| 90 86 4 8 17 7 alias 82 76| 74 0 0| 0 0 0| 2 8 136| 90 97 10 3 12 0 title 73 73| 69 0 0| 0 0 0| 4 4 142| 94 94 5 5 10 0 subtotals 389 393|353 0 15| 0 0 0| 25 21 278| 91 90 5 6 15 4 --------------------+-----------+-----------+-----------+----------------------- ALL OBJECT 2414 1787|1484 0 184| 0 0 0|119 746 2344| 61 83 31 7 41 11 MATCHED ON 2045 1679|1484 0 147| 0 0 0| 48 414 2344| 72 88 20 3 29 9 --------------------+-----------+-----------+-----------+----------------------- P&R 2P&R P&2R F-MEASURES 70.65 77.6 64.84 ================================================================================ System Pseudonym: Kosh * * * TOTAL SLOT SCORES * * * --------------------+-----------+-----------+-----------+----------------------- SLOT POS ACT|COR PAR INC|XCR XPA XIC|SPU MIS NON|REC PRE UND OVG ERR SUB --------------------+-----------+-----------+-----------+----------------------- 234 296|189 0 37| 0 0 0| 70 8 0| 81 64 3 24 38 16 name 234 296|170 0 56| 0 0 0| 70 8 0| 73 57 3 24 44 25 alias 82 64| 61 0 0| 0 0 0| 3 21 112| 74 95 26 5 28 0 title 74 63| 53 0 0| 0 0 0| 10 21 112| 72 84 28 16 37 0 subtotals 390 423|284 0 56| 0 0 0| 83 50 224| 73 67 13 20 40 16 --------------------+-----------+-----------+-----------+----------------------- ALL OBJECT 2411 1857|1278 2 298| 0 0 0|279 833 2192| 53 69 34 15 52 19 MATCHED ON 1922 1579|1278 2 202| 0 0 0| 97 440 2192| 66 81 23 6 37 14 --------------------+-----------+-----------+-----------+----------------------- P&R 2P&R P&2R F-MEASURES 59.93 65.0 55.6 ================================================================================ System pseudonym: Quartz * * * TOTAL SLOT SCORES * * * --------------------+-----------+-----------+-----------+----------------------- SLOT POS ACT|COR PAR INC|XCR XPA XIC|SPU MIS NON|REC PRE UND OVG ERR SUB --------------------+-----------+-----------+-----------+----------------------- 234 230|200 0 11| 0 0 0| 19 23 0| 85 87 10 8 21 5 name 234 234|189 0 22| 0 0 0| 23 23 0| 81 81 10 10 26 10 alias 82 78| 72 0 0| 0 0 0| 6 10 122| 88 92 12 8 18 0 title 73 70| 69 0 0| 0 0 0| 1 4 130| 94 98 5 1 7 0 subtotals 389 382|330 0 22| 0 0 0| 30 37 252| 85 86 10 8 21 6 --------------------+-----------+-----------+-----------+----------------------- ALL OBJECT 2416 1808|1213 0 330| 0 0 0|265 873 1951| 50 67 36 15 55 21 MATCHED ON 1805 1563|1213 0 258| 0 0 0| 92 334 1951| 67 78 18 6 36 18 --------------------+-----------+-----------+-----------+----------------------- P&R 2P&R P&2R F-MEASURES 57.43 62.86 52.87 ================================================================================ System pseudonym: everhopeful * * * TOTAL SLOT SCORES * * * --------------------+-----------+-----------+-----------+----------------------- SLOT POS ACT|COR PAR INC|XCR XPA XIC|SPU MIS NON|REC PRE UND OVG ERR SUB --------------------+-----------+-----------+-----------+----------------------- 234 149|139 0 9| 0 0 0| 1 86 0| 59 93 37 1 41 6 name 234 148| 86 0 61| 0 0 0| 1 87 0| 37 58 37 1 63 41 alias 82 46| 42 0 1| 0 0 0| 3 39 69| 51 91 48 6 50 2 title 74 67| 43 0 10| 0 0 0| 14 21 61| 58 64 28 21 51 19 subtotals 390 261|171 0 72| 0 0 0| 18 147 130| 44 66 38 7 58 30 --------------------+-----------+-----------+-----------+----------------------- ALL OBJECT 2414 1571|987 23 256| 0 0 0|305 1148 1610| 41 64 48 19 63 21 MATCHED ON 1612 1336|987 23 196| 0 0 0|130 406 1610| 62 75 25 10 43 17 --------------------+-----------+-----------+-----------+----------------------- P&R 2P&R P&2R F-MEASURES 50.11 57.4 44.47 ================================================================================ System pseudonym: sinatra * * * TOTAL SLOT SCORES * * * --------------------+-----------+-----------+-----------+----------------------- SLOT POS ACT|COR PAR INC|XCR XPA XIC|SPU MIS NON|REC PRE UND OVG ERR SUB --------------------+-----------+-----------+-----------+----------------------- 234 261|204 0 17| 0 0 0| 40 13 0| 87 78 6 15 26 8 name 234 261|155 0 66| 0 0 0| 40 13 0| 66 59 6 15 43 30 alias 82 0| 0 0 0| 0 0 0| 0 82 128| 0 * 100 * 100 * title 74 171| 62 0 8| 0 0 0|101 4 75| 84 36 5 59 64 11 subtotals 390 432|217 0 74| 0 0 0|141 99 203| 56 50 25 33 59 25 --------------------+-----------+-----------+-----------+----------------------- ALL OBJECT 2412 1932|1108 0 334| 0 0 0|490 970 2320| 46 57 40 25 62 23 MATCHED ON 2009 1506|1108 0 252| 0 0 0|146 649 2320| 55 74 32 10 48 18 --------------------+-----------+-----------+-----------+----------------------- P&R 2P&R P&2R F-MEASURES 51.01 54.64 47.84 ================================================================================ System pseudonym: world_beater * * * TOTAL SLOT SCORES * * * --------------------+-----------+-----------+-----------+----------------------- SLOT POS ACT|COR PAR INC|XCR XPA XIC|SPU MIS NON|REC PRE UND OVG ERR SUB --------------------+-----------+-----------+-----------+----------------------- 234 31| 25 0 5| 0 0 0| 1 204 0| 11 81 87 3 89 17 name 234 31| 5 0 25| 0 0 0| 1 204 0| 2 16 87 3 98 83 alias 82 1| 0 0 0| 0 0 0| 1 82 5| 0 0 100 100 100 * title 73 31| 19 0 1| 0 0 0| 11 53 0| 26 61 73 35 77 5 subtotals 389 63| 24 0 26| 0 0 0| 13 339 5| 6 38 87 21 94 52 --------------------+-----------+-----------+-----------+----------------------- ALL OBJECT 2406 251|147 0 83| 0 0 0| 21 2176 333| 6 58 90 8 94 36 MATCHED ON 325 216|147 0 62| 0 0 0| 7 116 333| 45 68 36 3 56 30 --------------------+-----------+-----------+-----------+----------------------- P&R 2P&R P&2R F-MEASURES 11.07 21.55 7.44