Evaluation with attelo report

The attelo report command generates a set of evaluation reports by comparing attelo decode results against a gold standard. So far it creates:

• global precision/recall reports
• confusion matrices

There are two ways to use attelo report. You have it report scores on a single predictions file (produced by attelo decode); or you can have it report on a full set of predictions generated by a harness over multiple folds.

Mode A: predictions file

For one-off tests on attelo decode results, use the predictions file mode (–prediction <FILE>).

(NB: if your test was on a particular fold of the data you can also supply the –fold and –fold-file arguments to slice the data)

Mode B: harness mode

Harness-level reporting is only available programmatically via the attelo API