attelo.metrics package

Submodules

attelo.metrics.tree module

Metrics to assess performance on tree-structured predictions.

Functions named as *_loss return a scalar value to minimize: the lower the better.

attelo.metrics.tree.labelled_tree_loss(ref_tree, pred_tree)

Compute the labelled tree loss.

The labelled tree loss is the fraction of edges that are incorrectly predicted, with a lesser penalty for edges with the correct attachment but the wrong label.

Parameters:

• ref_tree (list of edges (source, target, label)) – reference tree
• pred_tree (list of edges (source, target, label)) – predicted tree

Returns:

loss – Return the tree loss between edges of ref_tree and pred_tree.

Return type:

float

See also

tree_loss()

Notes

The labelled tree loss counts only half of the penalty for edges with the right attachment but the wrong label.

attelo.metrics.tree.tree_loss(ref_tree, pred_tree)

Compute the tree loss.

The tree loss is the fraction of edges that are incorrectly predicted.

Parameters:

• ref_tree (list of edges (source, target, label)) – reference tree
• pred_tree (list of edges (source, target, label)) – predicted tree

Returns:

loss – Return the tree loss between edges of ref_tree and pred_tree.

Return type:

float

See also

labelled_tree_loss()

Notes

For labelled trees, the tree loss checks for strict correspondence: it does not differentiate between incorrectly attached edges and correctly attached but incorrectly labelled edges.