- Run validation methods on your annotations against a gold standard
- Extract evaluation/correlation metrics between annotations (precision, recall, f-score, correlation metrics)
- Interpret implications of those metrics on the data, explore other/complementary evaluation methods.
Meet up with lecturers (Cristia, Cychosz, Soderstrom), topics:
- Sampling, annotation scheme, & software for your own research goals
- Feedback on annotator instructions