09:00
10:00
11:00
12:00
13:00
14:00
15:00
16:00
›9:00 (10min)
9:00 - 9:10 (10min)
Welcome & feedback on previous day's task
›9:10 (1h05)
9:10 - 10:15 (1h05)
manual annotations and reliability
Cristia, Soderstrom, Cychosz
›10:15 (15min)
10:15 - 10:30 (15min)
Break
›10:30 (1h)
10:30 - 11:30 (1h)
Practice annotating data
Soderstrom
›11:30 (30min)
11:30 - 12:00 (30min)
Practice validating your manual annotations
- Run validation methods on your annotations against a gold standard
- Extract evaluation/correlation metrics between annotations (precision, recall, f-score, correlation metrics)
- Interpret implications of those metrics on the data, explore other/complementary evaluation methods.
- Extract evaluation/correlation metrics between annotations (precision, recall, f-score, correlation metrics)
- Interpret implications of those metrics on the data, explore other/complementary evaluation methods.
›14:00 (2h)
14:00 - 16:00 (2h)
Office hours (with appointment)
Meet up with lecturers (Cristia, Cychosz, Soderstrom), topics:
- Sampling, annotation scheme, & software for your own research goals
- Feedback on annotator instructions
- Sampling, annotation scheme, & software for your own research goals
- Feedback on annotator instructions
