-
Notifications
You must be signed in to change notification settings - Fork 226
Open
Description
Problem: MOABB assumes paradigms use one score for evaluations. However, besides discrimination (e.g. accuracy, AUROC) measuring calibration (e.g. NLL, Brier score, ECE) is also useful.
Proposed Solution: Allows custom paradigms to use multiples scorers. The parameter additional_columns from evaluations is used for multiple scores in results. The column score in results is kept and used as a register for statistical analysis and plotting.
Google Colab: https://colab.research.google.com/drive/1gFDBIfdbAWl2UdvslxVnLFhV7qws6_qm?usp=sharing
Metadata
Metadata
Assignees
Labels
No labels