-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Description
Issue from PR #540
Load this benchmark: https://dev.explainaboard.inspiredco.ai/benchmark?id=globalbench_ner
I got this error when loading this benchmark:
[0] File "/Users/jiangqi/Desktop/Capstone/explainaboard_web/backend/src/gen/explainaboard_web/impl/db_utils/benchmark_db_utils.py", line 187, in <setcomp>
[0] (x.dataset.dataset_name, x.dataset.sub_dataset, x.dataset.split)
[0] AttributeError: 'NoneType' object has no attribute 'dataset_name'
This is because this benchmark try to find all ner systems with 'system_query': {'task_name': 'named-entity-recognition'}, but there are systems with undefined/custom dataset, so their dataset is None. One way to deal with undefined datasets is to ignore undefined datasets in benchmark.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels