ExplainaBoard
ExplainaBoard copied to clipboard
neulab
Interpretable Evaluation for AI Systems
This is a major refactoring of ExplainaBoard to: 1. Introduce the concept of "analysis levels", where analysis can be performed over examples, spans, tokens, etc. separately. In order to implement...
Several functions use NumPy's structured arrays. I thought it is better to replace them to dataclasses for several reasons: - It is basically not informative to users to know what...
Hi everyone, maybe especially @odashi and @pfliu-nlp: I'm thinking that we might want to make a more complete contribution guide with the dual goal of: 1. Making it easier for...
This PR aims to make feature functions customizable, either through build-in or build-out definitions. For example, ##### (1) Build-out ```python loader = get_loader_class(TaskType.text_classification).from_datalab( dataset=DatalabLoaderOption( "sst2", custom_features={ "long_text_50": { "dtype": "string",...
This adds rudimentary visualization for confusion matrices to the offline visualization tool. These will be output automatically when using the `draw_charts` utility (which used to be called the `draw_hist` utility)...
SysOutputInfo.write_to_directory() is useful for writing out the results of an analysis. However, if the directory doesn't exist it isn't created automatically, which is a bit annoying.
(This is suggested by @odashi ) The current documentation mainly lacks things except how-to guides that makes other developers hard to understand how the software is developed.
Version 0.11 can be merged into main once a certain level of support has been achieved in explainaboard_web.
# Background This repository hosts all task/metric definitions that the ExplainaBoard handles, and it seems we face several problems to maintain the current development manner: - The size of the...
Currently the visualization scripts will not output visualizations for CombinationCountAnalysis, but they should.