crowd-kit

crowd-kit copied to clipboard

Binary Relevance aggregation

1

## Description I have added code for Binary Relevance aggregation - simple method for multi-label classification. This approach treats each label as a class in binary classification task and aggregates...

denaxen

Ordinal Labels

2

Is it possible to support aggregation of ordinal labels as a part of this toolkit via this reduction algorithm. - Labels are categorical but have an ordering defined 1 <...

vikasraykar

crowdkit/__init__.py import fix

1

fix the import bug in https://github.com/Toloka/crowd-kit/issues/88 The following test passes: ```python import crowdkit def test_mmsr(): try: mmsr = crowdkit.aggregation.classification.m_msr.MMSR except AttributeError as e: print(f"An error occurred: {e}") test_mmsr() print("Test passed!")...

ahundt

### Observed behavior ``` import crowdkit # ... mmsr = crowdkit.aggregation.classification.m_msr.MMSR( n_iter=10000, tol=1e-10, n_workers=len(worker_to_id), n_tasks=len(st2_int), n_labels=2, # Assuming binary responses workers_mapping=worker_to_id, tasks_mapping=task_to_id, labels_mapping=label_to_id, ) ``` ``` Exception has occurred: AttributeError...

ahundt

[FEATURE] Unifying classification aggregators interfaces

3

### Problem description I wanted to test quality metrics of several different algorithms from `crowdkit.aggregation.classification` and found myself writing such kind of function: ```python def get_scores(model, data, fit=True): if fit:...

alexdremov