[FEATURE] Update responses of RemoteFeedbackRecords

[plaguss]

Is your feature request related to a problem? Please describe. We cannot update the responses of RemoteFeedbackRecords.

Describe the solution you'd like I would like to update the responses of RemoteFeedbackRecords using the current API.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Running the following code should result in the records' responses being updated, but it doesn't:

dataset = rg.FeedbackDataset.from_argilla("end2end_textclassification")
dataset_org = load_dataset("ag_news", split="train[:1000]")
id2label = {0: "World", 1: "Sports", 2: "Business", 3: "Sci/Tech"}
mapped_labels = [id2label[label] for label in dataset_org["label"]] 

modified_records = [record for record in dataset.records]

for index, record in enumerate(modified_records):
    record.responses = [
        {
            "values":{
                "label":{
                    "value": mapped_labels[index],
                }
            },
            "inserted_at": datetime.now(),
            "updated_at": datetime.now(),
        }
    ]
    #print(record.responses)
dataset.update_records(modified_records)

The update is done here

[dosubot[bot]]

🤖

Hello @plaguss,

Nice to meet you! I'm Dosu, a bot here to assist you with your issues and queries regarding the Argilla repository. While we're waiting for a human maintainer, I'm here to help you troubleshoot bugs, answer questions, and guide you on becoming a contributor. Let's work on this together!

Thank you for your feature request. Currently, the update_record function in the Argilla repository does not support updating the responses field of RemoteFeedbackRecords through the API. The function currently only supports updating the metadata, suggestions, and vectors fields of a record. Here is the relevant code:

def update_record(
    client: httpx.Client,
    id: Union[str, UUID],
    data: Dict[str, Any],
) -> Response[Union[FeedbackItemModel, ErrorMessage, HTTPValidationError]]:
    ...
    body = {}
    if "metadata" in data:
        body["metadata"] = data["metadata"]
    if "suggestions" in data:
        body["suggestions"] = data["suggestions"]
    if "vectors" in data:
        body["vectors"] = data["vectors"]
    ...

As you can see, the function checks if metadata, suggestions, and vectors are present in the data dictionary and if they are, it adds them to the body dictionary which is then sent in the PATCH request. There is no check or handling for a responses field.

This is something that could be considered for a future update of the Argilla repository. However, as of the current version (v1.19.0), this feature is not available.

You can find the update_record function in the src/argilla/client/sdk/v1/records/api.py file in the Argilla repository.

Sources

• src/argilla/client/sdk/v1/records/api.py

[nataliaElv]

Hi @plaguss . Just trying to understand the issue here: in which scenario it's needed to modify a response from the SDK? Isn't this a risk for the consistency of the dataset?

[plaguss]

Hi! When @kursathalat was adding one of the examples for the end2end task (https://github.com/argilla-io/argilla/issues/4183) we found that responses cannot be updated from the sdk. The update_record method is not showing anything when we try to update the responses from the records as if everything worked out, but it's not really updating them.

[nataliaElv]

Let's consider also the ability to add responses that already have a user_id: https://github.com/argilla-io/argilla/issues/4429

[github-actions[bot]]

This issue is stale because it has been open for 90 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 30 days since being marked as stale.