croissant icon indicating copy to clipboard operation
croissant copied to clipboard
verified publisher icon mlcommons

Define and set a unique user-agent

Open goeffthomas opened this issue 1 year ago • 1 comments

If the metadata for Croissant is pulled via URL (done here), we should set a user-agent that allows the package to be identified.

For reference, kagglehub does something similar here

goeffthomas avatar Nov 01 '24 17:11 goeffthomas

We should probably add that same user agent when downloading the files over HTTP: https://github.com/mlcommons/croissant/blob/main/python/mlcroissant/mlcroissant/_src/operation_graph/operations/download.py#L188-L190

goeffthomas avatar Nov 01 '24 17:11 goeffthomas