crawlee icon indicating copy to clipboard operation
crawlee copied to clipboard

Add ability to determine which column should be exported

Open rikusen0335 opened this issue 2 years ago • 0 comments

Which package is the feature request for? If unsure which one to select, leave blank

@crawlee/core

Feature

For now, Dataset.exportToCSV only acccpets the key as first argument and options which only include toKSV property due to exportTo function do.

I need to set column name, if I store the data like:

...

await dataset.pushData({ name: someName, value: someValue, isExist: true })

and then I only need just name

await dataset.exportToCSV("something", { columnKey: [
  { key: "name", label: "fooName" },
  { key: "isExist", label: "existence" },
] })

Thus, we can get csv like this:

fooName,existence
someName1,true
someName2,false
...

instead of:

name,value,isExist
someName1,someValue1,true
someName2,someValue2,false
...

You may think "if the column name is only English that's already fine", but if we need like Japanese, Chinese instead of English we may need translations of column name.

Not just to say about exportToCSV, it is very nice if we can make custom of exportToCSV.

Motivation

I need to limit the column name which is exported, and I want to change name, so that we can use it with our language and useful for excel or something to use.

Ideal solution or implementation, and any additional constraints

  1. Add option peroperty to function

When store the data like:

...

await dataset.pushData({ name: someName, value: someValue, isExist: true })

and then I only need just name

await dataset.exportToCSV("something", { columnKey: [
  { key: "name", label: "fooName" },
  { key: "isExist", label: "existence" },
] })
  1. Add ability to define a custom function like plugin, that you need to export more flexible API
// The 3rd argument is currently only accepts content-type, but need to be change in this situation
await dataset.exportTo("something", {}, "custom")

Alternative solutions or implementations

No response

Other context

No response

### Tasks
- [ ] tst

rikusen0335 avatar Jan 13 '23 06:01 rikusen0335