allow configuring carSplitter targetSize when storing using client.store

[gobengo]

trafficstars

When using a client to store an nft, I want to be able to configure the targetSize used by the TreewalkCarSplitter.

Motivation:

• @alanshaw suggested a good thought on #1943:

  We don't have to upload 10MB every time. In the Web3.Storage client we expose the DAG splitting size as a param to put. In NFT.Storage you could use carbites to split the DAG by a smaller size and then call storeCar multiple times.

  • but when I went to implement this, I couldn't figure out how to do this without this type of change that would make the targetSize configurable instead of using the MAX_CHUNK_SIZE constant

Candidate Solutions

1. allow configuring targetSize
2. allow injecting a custom splitter, which can have a customized targetSize as well as be any other implementation of something like typeof TreewalkCarSplitter

Current Preferred Solution

• Candidate solution 2. This seems more flexible with little downsides

Afterward:

• make use of this in cron nft-ttr so it stores smaller files but with a carsplit

[alanshaw]

The service requires the CAR to be split with a particular strategy, so exposing a splitter option could be problematic.

Exposing the chunk size is something we could do...

Do we need to add this low level configurability though - why do we need it?

[gobengo]

@alanshaw I don't think we need it. But without it I'm not sure how to implement your thought on #1943 to use smaller than 10Mb images while also testing the carsplit path, so I made this issue to track the perceived blocker

[alanshaw]

@gobengo lets add a maxChunkSize option to putCar but ensure validation is in place to ensure folks pick suitable values (as per this PR).