icu4x icon indicating copy to clipboard operation
icu4x copied to clipboard

Allow combinatorics on provider::Baked

Open sffc opened this issue 2 years ago • 5 comments
trafficstars

Users should be able to chain provider::Baked providers to make a multi-crate AnyProvider.

@Manishearth suggested aked.add(otherprovider::Baked) making a fork provider.

sffc avatar Aug 24 '23 21:08 sffc

crate::provider::Baked is an implementation detail in the unstable provider modules. Users can use compiled data by depending on the data crates and creating their own baked providers:


struct MyBakedProvider;

icu_datetime_data::impl_datetime_chinese_datelengths_v1!(MyBakedProvider);
icu_datetime_data::impl_datetime_chinese_datesymbols_v1!(MyBakedProvider);
icu_list_data::impl_and_list_v1!(MyBakedProvider);

robertbastian avatar Aug 25 '23 13:08 robertbastian

Discuss with:

  • @Manishearth
  • @robertbastian
  • @sffc

sffc avatar Sep 05 '23 18:09 sffc

Discussion: revisit this in the context of #3947, when we remove AnyProvider. Leaving some breadcrumbs for my mental model of this and #1893:

  • We should have a macro impl_forking_keyed_provider!([key1, key2, key3], Provider1, Provider2) that generates a impl<M: KeyedDataMarker> DataProvider<M> where the list of keys go to Provider1 and any key not in the list goes to Provider2. Note that DCE should work fine through this layer because the match arm is statically known based on M::KEY.
  • The provider::Baked impls can have a function that internally contains this match statement for all of the keys it supports.

sffc avatar Oct 31 '23 21:10 sffc

Another use case here is if you want your language packs to contain only non-singleton keys and get the singleton keys from compiled data with the non-singleton keys from dynamically loaded data.

A few approaches here:

  1. Use with_any_provider constructors and a registry macro that forks between the singleton compiled data and the blob dynamic data
  2. Use _unstable constructors (but then we don't handle older blob data)
  3. Use a new type of constructor called _unstable_with_backwards_compatibility
  4. Use _with_buffer_provider and export the singleton keys as blobs from the data crates
  5. In 2.0 make _with_provider constructors that take an enum between BufferProvider and AnyProvider (or equivalent) and fork between the singleton static data and the blob data (and use this opportunity to reduce the number of constructors -- rather than deleting _with_any_provider we merge it in with the buffer provider constructor, which slightly increases the buffer provider constructor code size but only by the size of a ZeroFrom)

sffc avatar Nov 06 '23 06:11 sffc

Probably doesn't block 2.0 so removing it from that milestone, but it is still important.

sffc avatar Mar 14 '24 17:03 sffc