Epic: Polyfill Flexibility

[Shaptic]

Epic: Polyfill Flexibility

This is the master issue to track work necessary to tackle the inflexibility of the SDK in regards to dependencies and polyfills. It lives in this repo (stellar/js-stellar-sdk) but the work covers multiple repos.

First, the doc that put us on this path: link (see 2. Packaging).

Package Structure

We need our packages to be more modular:

• [x] Drop polyfills from all packages
  • [x] js-xdr
  • [x] stellar-base
  • [x] stellar-sdk
• [ ] Create a version of stellar-base with different crypto backends:
  • [ ] stellar-base will use sodium-native
  • [ ] stellar-base@unsalted will use tweetnacl
  • [ ] stellar-base@node will use Node's crypto (or a polyfill)
• [ ] Create a bundled version of the main libraries with everything polyfilled, installable with the @full tag, e.g. yarn add stellar-base@full:
  • [ ] stellar-base
  • [ ] stellar-sdk
  • These will be "combine-able" with the crypto backends
• [x] Merge soroban-client and stellar-sdk into one library:
  • They will share the entire build pipeline except...
  • Soroban RPC code will live under a Soroban namespace
  • Horizon code will live under a Horizon namespace
  • XDR will continue to live under the xdr namespace
• [ ] Move strkey (SEP-23) implementation to its own package: @stellar/strkey
• [ ] Move SEP-10 utilities to their own package: @stellar/webauth

Problematic Dependencies

We need to eliminate the use of dependencies that are problematic (meaning: not available in every deployment environment, not cross-platform, not pleasant to use, etc.) or provide thorough instructions on what does or does not need to be polyfilled by the application developer.

The philosophy imbued here follows w3ctag/polyfills#6/comment. The libraries will make the best possible assumption about which library to use if some functionality is needed. For example, if dealing with bytes is commonplace in a library, it will rely on the existence of Buffer. However, as a library, it will not polyfill it on its own. Instead, it should detect the existence of the object at runtime and provide an appropriate error + instructions.

For example, if we started a project:

npm init -y && npm install stellar-sdk
echo 'const sdk = require("stellar-sdk");' > index.js
firefox index.js 

We would get an error in the console:

Error: 'Buffer' polyfill not detected. Configure your bundling pipeline to provide the Buffer class (e.g. https://github.com/feross/buffer).

The following dependencies need work:

• [ ] Buffer, which is problematic because it's only available in Node environments and can be replaced with (a) a custom wrapper over the native Uint8Array, (b) an explicit polyfill instruction, or (c) a full port over to Uint8Array.
  • [ ] in js-xdr
  • [ ] in stellar-base
  • [ ] in stellar-sdk
  • [ ] in stellar-sdk
• [x] axios in stellar-sdk is problematic because it doesn't support HTTP/2 (see stellar/js-stellar-sdk#612 and axios/axios#1175) and can't be used in a lot of environments (see stellar/js-stellar-sdk#241)
• [x] eventsource is problematic because not everyone needs streaming support and SSE isn't supported in all environments
• [ ] sodium-native is problematic because it requires complicated compilation steps and can't always be elided (see https://github.com/stellar/js-stellar-base/issues/646 for a proposed solution)

In fact, many of the dependencies can be replaced.

• in stellar-base, we have:
  • [ ] base32.js can be inlined
  • [ ] bignumber.js can use bigints plus https://github.com/stellar/js-stellar-base/pull/607
  • [ ] buffer can be polyfilled or ported to use native Uint8Arrays (noted above)
  • [x] crc can be inlined (see https://github.com/stellar/js-stellar-base/pull/621)
  • [ ] sha.js can probably be inlined

[kalepail]

fwiw I don't personally think this needs to get shipped before Testnet. That seems very ambitious and I don't think it will greatly affect the ability of developers to build and deploy dapps. Definitely important but not essential. Apologies if I gave that impression. It will be a tremendous quality of life improvement but as it's not really Soroban specific and touches more generally all of the Stellar JS experience I think this can get punted downstream a little if necessary.