posthog-js feat: safer string truncation

We truncate strings by slicing them at a given length both in replay and in general capture property processing

String.prototype.slice() and String.prototype.substring() are not unicode aware

so

const str = 'A 😄 B';
console.log(str.slice(2, 3)); // Outputs broken part of emoji

Major browsers and Node 20 have toWellFormed

Our CI/tests/etc don't use Node 20 so even though we could use toWellFormed in prod it would be a pain to add. Moving to Node20 would probably fix some people's builds and break others. Let's avoid confusion for now.

Luckily the one consistent thing about JS is that it is not consistent.

Array.from(someString) is unicode aware and correctly splits a string into an array of valid characters. So we can then slice that array. To get a string that is sort-of the right length.

Our string truncation in the JS is set arbitrarily so it doesn't matter that this mechanism returns strings that may not have a .length that exactly matches the maxLength passed to truncation

Oct 16 '24 13:10 pauldambra

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
posthog-js	✅ Ready (Inspect)	Visit Preview	Dec 17, 2024 10:36am

Oct 16 '24 13:10 vercel[bot]

@daibhin since this would maybe help with your upgrade PR...

I'm considering adding a (lazy loaded?) polyfill for array.from

Oct 16 '24 13:10 pauldambra

Size Change: +1.47 kB (+0.05%)

Total Size: 3.22 MB

Filename	Size	Change
`dist/array.full.es5.js`	262 kB	+153 B (+0.06%)
`dist/array.full.js`	365 kB	+147 B (+0.04%)
`dist/array.full.no-external.js`	364 kB	+147 B (+0.04%)
`dist/array.js`	179 kB	+146 B (+0.08%)
`dist/array.no-external.js`	178 kB	+145 B (+0.08%)
`dist/main.js`	180 kB	+146 B (+0.08%)
`dist/module.full.js`	365 kB	+147 B (+0.04%)
`dist/module.full.no-external.js`	364 kB	+147 B (+0.04%)
`dist/module.js`	179 kB	+145 B (+0.08%)
`dist/module.no-external.js`	178 kB	+145 B (+0.08%)

ℹ️ View Unchanged

Filename	Size
`dist/all-external-dependencies.js`	206 kB
`dist/customizations.full.js`	12.6 kB
`dist/dead-clicks-autocapture.js`	14.4 kB
`dist/exception-autocapture.js`	9.48 kB
`dist/external-scripts-loader.js`	2.48 kB
`dist/recorder-v2.js`	115 kB
`dist/recorder.js`	115 kB
`dist/surveys-preview.js`	57.6 kB
`dist/surveys.js`	63.3 kB
`dist/tracing-headers.js`	1.76 kB
`dist/web-vitals.js`	10.4 kB

_{compressed-size-action}

Oct 16 '24 13:10 github-actions[bot]

NO, that polyfill definitely does not work 🤣

Oct 16 '24 15:10 pauldambra

ok, corejs array.fom polyfill adds 11% to the bundle... that's no bueno

Oct 16 '24 22:10 pauldambra

interesting... looks like the array.from polyfill in IE11 bundle makes this timeout 🤯

Oct 21 '24 09:10 pauldambra