generate-template-files icon indicating copy to clipboard operation
generate-template-files copied to clipboard

Perform replacement in one pass of the output

Open TikiTDO opened this issue 2 years ago • 4 comments

This changes the transform pass of the generator to run in one pass of the output string, instead of using string replacer. It might make sense to just make something like an applyTransformersToString, or whatever name you might like, and put it in a util file. If you have an opinion on where it should go I'd be happy to move it.

TikiTDO avatar Jul 15 '21 02:07 TikiTDO

The issue with replaceString implementation is that it will have to go through the entire string once for every potential replacement token/case combination.

With the addition of the underline case values that means each string has to be iterated through 12 times for every single token. It's a few milliseconds for a small file with one or two tokens, but if you were trying to generate a really big template with a bunch of tokens it might actually start to matter.

The implementation above will iterate through the output string once, and replace each appropriate value as it finds it. It is 12 * # of Tokens times faster, minus the time to compile the regex, which should be trivial for a mostly static regex like this one.

TikiTDO avatar Jul 15 '21 02:07 TikiTDO

For more concrete numbers, I did a test with three approaches. The simple | approach up above, a regex-trie implementation (with the downside of extra deps), and the original one from the code with a worst case, 25 replacer run, repeated 100,000 times. Results came out as follows:

trie tools/templates/react/component/__name__.module.scss: 88.985ms
simple tools/templates/react/component/__name__.module.scss: 46.941ms
orig tools/templates/react/component/__name__.module.scss: 641.433ms

trie tools/templates/react/component/__name__.spec.tsx: 97.956ms
simple tools/templates/react/component/__name__.spec.tsx: 154.666ms
orig tools/templates/react/component/__name__.spec.tsx: 1.104s

trie tools/templates/react/component/__name__.tsx: 90.612ms
simple tools/templates/react/component/__name__.tsx: 144.562ms
orig tools/templates/react/component/__name__.tsx: 1.127s

The trie is more effective when working with larger files, while the simple case rips through smaller files. Both were around 7-12x faster than the replaceString function for this use case. I'll commit both implementations so you can compare.

TikiTDO avatar Jul 15 '21 04:07 TikiTDO

Thanks for the insight why you are changing out replaceString. I want to look into this more but sadly I will away from my computer for the next two weeks. I will have to revisit this later.

codeBelt avatar Jul 15 '21 14:07 codeBelt

No rush. The other two PRs were the really important ones. This one is just a performance optimization for super extreme edge cases. Really just idle work while I was waiting for a build.

TikiTDO avatar Jul 15 '21 14:07 TikiTDO