preprocessor refactor

Open mumencoder opened this issue 4 years ago • 5 comments

has a few things going for it so far:

fully tokenizes most/all of the major SS13 codebases (output not verified yet) same token output as master for the testing version of tgstation no token buildup in the preprocessor, uses 20-30MB memory throughout the whole tokenization phase

Oct 02 '21 00:10 mumencoder

output.txt took 10 minutes and came out with this

Oct 31 '21 00:10 redmoogle

This pull request has conflicts, please resolve those before we can evaluate the pull request.

Mar 31 '22 16:03 github-actions[bot]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

May 30 '22 02:05 github-actions[bot]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

Jun 06 '22 02:06 github-actions[bot]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

Jun 17 '22 04:06 github-actions[bot]