mathics-scanner
mathics-scanner copied to clipboard
Mathics3
Tokenizer, and character tables, and conversion routines for the Wolfram Language.
While working in https://github.com/Mathics3/mathics-scanner/pull/11 it became clear to me that the tokeniser is still very much tied up to internals of Mathics core. For instance, the whole messaging mechanism is...
We should add some documentation on the tokenizer before releasing the package. Simple docstrings will do for now.
We should decide what's going to be part of our public API before releasing. Things that aren't in the public API should be marked with a `_` at the start...
This issue is to discuss and track the following work I propose to do. For the sake of limiting scope, this issue is restricted to working on "incorrectly" named operators...
To handle escape sequences better, such as ignoring them in comments, branch "revise-escape-sequence-scanning" was started as more major refactor. However, that has become too large and is too hard to...
Refactor the scanner. Remove prescanner. Handling escape sequences is not a separate phase, but instead is integrated into the scanning phase.
Example: `Print["Hello q\.b4"]` (for example after starting the command `mathics`) I found this when trying to get [xAct](https://josmar493.dreamhosters.com/download/xAct_1.2.1.tgz) to work. Specifically, I tried xTensor/xTensor.m there.
