Operater optimization

[shigma]

Some of the changes are proposals and others are features.

1. [Adjustment] Add context declarations into expressions for supporting following condition:

entranceFunction = Module[{internalFunc},
    internalFunc[foo_] := bar;
    Return[doSomethingWith[internalFunc]];
]

2. [Chore] Delete unnecessary regex grouping and quotation marks.
3. [Feature] Support numbers with base, scientific notation, precision or accuracy. See here.

   11^^11.11`11*^11
(* ^^^^^^^^^^^^^^^^^ constant.numeric *)

4. [Adjustment] Display built_in_options with variable.parameter. I know this is not that accurate, but much better than variable.function. Any ideas for this scope are welcomed.
5. [Feature] Support shorthand syntax for Out.

  %%%
(*^^^ storage.type.Out *)
  %12
(*^^^ storage.type.Out *)

6. [Feature] Support operator for Power.
7. [Adjustment] Display & with variable.function instead of keyword.operator. I think this would be more appropriate.
8. [Break] Remove the rule below. I don't know what does a prefix have to do with "constant".

match: ((?:{{symbol}}`)*(\${{symbol}}))(?=\s*:?=)
scope: entity.name.constant.wolfram

9. [Feature] Support shorthand syntax for Infix.

  a ~ f ~ b ~ g ~ c
(*    ^ variable.function *)
(*            ^ variable.function *)

10. [Feature] Support operator for Definition and FullDefinition.
11. [Adjustment] Better display with specific functions, for example:

  Sum[x, {x, 1, 100}]
(*        ^ variable.parameter *)
  Map[func, list]
(*    ^^^^ variable.function *)
  TakeWhile[list, func]
(*                ^^^^ variable.function *)

These functions are currently listed as variables. With further development of branch new-build-system, we can easily generate them within lines of code.

[shigma]

Updates

1. Optimize number pattern, support scope names with base, precision, accuracy and exponent.

  11`
(*^^ constant.numeric *)
(*  ^ constant.numeric.precision *)
  11.`11
(*^^^ constant.numeric *)
(*   ^^^ constant.numeric.precision *)
  .11``
(*^^^ constant.numeric *)
(*   ^^ constant.numeric.accuracy *)
  11.11``
(*^^^^^ constant.numeric *)
(*     ^^ constant.numeric.accuracy *)
  11^^1a
(*^^^^ constant.numeric.base *)
(*    ^^ constant.numeric *)
  11.11*^-11
(*^^^^^ constant.numeric.wolfram *)
(*     ^^^^^ constant.numeric.exponent *)

2. Optimize parameter pattern, support scope names with blank, head and default (which has not been supported in the past).

  _.
(*^^ variable.parameter.default *)

  var_head
(*^^^^^^^^ meta.parameter *)
(*^^^ variable.parameter *)
(*   ^ variable.parameter.blank *)
(*    ^^^^ variable.parameter.head *)

  var__head : foo
(*          ^ meta.pattern keyword.operator.Optional *)

  var___head ? EvenQ
(*           ^ meta.pattern keyword.operator.PatternTest *)

  var: patt ? EvenQ : foo
(*^^^ variable.parameter *)
(*   ^ keyword.operator.Pattern *)
(*     ^^^^^^^^^^^^ meta.pattern *)
(*     ^^^^ variable.other*)
(*          ^ keyword.operator.PatternTest *)
(*            ^^^^^ variable.function *)
(*                  ^ keyword.operator.Optional *)
(*                    ^^^ variable.other *)

3. Remove function classifications, which will be discussed in another PR.
4. Add DynamicModule into scoping_functions.
5. Merge pattern-short into variables for better semantics.
6. Fix a pattern capture mistake.
7. Add some comments.

[chere005]

Re 7 I would keep & an operator.. There's some good features in this PR, but a lot to go through all at once. Smaller PRs might make it easier to get things merged since we can't approve it until all changes are OK. I see good work on supporting : as Optional, I haven't had a chance to check this out and try it yet but does it work for recognizing foo : (_Integer | _String) as pattern? I'll try to look at this more carefully in the next couple weeks.

[chere005]

I did install it to start looking at things actually... first issue:

If[rowStart > 1,
	rowList = Replace[rowList, x_Integer :> x - rowStart + 1, {1}];
];

[chere005]

Another issue:

Module[{data, fileMode, dims, enc, dataStrm, ignoreEmptyLines, summary},

	(* Load LibraryLink functions *)
	If[!loadAdapter[],
		Return@CreateSVToolsException["AdapterNotLoaded"];
	];

	enc					= ValidateOption[True, SVGetDimensions, CharacterEncoding, Alternatives @@ Join[System`$CharacterEncodings, {None, Automatic, "UTF8ISOLatin1"}], opts];
	ignoreEmptyLines	= ValidateOption[True, SVGetDimensions, "IgnoreEmptyLines", True | False, opts];
	td					= ValidateOption[True, SVGetDimensions, "TextDelimiters", t_String /; 0 <= StringLength[t] <= 1, opts];
	summary				= ValidateOption[True, SVGetDimensions, "Summary", _?BooleanQ, opts];
]

[chere005]

Re 7 I would keep & an operator.. There's some good features in this PR, but a lot to go through all at once. Smaller PRs might make it easier to get things merged since we can't approve it until all changes are OK. I see good work on supporting : as Optional, I haven't had a chance to check this out and try it yet but does it work for recognizing foo : (_Integer | _String) as pattern? I'll try to look at this more carefully in the next couple weeks.

I tested the : as pattern and it works

[chere005]

This is causing MAJOR performance detriments on my machine. I couldn't even open a file with 10k+ lines of code, but when I switched to the plugin on master it loaded instantly.