M2 icon indicating copy to clipboard operation
M2 copied to clipboard

locate and code for functions involving options, caching, or @@ are useless

Open mahrud opened this issue 3 years ago • 20 comments

Say I want to see where net Variety is defined, so I use locate:

i5 : locate(net, Variety)

o5 = (../../Macaulay2/m2/classes.m2, 90, 47, 90, 54, 90, 50)

But that doesn't make sense, so instead I look up its code:

i8 : code(net, Variety)

o8 = -- code for method: net(Variety)
     ../../Macaulay2/m2/classes.m2:90:48-90:55: --source code:
     Function @@ Function := Function => (f,g) -> x -> f g x
     | symbol   class                   value      location of symbol
     | ------   -----                   -----      ------------------                       
     | f      : MethodFunctionSingle -- net        ../../Macaulay2/m2/classes.m2:90:38-90:39
     | g      : MethodFunctionSingle -- expression ../../Macaulay2/m2/classes.m2:90:40-90:41
     | -- function f:
     | function net: source code not available
     | -- function g:
     | function expression: source code not available

But this is useless: nowhere in this output does it say that net Variety is defined on line 24 of varieties.m2, or even what net and expression do or even where they are defined -- it just says the are also in classes.m2! Even the most useful part of this output, which says that net Variety is the composition of net and expression, is written in too much generality and would be much easier to understand if it just said something like x -> net expression x.

To a smaller extent, the same is true for functions involving options:

i17 : locate(codim, ProjectiveVariety)

o17 = (../../Macaulay2/m2/option.m2, 15, 19, 17, 33, 17, 7)

o17 : Sequence

i18 : code(codim, ProjectiveVariety)

o18 = -- code for method: codim(ProjectiveVariety)
      ../../Macaulay2/m2/option.m2:15:20-19:34: --source code:
        (opts,f) -> args -> (
             -- Common code for functions created with >> to process options and arguments.
             uncurry(f, override (opts,args))
             )
        )
      | symbol   class              value                         location of symbol
      | ------   -----              -----                         ------------------                     
      | f      : FunctionClosure -- ...                           ../../Macaulay2/m2/option.m2:15:9-15:10
      | opts   : OptionTable     -- OptionTable{Generic => false} ../../Macaulay2/m2/option.m2:15:4-15:8 
      | -- function f:
      | ../../Macaulay2/m2/varieties.m2:299:66-299:87: --source code:
      | codim ProjectiveVariety := options(codim,PolynomialRing) >> opts -> X -> codim(ring X,opts)
      | -- option table opts:
      | OptionTable{Generic => false}

and those involving caching:

i36 : locate(poincare,MonomialIdeal)

o36 = (../../Macaulay2/m2/methods.m2, 654, 51, 663, 25, 655, 56)

o36 : Sequence

i37 : code(poincare, MonomialIdeal)

o37 = -- code for method: poincare(MonomialIdeal)
      ../../Macaulay2/m2/methods.m2:654:52-663:26: --source code:
      cacheValue = key -> f -> new CacheFunction from (x -> (
                c := try x.cache else x.cache = new CacheTable;
                if c#?key then (
                     val := c#key;
                     if class val === CacheFunction then (
                          remove(c,key);
                          c#key = val x)
                     else val
                     )
                else c#key = f x))
      | symbol   class              value    location of symbol
      | ------   -----              -----    ------------------                         
      | f      : FunctionClosure -- ...      ../../Macaulay2/m2/methods.m2:654:21-654:22
      | key    : Symbol          -- poincare ../../Macaulay2/m2/methods.m2:654:14-654:17
      | -- function f:
      | ../../Macaulay2/m2/monideal.m2:98:59-98:123: --source code:
      | poincare MonomialIdeal := (cacheValue symbol poincare) (M -> new degreesRing M from rawHilbert rawMonomialIdealToMatrix M.RawMonomialIdeal)

Together, this makes the output of code methods annoyingly long.

mahrud avatar Aug 17 '22 20:08 mahrud

Oh my goodness this is the worst:

i1 : code(options, PolynomialRing)

o1 = -- code for method: options(PolynomialRing)
     ../linuxbrew/.linuxbrew/share/Macaulay2/Core/classes.m2:90:48-90:55: --source code:
     Function @@ Function := Function => (f,g) -> x -> f g x
     | symbol   class                   value   location of symbol
     | ------   -----                   -----   ------------------                                                 
     | f      : MethodFunctionSingle -- options ../linuxbrew/.linuxbrew/share/Macaulay2/Core/classes.m2:90:38-90:39
     | g      : MethodFunctionSingle -- monoid  ../linuxbrew/.linuxbrew/share/Macaulay2/Core/classes.m2:90:40-90:41
     | -- function f:
     | function options: source code not available
     | -- function g:
     | ../linuxbrew/.linuxbrew/share/Macaulay2/Core/option.m2:15:20-19:34: --source code:
     |   (opts,f) -> args -> (
     |        -- Common code for functions created with >> to process options and arguments.
     |        uncurry(f, override (opts,args))
     |        )
     |   )
     | | symbol   class              value                                                   location of symbol
     | | ------   -----              -----                                                   ------------------                                               
     | | f      : FunctionClosure -- ...                                                     ../linuxbrew/.linuxbrew/share/Macaulay2/Core/option.m2:15:9-15:10
     | | opts   : OptionTable     -- OptionTable{Constants => false                        } ../linuxbrew/.linuxbrew/share/Macaulay2/Core/option.m2:15:4-15:8 
     | |                                         DegreeLift => null                          
     | |                                         DegreeMap => null                           
     | |                             ....................................................... 
     | | -- function f:
     | | ../linuxbrew/.linuxbrew/share/Macaulay2/Core/methods.m2:144:33-146:100: --source code:
     | |     methodFunction := opts >> o -> arg -> (
     | |         -- Common code for every method with options and a single argument
     | |         singleCaller(methodFunction, (methodFunction, dispatchBy arg), arg, outputs, dispatcher(o, arg)));
     | | | symbol           class                   value                                                   location of symbol
     | | | ------           -----                   -----                                                   ------------------                                                   
     | | | opts           : OptionTable          -- OptionTable{Constants => false                        } ../linuxbrew/.linuxbrew/share/Macaulay2/Core/methods.m2:136:23-136:27
     | | |                                                      DegreeLift => null                          
     | | |                                                      DegreeMap => null                           
     | | |                                          ....................................................... 
     | | | dispatchBy     : CompiledFunction     -- class                                                   ../linuxbrew/.linuxbrew/share/Macaulay2/Core/methods.m2:140:5-140:15 
     | | | dispatcher     : FunctionClosure      -- ...                                                     ../linuxbrew/.linuxbrew/share/Macaulay2/Core/methods.m2:141:5-141:15 
     | | | outputs        : Boolean              -- false                                                   ../linuxbrew/.linuxbrew/share/Macaulay2/Core/methods.m2:136:29-136:36
     | | | methodFunction : MethodFunctionSingle -- monoid                                                  ../linuxbrew/.linuxbrew/share/Macaulay2/Core/methods.m2:144:5-144:19 
     | | -- option table opts:
     | | OptionTable{Constants => false                        }
     | |             DegreeLift => null
     | |             DegreeMap => null
     | |             DegreeRank => null
     | |             Degrees => null
     | |             Global => true
     | |             Heft => null
     | |             Inverses => false
     | |             Join => null
     | |             Local => false
     | |             MonomialOrder => {GRevLex, Position => Up}
     | |             MonomialSize => 32
     | |             SkewCommutative => {}
     | |             VariableBaseName => p
     | |             Variables => null
     | |             Weights => {}
     | |             WeylAlgebra => {}

I have no idea what prompted all of this! Here is the actual code in ofcm.m2

options PolynomialRing := options @@ monoid

Nowhere in all of that text is ofcm.m2 mentioned.

mahrud avatar Aug 18 '22 04:08 mahrud

The point is that in the line options PolynomialRing := options @@ monoid, the code to the right of the := is executed, not saved for later execution. If you'd like it to be clearer, just change it to options PolynomialRing := R -> options monoid R.

DanGrayson avatar Aug 18 '22 10:08 DanGrayson

That has nothing to do with locate not showing where (options, Polynomial) is set to the value of whatever the right hand side evaluates to, or code showing all that extra stuff.

mahrud avatar Aug 18 '22 13:08 mahrud

No, locate operates on code stored in function bodies -- that's where location information is stored.

DanGrayson avatar Aug 18 '22 14:08 DanGrayson

Okay ... sounds like you agree with what I'm claiming in this issue, then? That "locate and code for functions involving options, caching, or @@ are useless"?

mahrud avatar Aug 18 '22 14:08 mahrud

Is there any easy way we can get these to work (and not be 'useless')?

mikestillman avatar Aug 18 '22 14:08 mikestillman

Probably the file/line location of the actual method setting should be given instead of where the function is defined?

mikestillman avatar Aug 18 '22 14:08 mikestillman

If the execution stack was somehow accessible (which is another feature request on its own), then I think the easiest way would be for Function @@ Function to store location of the one before the last value in the execution stack, which would be the line that called Function @@ Function. If the location is stored in the function frame then it could be accessed later.

mahrud avatar Aug 18 '22 14:08 mahrud

options PolynomialRing := options @@ monoid

Yes, I agree that you cannot find the line options PolynomialRing := options @@ monoid by using locate. However, you could change that line to options PolynomialRing := R -> options monoid R.

I find it easy to use emacs' tags search function to find the line options PolynomialRing := options @@ monoid in the code.

DanGrayson avatar Aug 18 '22 14:08 DanGrayson

So you're suggesting just getting rid of @@ all over the place? That seems like erasing the problem statement to me.

Hooks are also kind of opaque, you can't tell what's in the code from code(length, Module), for instance, but if you actually want to make it useful (instead of just saying "well don't use hooks" or "use emacs") then there's a solution:

i1 : code(length, Module)

o1 = -- code for method: length(Module)
     ../linuxbrew/.linuxbrew/share/Macaulay2/Core/hilbert.m2:148:26-151:89: --source code:
     length Module := ZZ => M -> (
         computation := (cacheValue symbol length) (M -> runHooks((length, Module), M));
         if (n := computation M) =!= null then return n;
         error("no applicable strategy for computing length of modules over ", toString ring M))

i2 : code hooks(length, Module)

o2 = -- code for method: length(Module)
     ../linuxbrew/.linuxbrew/share/Macaulay2/Core/hilbert.m2:153:50-155:44: --source code:
     addHook((length, Module), Strategy => Default, M -> (
         if not isHomogeneous M then notImplemented();
         if dim M > 0 then infinity else degree M))

mahrud avatar Aug 18 '22 14:08 mahrud

So you're suggesting just getting rid of @@ all over the place? That seems like erasing the problem statement to me.

This is just like the code x = 2 + 2. If later on, you have that number 4, there is no way to use it to find out the location of the code 2 + 2 that resulted in it.

DanGrayson avatar Aug 19 '22 13:08 DanGrayson

This is just like the code x = 2 + 2. If later on, you have that number 4, there is no way to use it to find out the location of the code 2 + 2 that resulted in it.

  1. How is this relevant? I'm talking about method functions, not symbols.
  2. You can still find where the symbol x was created with locate x.

I'm not surprised that you disagree with my improvement suggestions, but you don't need to keep insisting.

mahrud avatar Aug 19 '22 14:08 mahrud

This is just like the code x = 2 + 2. If later on, you have that number 4, there is no way to use it to find out the location of the code 2 + 2 that resulted in it.

  1. How is this relevant? I'm talking about method functions, not symbols.

It's relevant because both x = 2 + 2 and options PolynomialRing := options @@ monoid are assignment statements. In both of them, the right hand side is evaluated, and the value is stored in the location indicated by the left hand side. The location of the code that computes the value is not stored in the value.

There are other ways to produce functions -- not just f @@ g. How could you handle all of them uniformly?

DanGrayson avatar Aug 19 '22 15:08 DanGrayson

Sure, there are many ways to produce functions, but the three classes of functions I'm highlighting in this issue are extremely common, so even ad hoc solutions (e.g. changing Function @@ Function as I suggested above) would go a long way.

But if you must have a general solution: the left hand side of options PolynomialRing := X also evaluates something, and that code can query the current location and store it in the function produced by the right hand side, however it is produced. This wouldn't be too hard with some internal tweaking to make the register storing the location of a function mutable.

mahrud avatar Aug 19 '22 16:08 mahrud

This is just like the code x = 2 + 2. If later on, you have that number 4, there is no way to use it to find out the location of the code 2 + 2 that resulted in it.

I'm reading the bold part again, and it seems even less relevant. In all the cases I highlighted locate and code don't start with a function, they start with a method sequence like (options, Polynomial).

mahrud avatar Aug 19 '22 16:08 mahrud

It seems to me that if one has a method definition, such as

f = method()
f Matrix := g

where g is a function, then code(f, Matrix) (or locate?) could give one of two locations: the location of the line f Matrix := g, or the location of the definition of g. It seems like the former might be more useful information?

mikestillman avatar Aug 19 '22 16:08 mikestillman

I think both should be present in the output of code, at least.

mahrud avatar Aug 19 '22 22:08 mahrud

But if you must have a general solution: the left hand side of options PolynomialRing := X also evaluates something, and that code can query the current location and store it in the function produced by the right hand side, however it is produced. This wouldn't be too hard with some internal tweaking to make the register storing the location of a function mutable.

That might work. But don't store it in the function, as one still wants to be able to find the location of the function itself. Also, the same function might be installed as multiple method functions. So the location information should be stored alongside the function in one of the types on the left hand side.

DanGrayson avatar Aug 20 '22 11:08 DanGrayson

Here's another case where code not only has a lot of random output but is literally useless: I noticed there's a method Constant ^ Ring, which is not documented and somehow has options (but how do you even pass options to this?!) so I wanted to look at the code, but this is what I got:

i1 : code(symbol^, Constant, Ring)

o1 = -- code for method: Constant ^ Ring
     /home/linuxbrew/.linuxbrew/share/Macaulay2/Core/option.m2:15:19-17:33: --source code:
       (opts,f) -> args -> (
            -- Common code for functions created with >> to process options and arguments.
            uncurry(f, override (opts,args))
            )
       )
     | symbol  class            value                                                    location of symbol
     | ------  -----            -----                                                    ------------------                                                 
     | f       FunctionClosure  FunctionClosure[/home/linuxbrew/.linuxbrew/share/Macau.  /home/linuxbrew/.linuxbrew/share/Macaulay2/Core/option.m2:15:8-15:9
     | opts    OptionTable      OptionTable{Verify => true}                              /home/linuxbrew/.linuxbrew/share/Macaulay2/Core/option.m2:15:3-15:7
     | -- function f:
     | /home/linuxbrew/.linuxbrew/share/Macaulay2/Core/methods.m2:154:69-154:101: --source code:
     |      methodFunction := new MethodFunctionWithOptions from (opts >> o -> arg -> innerMethodFunction(o,arg));
     | | symbol               class                      value                                         location of symbol
     | | ------               -----                      -----                                         ------------------                                                      
     | | opts                 OptionTable                OptionTable{Verify => true}                   /home/linuxbrew/.linuxbrew/share/Macaulay2/Core/methods.m2:151:37-151:41
     | | innerMethodFunction  CompiledFunctionClosure    CompiledFunctionClosure[]                     /home/linuxbrew/.linuxbrew/share/Macaulay2/Core/methods.m2:153:11-153:30
     | | outputs              List                       {false, true, true}                           /home/linuxbrew/.linuxbrew/share/Macaulay2/Core/methods.m2:151:42-151:49
     | | methopts             OptionTable                OptionTable{Binary => false                }  /home/linuxbrew/.linuxbrew/share/Macaulay2/Core/methods.m2:151:28-151:36
     | |                                                             Dispatch => {Thing, Type, Type}   
     | |                                                             Options => {Verify => true}       
     | |                                                             TypicalValue => Thing             
     | | methodFunction       MethodFunctionWithOptions  lift                                          /home/linuxbrew/.linuxbrew/share/Macaulay2/Core/methods.m2:154:5-154:19 
     | -- option table opts:
     | OptionTable{Verify => true}

This gives me zero information, other than a mysterious lift near the end. Even the place where f is defined is obscured:

     | f       FunctionClosure  FunctionClosure[/home/linuxbrew/.linuxbrew/share/Macau.  /home/linuxbrew/.linuxbrew/share/Macaulay2/Core/option.m2:15:8-15:9

mahrud avatar May 08 '24 18:05 mahrud