General::noopen even if $Path is set

[tueda]

Thanks to #1229, $Path is now modifiable. But I was wondering if changing $Path is reflected in the Get command.

Suppose I have lib/Lib.m, whose file content is

42

Then, with wolframscript,

AppendTo[$Path, FileNameJoin[{Directory[], "lib"}]];
Print[<< Lib`];

or simply

AppendTo[$Path, "lib"];
Print[<< Lib`];

prints the number in lib/Lib.m:

42

But with mathicsscript, both give

General::noopen: Cannot open Lib`.

Version:

Mathicscript: 2.1.2, Mathics 2.1.0
on CPython 3.9.4 (default, Apr  7 2021, 05:45:11)
using SymPy 1.7.1, mpmath 1.2.1, numpy 1.20.2

[rocky]

There are probably a couple of things here. The first bug I'd see is that Get[] isn't looking at $Path for relative files. In particular:

Print[<< "Lib.m"];

isn't resolving the full path for Lib.m.

And the second problem (which I'd look at after fixing the first one), is that "Lib`" is supposed to be the same thing as "Lib.m" in this context.

As for fixing, I will defer fixing the first problem which seems pretty simple until after https://github.com/mathics/Mathics/pull/1338 gets merged into master, because the file where this bug appears is in that and this will move its location.

The other thing that I would suggest in issues is to give why it might be useful to fix this bug over the swarm of other bugs that are around. If this is part of some open source package that is of interest to general users, then the priority of addressing this is increased as far as I my time goes. If on the other hand this is just something you casually noticed and thought you'd report, thanks for reporting it and it may help others to be aware that there is a problem. But as far as addressing this, at least as far as my time goes, this is much less important.

On the other hand if you want to fix, then describing why it might be important to fix is mute. We welcome contributors and would like to see others contribute more.

[tueda]

Thank you for looking into this and giving the explanation.

Here I describe some background of when I hit this issue.

Actually, what I casually tried with Mathics is FeynRules, whose latest major-version paper got cited about 1600 times, so it is fair to say that this software is well known in the particle physics community (phenomenology people and some experimentalists). Currently, the latest version FeynRules 2.3.43 does not work with Mathematica 12.2. So I just tried Mathics although I expected there would be many unimplemented features and probably I have to switch back to Mathematica 12.1 (which worked).

After downloading and extracting the FeynRules distribution

wget http://feynrules.irmp.ucl.ac.be/downloads/feynrules-current.tar.gz  # they use a rolling release
tar -x -f feynrules-current.tar.gz

I would run a simple example

$oldPath = Directory[]
$FeynRulesPath = SetDirectory["/path/to/feynrules-current"]
<< FeynRules`
SetDirectory[FileNameJoin[{$FeynRulesPath, "Models", "SM"}]]
LoadModel["SM.fr"]
LoadRestriction["Massless.rst", "DiagonalCKM.rst"]
SetDirectory[$oldPath]
WriteUFO[LSM]

which generates Python files in a subdirectory, Standard_Model_UFO.

The first thing I encountered with Mathics and FeynRules was a syntax error due to missing implementation of \!\(...\), which has already been seen at the end of the log of https://github.com/mathics/Mathics/issues/1174#issuecomment-806279917, around FeynRulesPackage.m:589:

\!\(TraditionalForm\`PYReorderParticles\)::usage = ...
\!\(TraditionalForm\`PYOrderFermions\)::usage = ...

Because they are just usage messages, I can just comment them out, I think.

Then, I needed to give MATHICS_MAX_RECURSION_DEPTH=2500, ignored failures to overwrite C::usage and D::usage.

The next thing I hit was the General::noopen error, around FeynRulesPackage.m:1849:

Block[{$Path = {$LocalFeynRulesPath,
                StringJoin[$LocalFeynRulesPath, "/Core"],
                StringJoin[$LocalFeynRulesPath, "/Interfaces"],
                StringJoin[$LocalFeynRulesPath, "/Interfaces/UFO"]}
       },

    << "FRFormat.m";
    << "PythonForm.m";

    ...

];

which was the reason why I got into this issue, and I tried to make a minimal reproducible example, leading to the first post.

[rocky]

@tueda Thanks for the explanation and context. This helps a lot.

As I have said probably too many times, we need help; and help can come in all varieties. For example, good bug reporting is help.

Another thing we could use help on breaking down the many bugs encountered in trying to get large packages (like this one) to run, is isolating problems into small reportarable bug reports that we can more easily fix. You have already started doing this, so keep up the great work!

I think it fair to say that the two of us most actively working on this would love to see more packages like this working.

Here is an approach that is the most satisfiying and shows the most benefit to people wanting to use this. If there is a way to start with a very very simple FeynRules, get that working and then expand that, then people benefit as the bugs in Mathics get fixed.

One technique I have found that helps is to start with the older version of the package - we are pretty good at supporting the v6 subset of Mathematica. Then as those bugs are fixed move forward in versions.

I did this with for example Combinatorica and was able to get at least something working and can work on this in a way where I can directly see the results and users can benefit.

Of course another way if you know a package very well and it is very modular, is to start with a small core part and then expand add "modules". For example in theory Rubi, a package for doing Rule-based Integration is like that. One could start out with a small number of rules for integration and expand that. In practice though we haven't been able to get sufficient help along these lines, so the "start with an old package" more or less approximates this approach.

Also, when thinking about how to fix obstacles making a package work, another thing that can be done is to try rewrite the code (temporarily) when that can be done. For example I have a fix now for the problem of not tracking changes to $Path which is necessary when Get is called. Get still now only handles accepting strings. So if it is possible to rewrite a version of FeyRules that uses only Get with a string, that would move us forward, and we could address those issues where we can't workaround first and then come back to things like this.

Again, thanks in advance for the help.

[rocky]

Note that I have created the FeynRules tag so we can keep track of which bugs address which packages.