improve cli query error message or support numpy inputs

[cyrush]

Describe the bug

User passed a numpy array as an input to Query and received a cryptic error message about not being able to parse arguments. (Also, it didn't mention which if several argument was the issue)

To Reproduce

Steps to reproduce the behavior. For example:

Pass a numpy array to Query.

Expected behavior

Better error message (which arg is a problem) or maybe just accept numpy arrays since we are building with numpy support everywhere now.

[brugger1]

Since these are small arrays, we can just convert them when we see them. A separate issue is that the error message was poor, we should do a better job of that as well.

[brugger1]

Cyrus believes this was with the XRay Image query but it would be an issue with any query that takes a python list or tuple. We can probably demonstrate this with a simpler example than an XRay Image query.

[markcmiller86]

@brugger1 when I try to duplicate code in xrayimage.py test like this...

# old style argument passing
Query("XRay Image", 2, ".", 1, 0.0, 2.5, 10.0, 0, 0, 10., 10., 300, 300, ("d", "p"))

I get an error...

>>> Query("XRay Image", 2, ".", 1, 0.0, 2.5, 10.0, 0, 0, 10., 10., 300, 300, ("d", "p"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: function takes at most 2 arguments (14 given)

How does this work in test suite?

[markcmiller86]

@cyrush I think I am getting the cryptic error message you mention here. Does this look like what you mean...

>>> params = dict(output_type=2, output_dir=".", divide_emis_by_absorb=1, origin=(0.0, 2.5, 10.0), up_vector=(0, 1, 0), theta=0, phi=0, width = 10., height=10., image_size=(300, 300), vars=("da", "pa"))
>>> a1D = numpy.array([0.0, 2.5, 10.0])
>>> params = dict(output_type=2, output_dir=".", divide_emis_by_absorb=1, origin=a1D, up_vector=(0, 1, 0), theta=0, phi=0, width = 10., height=10., image_size=(300, 300), vars=("da", "pa"))
>>> Query("XRayImage",params)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
visit.VisItException: Query:  could not parse dictionary argument.

[markcmiller86]

If above investigation is correct, the related code is here...

https://github.com/visit-dav/visit/blob/ccfa916ccae40ea12b11dbbde7aa0bca61d90755/src/visitpy/common/visitmodule.C#L12212-L12223

So, what is failing is the PyDict_To_MapNode() call and there isn't any more information coming back about why it is failing. I suppose we could enhance PyDict_To_MapNode to identify the name of the first dict member causing problems and then we could include that in the error message string.

[markcmiller86]

So, to fix this, I would propose the following...

1. Add a return arg to PyDict_To_MapNode which is either a string ref or string pointer named errmsg and is by default void/null.
2. If errmsg is non-void/null, populate it with same/similar string we send to debug logs here... https://github.com/visit-dav/visit/blob/d1fbae6caeb3364954f3c2b54eb0a27ae69fd889/src/visitpy/common/PyMapNode.C#L253-L259
3. Use this feature to gather an error message we report in the CLI

[JustinPrivitera]

How does this work in test suite?

That's odd... I've been able to call the x ray query from the CLI with all the arguments. I'm not sure why that's happening for you.

[brugger1]

Works for me as well.

visit -v 3.3.0 -cli -nowin
Running: cli3.3.0 -nowin -forceversion 3.3.0
Running: viewer3.3.0 -nowin -forceversion 3.3.0 -noint -host 127.0.0.1 -port 5600
VisIt: Warning - LabelAttributes::textColor1 is deprecated and will be removed in version 3.3.2. Please use textFont1.color instead. Please update your session and/or config files.
VisIt: Warning - LabelAttributes::textColor2 is deprecated and will be removed in version 3.3.2. Please use textFont2.color instead. Please update your session and/or config files.
>>> OpenDatabase("/usr/gapps/visit/data/curv3d.silo")
Running: mdserver3.3.0 -forceversion 3.3.0 -host 127.0.0.1 -port 5600
Running: engine_ser3.3.0 -forceversion 3.3.0 -dir /usr/gapps/visit -idle-timeout 480 -host 127.0.0.1 -port 5600
1
>>> AddPlot("Pseudocolor", "d")                
1
>>> DrawPlots()
1
>>> Query("XRay Image", 2, ".", 1, 0.0, 2.5, 10.0, 0, 0, 10., 10., 300, 300, ("d", "p"))
'The x ray image query results were written to the file output00.png\n'
>>> quit()

[brugger1]

You're solution sounds reasonable to me.

[markcmiller86]

Well, I guess it would help if I was using correct version of VisIt. It does work on develop. It was failing on 3.2.2.

By it, I mean the old way of querying.

[brugger1]

Yes, it sure does fail with 3.2.2. I tried old versions going back to 2.13.3 and all failed. I guess I'm also confused by that as well. Maybe the old style was slightly different and the error message is incorrect, since I'm pretty sure you could give it a list of arguments.

[markcmiller86]

You're solution sounds reasonable to me.

Ok, I will proceed with that.

[markcmiller86]

Ok, with that change, here is what we see for the same error...

import numpy

foo = numpy.array([0.0, 2.5, 10.0])

OpenDatabase("/Users/miller86/visit/visit/data/silo_hdf5_test_data/curv3d.silo")
AddPlot("Pseudocolor","d")
DrawPlots()
DefineScalarExpression("d1", 'recenter(d, "zonal")')
DefineScalarExpression("p1", 'recenter(p, "zonal")')
DefineArrayExpression("da", "array_compose(d1,d1)")
DefineArrayExpression("pa", "array_compose(p1,p1)")
params = dict(output_type=2, output_dir=".", divide_emis_by_absorb=1, origin=foo, up_vector=(0, 1, 0), theta=0, phi=0, width = 10., height=10., image_size=(300, 300), vars=("da", "pa"))
Query("XRay Image", params)
Traceback (most recent call last):
  File "foo.py", line 13, in <module>
    Query("XRay Image", params)
visit.VisItException: PyDict_To_MapNode: argument named "origin" at position 4 with type numpy.ndarray not currently implemented.

[markcmiller86]

@cyrush and @brugger1, surprisingly, it looks like assigning numpy arrays to other VisIt attribute members works fine including even error checking they are the correct length...

>>> ca = CylinderAttributes()
>>> print(ca)
point1 = (0, 0, 0)
point2 = (1, 0, 0)
radius = 1
inverse = 0

>>> p1 = numpy.array([1,1,1])
>>> ca.point1 = p1
>>> print(ca)
point1 = (1, 1, 1)
point2 = (1, 0, 0)
radius = 1
inverse = 0

>>> p1 = numpy.array([1,1,1,1])
>>> ca.point1 = p1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Expecting 3 numeric args

So, this begs the question...why don't they work in this Query interface? I suspect it has to do with expected way in which args are wrapped by CLI before arriving in our C/C++ modules.

[brugger1]

That error message looks great!!