DIM_OF gives different results on D3D when used as member function call and inside MDS expression

[rkube]

Hi, I'm using MDSplus to fetch data from D3D and have a question about the python API. When I call dim_of to get the time-base for a node the returned array is correct only when using dim_of inside the MDSplus expression. When calling the member function .dim_of() , MDS returns an integer array:

import MDSplus as mds

c = mds.Connection("atlas.gat.com")
c.openTree("D3D", 178631)
xdata = c.get("NB:PINJ").dim_of().data()

xdata2 = c.get("DIM_OF(NB:PINJ)").data()

print("xdata = ", xdata)
print("xdata2 = ", xdata2)
[kuber@irisb source]$ python mdstest.py 
xdata =  [     0      1      2 ... 130998 130999 131000]
xdata2 =  [0.00000e+00 1.00000e-01 2.00000e-01 ... 1.30998e+04 1.30999e+04
 1.31000e+04]

Is this expected? And is it possible to use .dim_of() to get the timebase?

[joshStillerman]

can somone explain to Ralph about connection objects and tree and node objects. There is the MdsConnector remote python objects API

-Josh

On Oct 13, 2022, at 1:31 PM, Ralph Kube @.***> wrote:

Hi, I'm using MDSplus to fetch data from D3D and have a question about the python API. When I call dim_of to get the time-base for a node the returned array is correct only when using dim_of inside the MDSplus expression. When calling the member function .dim_of() , MDS returns an integer array:

import MDSplus as mds

c = mds.Connection("atlas.gat.com") c.openTree("D3D", 178631) xdata = c.get("NB:PINJ").dim_of().data()

xdata2 = c.get("DIM_OF(NB:PINJ)").data()

print("xdata = ", xdata) print("xdata2 = ", xdata2) @.*** source]$ python mdstest.py xdata = [ 0 1 2 ... 130998 130999 131000] xdata2 = [0.00000e+00 1.00000e-01 2.00000e-01 ... 1.30998e+04 1.30999e+04 1.31000e+04] Is this expected? And is it possible to use .dim_of() to get the timebase?

— Reply to this email directly, view it on GitHub https://github.com/MDSplus/mdsplus/issues/2493, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABY5AZIYIKMABGYCQ6VZE43WDBBO7ANCNFSM6AAAAAAREP7X6U. You are receiving this because you are subscribed to this thread.

[joshStillerman]

Ralph -

The behavior you are getting is expected. Let me try to explain…

The MDSplus.Connection object provides get and put methods. These are equivalent to the IDL/MATLAB MdsValue, MdsPut functions.
Connection.get takes an expression (and arguments for that expression?) as an argument and returns native python/numpy data. When you ‘get’ a node, you are implicitly getting the data of the node. Since that data is a simple scalar or array it does not have the node methods like dim_of(), path, fullpath, etc… If you want the dim of the node you have to use TDI syntax inside of the expression you provide:

c.get(“DIM_OF(NB:PINJ)“)

This will (like get of the node) return a native python type Connection.put takes a node, an expression to write to the node, and arguments (python/numpy native data) to substitute into the expression. You can use the connection object locally at D3D by connecting to ‘localhost’ or possibly even ‘local’. Though then you do not have access to the nice object oriented API that native connections have

There is a python library called MDSConnector, that allows for remote python objects. I provide an example below. It has a few caveats though: It is slow to start up since it sends a server program to the server, fires it up, and then connects to it. It is a bit finicky to install since it relies on SSH and SSH’s authentication methods. It is a bit finicky to install, we have had trouble with non matching python versions on the server and client It has a few great advantages: It does not require MDSplus installed on the client computer Only the final answers to your questions are transmitted over the network Example on ubuntu 18 with python2:

jas@mfews04:~$ python
Python 2.7.17 (default, Nov 28 2022, 18:51:39) 
[GCC 7.5.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import mdsconnector
>>> c = mdsconnector.mdsConnector('alcdata')
>>> t = c.Tree('cmod', 1090909009)
>>> ip = t._IP
>>> ip
\MAGNETICS::IP
>>> y = ip.data()
>>> x = ip.dim_of().data()
>>> y.shape
(13312,)
>>> x.shape
(13312,)
>>> 

on Ubuntu18 with python3:

jas@mfews04:~$ pip3 install mdsconnector --user
/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
Collecting mdsconnector
  Using cached mdsconnector-1.7-py3-none-any.whl (13 kB)
Requirement already satisfied: rpyc in ./.local/lib/python3.6/site-packages (from mdsconnector) (4.1.5)
Requirement already satisfied: dill in ./.local/lib/python3.6/site-packages (from mdsconnector) (0.3.2)
Requirement already satisfied: plumbum in ./.local/lib/python3.6/site-packages (from rpyc->mdsconnector) (1.6.9)
Installing collected packages: mdsconnector
Successfully installed mdsconnector-1.7
jas@mfews04:~$ python3
Python 3.6.9 (default, Nov 25 2022, 14:10:45) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from mdsconnector import mdsConnector
>>> c = mdsConnector('alcdata')
>>> t = c.Tree('cmod', 1090909009)
>>> t
Tree("CMOD",1090909009,"Normal")
>>> ip = t._IP
>>> ip
\MAGNETICS::IP
>>> y = ip.data()
>>> x = ip.dim_of().data()
>>> x.shape
(13312,)
>>> y.shape
(13312,)
>>> 

HOWEVER I could NOT get this to work on my mac or on UBUNTU20. It will probably have to be recoded for modern python.

If this is not all clear, or you need further help, please re-open this ticket, create a new one, or set up a call with us.

-Josh

[joshStillerman]

Actually I got mdsConnector to work with modern python:

With particular versions of rpyc, dill and plumbum it does work! Remove them with pip and reinstall:

$ pip uninstall rpyc dill plumbum mdsconnector

$ pip install --user rpyc==4.1.5 dill==0.3.2 plumbum==1.6.9 mdsconnector

I will look into:

• changing the requirements to include versions
• updating the code to work with current versions of those libraries
  • This 2nd step is a bit difficult

Let us know if setting the versions of the dependent packages works for you

[joshStillerman]

The new version of mdsconnector (1.8.0) should install and work with modern python. Let us know if you still have problems...