TypeError reading the WD atmcof*.dat files

[dirkterrell]

Running on CentOS 7, using scl to compile libphoebe (PHOEBE 2.3.3 distribution) with gcc 7 and Anaconda python 3.6.3. For scl specifically, I did:

scl enable devtoolset-7 '/usr/local/anaconda3/bin/pip install -U phoebe'

I downloaded the two atmcof files from the PHOEBE github.com site (using wget) and their md5sums are 1c7c95a0798e13fce199546c08a32796 atmcof.dat 58c0008d286524621072ae4b0d52e4d8 atmcofplanck.dat

Script that generates the TypeError:

[terrell@algolhome ~/phoebe]$ /usr/local/anaconda3/bin/python Python 3.6.3 |Anaconda, Inc.| (default, Oct 13 2017, 12:02:49) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license" for more information.

import libphoebe libphoebe.wd_readdata('/usr/local/anaconda3/lib/python3.6/site-packages/phoebe/atmospheres/tables/wd/atmcofplanck.dat'.encode('utf8'), '/usr/local/anaconda3/lib/python3.6/site-packages/phoebe/atmospheres/tables/wd/atmcof.dat'.encode('utf8')) Traceback (most recent call last): File "", line 1, in TypeError: wd_readdata::Problem reading data

[kecnry]

Any ideas what might be going on or how best to debug this @horvatm?

I also just realized that you can probably get around this in the meantime by requesting a version of the TESS passband without the WD atmospheres (which really only exist for more 1-to-1 comparison with WD and PHOEBE 1, but are not used by default).

Try:

phoebe.download_passband('TESS:T', content=['ck2004', 'blackbody', 'phoenix'])

or download any custom file from http://phoebe-project.org/tables/pbs and replace your locally installed copy. If you exclude extern_atmx and extern_planckint, then phoebe shouldn't even attempt to load those WD tables.

[dirkterrell]

On 10/21/2020 2:21 PM, Kyle Conroy wrote:

I also just realized that you can probably get around this in the meantime by requesting a version of the TESS passband without the WD atmospheres (which really only exist for more 1-to-1 comparison with WD and PHOEBE 1, but are not used by default).

Try:

|phoebe.download_passband('TESS:T', content=['ck2004', 'blackbody', 'phoenix']) |

I gave that a try but got the same result. I deleted the TESS file from the system location to be sure it wasn't using that one. The one that downloaded into my user account area has this in the headers:

CONTENT = '[''ck2004:Inorm'', ''phoenix:ld'', ''phoenix:Inorm'', &'             CONTINUE  '''blackbody:ext'', ''ck2004:ldint'', ''ck2004:ext'', &'              CONTINUE  '''blackbody:Inorm'', ''ck2004:ld'', ''phoenix:ext'', &'              CONTINUE  '''phoenix:ldint'', ''phoenix:Imu'', ''ck2004:Imu'']'                

So, I think that file does indeed not have the WD stuff included. But it is still trying to read the atm*.dat files.

[aprsa]

@dirkterrell, can you please provide a traceback for this:

import phoebe
pb = phoebe.get_passband('TESS:T')

The only wd_data() call I can see is in passbands.py, line 440, and it should never reach it if extern_* is not in contents.

[dirkterrell]

On 10/21/2020 6:54 PM, Andrej Prsa wrote:

|import phoebe pb = phoebe.get_passband('TESS:T') |

The only wd_data() call I can see is in passbands.py, line 440, and it should never reach it if extern_* is not in contents.

I agree. It's weird. If I uncomment that "if" stanza that should keep line 440 from being run, and add

pb = phoebe.get_passband('TESS:T') print(pb)

right before

b.run_compute(compute='fastcompute')

I get:

Passband: TESS:T Version:  1.0 Provides: ['ck2004:Inorm', 'phoenix:ld', 'phoenix:Inorm', 'blackbody:ext', 'ck2004:ldint', 'ck2004:ext', 'blackbody:Inorm', 'ck2004:ld', 'phoenix:ext', 'phoenix:ldint', 'phoenix:Imu', 'ck2004:Imu'] Traceback (most recent call last):   File "./test2.py", line 42, in     b.run_compute(compute='fastcompute')   File "/usr/local/anaconda3/lib/python3.6/site-packages/phoebe/parameters/parameters.py", line 401, in _send_if_client     return fctn(self, *args, **kwargs)   File "/usr/local/anaconda3/lib/python3.6/site-packages/phoebe/frontend/bundle.py", line 10135, in run_compute     system, pblums_abs, pblums_scale, pblums_rel, pbfluxes = self.compute_pblums(compute=compute, ret_structured_dicts=True, skip_checks=True, **{k:v for k,v in kwargs.items() if k in computeparams.qualifiers})   File "/usr/local/anaconda3/lib/python3.6/site-packages/phoebe/frontend/bundle.py", line 9125, in compute_pblums     system = kwargs.get('system', self._compute_intrinsic_system_at_t0(compute=compute, datasets=pblum_datasets, **kwargs))   File "/usr/local/anaconda3/lib/python3.6/site-packages/phoebe/frontend/bundle.py", line 8814, in _compute_intrinsic_system_at_t0     system = backends.PhoebeBackend()._compute_intrinsic_system_at_t0(self, system_compute, datasets=datasets, reset=False, lc_only=False, **kwargs)   File "/usr/local/anaconda3/lib/python3.6/site-packages/phoebe/backend/backends.py", line 892, in _compute_intrinsic_system_at_t0     system.populate_observables(t0, ['lc' for dataset in datasets], datasets, ignore_effects=True)   File "/usr/local/anaconda3/lib/python3.6/site-packages/phoebe/backend/universe.py", line 319, in populate_observables     body.populate_observable(time, kind, dataset, ignore_effects=ignore_effects)   File "/usr/local/anaconda3/lib/python3.6/site-packages/phoebe/backend/universe.py", line 1069, in populate_observable     new_mesh_cols = getattr(self, 'populate{}'.format(kind.lower()))(dataset, ignore_effects=ignore_effects, **kwargs)   File "/usr/local/anaconda3/lib/python3.6/site-packages/phoebe/backend/universe.py", line 1793, in _populate_lc     pb = passbands.get_passband(passband)   File "/usr/local/anaconda3/lib/python3.6/site-packages/phoebe/atmospheres/passbands.py", line 3871, in get_passband     pb = Passband.load(_pbtable[passband]['fname'], load_content=True)   File "/usr/local/anaconda3/lib/python3.6/site-packages/phoebe/atmospheres/passbands.py", line 441, in load     self.wd_data = libphoebe.wd_readdata(planck, atm) TypeError: wd_readdata::Problem reading data

If I comment out the stanza, it runs fine. I can do some more debugging tomorrow and see why that if statement is not working like we expect it to.

Dirk

[horvatm]

This error is returned only in two case:

• if the files are not opened for reading
• if the size of file is not what is should be

@kecnry Should I make the errors more verbose in libphoebe in these routines?

[kecnry]

@dirkterrell - that is definitely odd that get_passband works but run_compute is raising the error. One other possibility is that it is actually the bolometric passband (used for irradiation), not the tess passband, that is actually causing the error. Or it could be Johnson:V since your RV dataset is still using Johnson:V. Can you also try the following:

phoebe.get_passband('Bolometric:900-40000')
phoebe.get_passband('Johnson:V')

The other idea I have is that you may have multiple installed copies of either of these passbands and not consistently accessing the same file. If you either turn on the debugger or add a print(archive) before line 441, you can check the exact passband filename that is causing the issue.

Of course none of this addresses the actual issue of reading the WD data, it only helps us figure out why removing the extern atmosphere tables didn't get around that if statement. It looks like @horvatm added some additional debug statements in the C-code in the development branch (commit 8cc20ca020abc5711c742190a22ede950468524c) if you don't mind trying that and letting us know the errors you get now?

[dirkterrell]

On 10/22/2020 7:11 AM, Kyle Conroy wrote:

@dirkterrell https://github.com/dirkterrell - that is definitely odd that |get_passband| works but |run_compute| is raising the error. One other possibility is that it is actually the bolometric passband (used for irradiation), not the tess passband, that is actually causing the error. Or it could be Johnson:V since your RV dataset is still using Johnson:V. Can you also try the following:

|phoebe.get_passband('Bolometric:900-40000') phoebe.get_passband('Johnson:V')|

|You're onto it. It is the Johnson V passband file that is causing the WD files to be read.

|

||

Of course none of this addresses the /actual/ issue of reading the WD data, it only helps us figure out why removing the extern atmosphere tables didn't get around that if statement. It looks like @horvatm https://github.com/horvatm added some additional debug statements in the C-code in the development branch (commit 8cc20ca https://github.com/phoebe-project/phoebe2/commit/8cc20ca020abc5711c742190a22ede950468524c) if you don't mind trying that and letting us know the errors you get now?

I'll give it a try when I get a chance.

[horvatm]

Please note that all libphoebe routines accept PyBytes as strings in order to have backwards compatibility with python2.x. The reading of files should be successful is if filenames are converted to bytes. See the example below:

import libphoebe

s1='<phoebe_install_dir>/atmospheres/tables/wd/atmcofplanck.dat'
s2='<phoebe_install_dir>/atmospheres/tables/wd/atmcof.dat'

#libphoebe.wd_readdata(s1, s2)                #ERROR: TypeError: wd_readdata::Problem reading arguments
#libphoebe.wd_readdata(bytes(s1), bytes(s2))  #ERROR: TypeError: string argument without an encoding
libphoebe.wd_readdata(bytes(s1, encoding='utf8'), bytes(s2, encoding='utf8'))   # NO ERROR

[horvatm]

Added additional check if the file we want to open even exist in 59da18b .