bccp
ClassBadValue Error with Cosmology
So, I'm having trouble with a very simple task. Trying to do:
cosmo = cosmology.Cosmology(h=0.676).match(Omega0_m=0.31)
Yields this error:
ClassBadValueError Traceback (most recent call last)
in () ----> 1 cosmo = cosmology.Cosmology(h=0.676).match(Omega0_m=0.31) /home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/cosmology>/cosmology.pyc in match(self, sigma8, Omega0_cb, Omega0_m) 287 288 if Omega0_m is not None: --> 289 Omega0_cb = Omega0_m - (self.Omega0_ncdm_tot - self.Omega0_pncdm_tot) - >self.Omega0_dcdm 290 return self.match(Omega0_cb=Omega0_cb) 291
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/cosmology/cosmology.pyc in getattr(self, name) 222 if hasattr(iface, name): 223 if iface not in self.delegates: --> 224 self.delegates[iface] = iface(self.engine) 225 d = self.delegates[iface] 226 return getattr(d, name)
classylss/binding.pyx in classylss.binding.Background.init()
classylss/binding.pyx in classylss.binding.ClassEngine.compute()
ClassBadValueError: background_init(L:657) :error in background_solve(ppr,pba); =>background_solve(L:1550) :condition ((tau_end-tau_start)/tau_start < ppr->smallest_allowed_variation) is true; integration step: relative change in time =-nan < machine precision : leads either to numerical error or infinite loop
I also get the same error importing a cosmology from Astropy and then trying to calculate the 2PCF with SurveyData2PCF. Any ideas as to why this is happening?
The test case is clear.
I tried but could not reproduce this with classylss 0.2.8, on Python 3.6.
Which version of classylss are you using?
conda list classylss
import classylss; print(classylss.__version__)
This is what I get with conda list classylss:
classylss 0.2.8 py27h6091dcd_6 bccp
I got a 2.7 environment up and did
conda update -c bccp --all
Then run
pyt(2.7) [yfeng1@waterfall fastpm]$ python
Python 2.7.15 |Anaconda, Inc.| (default, Oct 23 2018, 18:31:10)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from nbodykit import cosmology
>>> cosmo = cosmology.Cosmology(h=0.676).match(Omega0_m=0.31)
>>>
Could it be possible you had local you see if
conda install -f -c bccp classylss
helps?
Did you set any special CLASS environment variables?
Doing conda update -c bccp --all worked.
But now I get another error when runing SurveyData2PCF. I'm trying to make the code reproducible so I can post it here, but in the meantime I'll post the error:
RuntimeError: calling the function 'Corrfunc.mocks.DDsmu_mocks.DDsmu_mocks' failed, likely due to issues with input data/parameters. Open at issue at https://github.com/bccp/nbodykit/issues for further help. exception: Could not import the C extension for the on-skypair counter. stdout: stderr:
This looks to be a different problem entirely, should I open another issue?
P.S.: I didn't do conda install -f -c bccp classylss, because it seems the Class issue has been solved, but correct me if I'm assuming wrong.
I found another thread with this problem, https://github.com/bccp/nbodykit/issues/508. The solution was to install gsl=2.4. I did that, and it was fixed.
But now, when running SurveyData2PCF I get an "Illegal instruction" message and automatically exits the IPython terminal.
I'm absolutely clueless as to how to proceed.
There are two paths to proceed, and we shall follow both.
- Can you post your cpuinfo (cat /proc/cpuinfo)?
I strongly suspect it may be because your CPU lacks some instruction sets (e.g. avx) that the binary CorrFunc (highly optimized) you downloaded could have targeted. We can find out this by comparing your cpuinfo with the one where the conda package is compiled.
If this is the case then a possible temporary fix is to compile CorrFunc locally by (or a similar command)
pip install -U -I --no-deps CorrFunc
- It could be useful if you post your script and a minimal data set that can trigger the illegal instruction, then I can try to reproduce it locally and find out why.
Ok.
This is what cat /proc/cpuinfo yields. I'm working on a cluster so I chose only one processor to copy here.
processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Xeon(R) CPU E5520 @ 2.27GHz stepping : 5 microcode : 25 cpu MHz : 2262.000 cache size : 8192 KB physical id : 0 siblings : 8 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid bogomips : 4533.31 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management:
This chunk of code should be sufficient:
`import numpy as np from nbodykit.lab import * from astropy.io import ascii from astropy.table import Table
ra = 360.*np.random.random(1000) dec = 90.*np.random.random(1000) z = (.15-.05)*np.random.random(1000)+.05 table = Table(np.column_stack((ra,dec,z)),names=['RA','DEC','Redshift']) data = ArrayCatalog(table,names=['RA','DEC','Redshift'])
ra = 360.*np.random.random(1000) dec = 90.*np.random.random(1000) z = (.15-.05)*np.random.random(1000)+.05 table = Table(np.column_stack((ra,dec,z)),names=['RA','DEC','Redshift']) ran = ArrayCatalog(table,names=['RA','DEC','Redshift'])
cosmo = cosmology.Cosmology(h=0.676).match(Omega0_m=0.31) edges=np.logspace(np.log10(.5),np.log10(12),4)
SurveyData2PCF('1d',data,ran,edges,cosmo=cosmo,show_progress=True)`
I cannot reproduce this locally, but your CPU (a 2009 model) indeed does not have AVX support. Could you try the pip install fix?
I just did. This is what I get
MissingCorrfuncError
Traceback (most recent call last)
in () 19 edges=np.logspace(np.log10(13),np.log10(12),4) 20 ---> 21 SurveyData2PCF('1d',data,ran,edges,cosmo=cosmo,show_progress=True) /home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/paircount_tpcf/tpcf.pyc in init(self, mode, data1, randoms1, edges, cosmo, Nmu, pimax, data2, randoms2, R1R2, ra, dec, redshift, weight, show_progress, **config) 423 424 # and run --> 425 self.run() 426 427 def run(self):
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/paircount_tpcf/tpcf.pyc in run(self) 462 """ 463 # this does most of the work --> 464 BasePairCount2PCF.run(self) 465 466 # compute wp(rp) if we computed xi(rp, pi)
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/paircount_tpcf/tpcf.pyc in run(self) 125 result = LandySzalayEstimator(pair_counter, self.data1, self.data2, 126 self.randoms1, self.randoms2, R1R2=self.R1R2, --> 127 logger=self.logger, **attrs) 128 self.D1D2, self.D1R2, self.D2R1, self.R1R2, self.corr = result 129
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/paircount_tpcf/estimators.pyc in LandySzalayEstimator(pair_counter, data1, data2, randoms1, randoms2, R1R2, logger, **kwargs) 181 logger.info("computing randoms1 - randoms2 pair counts") 182 if not R1R2: --> 183 R1R2 = pair_counter(first=randoms1, second=randoms2, **kwargs) 184 185 # data1 x data2
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/pair_counters/mocksurvey.pyc in init(self, mode, first, edges, cosmo, second, Nmu, pimax, ra, dec, redshift, weight, show_progress, domain_factor, **config) 120 121 # run the algorithm --> 122 self.run() 123 124 def run(self):
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/pair_counters/mocksurvey.pyc in run(self) 172 if attrs['mode'] in ['1d', '2d']: 173 from .corrfunc.mocks import DDsmu_mocks --> 174 func = DDsmu_mocks(attrs['edges'], Nmu, comm=self.comm, show_progress=attrs['show_progress']) 175 176 elif attrs['mode'] == 'projected':
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/pair_counters/corrfunc/mocks.pyc in init(self, edges, Nmu, comm, show_progress) 63 from Corrfunc.mocks import DDsmu_mocks 64 except ImportError: ---> 65 raise MissingCorrfuncError() 66 67 self.Nmu = Nmu
MissingCorrfuncError: use either
conda install -c bccp corrfuncorpip install git+git://github.com/nickhand/Corrfunc
Will doing what the output suggests undo the previous pip installation?
Looks like the installation of corrfunc failed. Could you post the output of the installation?
Doesn't look like it...
Collecting CorrFunc Downloading https://files.pythonhosted.org/packages/3c/17/b79988e20800a981e2ed8459f294e98d31e9da4f1e9caba4af3c6f1c8b96/Corrfunc-2.2.0.tar.gz (26.0MB) 100% |████████████████████████████████| 26.1MB 9.2kB/s Building wheels for collected packages: CorrFunc Running setup.py bdist_wheel for CorrFunc ... done Stored in directory: /home/fdavilakurban/.cache/pip/wheels/a3/78/65/86860e8f143cf9c9d446ecbd7953e984b5b87ca22047184d0b Successfully built CorrFunc Installing collected packages: CorrFunc Successfully installed CorrFunc-2.2.0 You are using pip version 9.0.1, however version 18.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command.
By the way, this is the IPython I'm using:
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15) Type "copyright", "credits" or "license" for more information. IPython 5.3.0 -- An enhanced Interactive Python.
Could you produce a stack trace by running the script with
gdb -ex r -ex bt -ex q -args python crashing-script.py
I believe the corrfunc there should have been compiled with no AVX support. So it is likely something else. The stacktrace will help us to figure this out.
Here's the stack trace:
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-92.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Reading symbols from /home/fdavilakurban/.conda/envs/fdavilakurban/bin/python...done. Starting program: /home/fdavilakurban/.conda/envs/fdavilakurban/bin/python crashing-script.py [Thread debugging using libthread_db enabled] [New Thread 0x7fffef3e6700 (LWP 15274)] [New Thread 0x7fffee9e5700 (LWP 15275)] [New Thread 0x7fffebfe4700 (LWP 15276)] [New Thread 0x7fffe95e3700 (LWP 15277)] [New Thread 0x7fffe6be2700 (LWP 15278)] [New Thread 0x7fffe41e1700 (LWP 15279)] [New Thread 0x7fffe17e0700 (LWP 15280)] [New Thread 0x7fffdeddf700 (LWP 15281)] [New Thread 0x7fffdc3de700 (LWP 15282)] [New Thread 0x7fffd99dd700 (LWP 15283)] [New Thread 0x7fffd6fdc700 (LWP 15284)] [New Thread 0x7fffd45db700 (LWP 15285)] [New Thread 0x7fffd1bda700 (LWP 15286)] [New Thread 0x7fffcf1d9700 (LWP 15287)] [New Thread 0x7fffcc7d8700 (LWP 15288)] [Thread 0x7fffef3e6700 (LWP 15274) exited] [Thread 0x7fffee9e5700 (LWP 15275) exited] [Thread 0x7fffe17e0700 (LWP 15280) exited] [Thread 0x7fffe41e1700 (LWP 15279) exited] [Thread 0x7fffebfe4700 (LWP 15276) exited] [Thread 0x7fffe6be2700 (LWP 15278) exited] [Thread 0x7fffdeddf700 (LWP 15281) exited] [Thread 0x7fffe95e3700 (LWP 15277) exited] [Thread 0x7fffdc3de700 (LWP 15282) exited] [Thread 0x7fffd99dd700 (LWP 15283) exited] [Thread 0x7fffd45db700 (LWP 15285) exited] [Thread 0x7fffd6fdc700 (LWP 15284) exited] [Thread 0x7fffcf1d9700 (LWP 15287) exited] [Thread 0x7fffd1bda700 (LWP 15286) exited] [Thread 0x7fffcc7d8700 (LWP 15288) exited] Detaching after fork from child process 15294. Detaching after fork from child process 15588. [New Thread 0x7fffcc7d8700 (LWP 15591)] [New Thread 0x7fffcf1d9700 (LWP 15592)] [New Thread 0x7fffd1bda700 (LWP 15593)] [New Thread 0x7fffbc1ec700 (LWP 15594)] [New Thread 0x7fffbb7eb700 (LWP 15595)] [New Thread 0x7fffbadea700 (LWP 15596)] [New Thread 0x7fffba3e9700 (LWP 15597)] [New Thread 0x7fffb99e8700 (LWP 15598)] [New Thread 0x7fffb0425700 (LWP 15615)]
Program received signal SIGILL, Illegal instruction.
0x00007fffaf77a3a6 in countpairs_countpairs_s_mu_mocks ()
from /home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/Corrfunc/_countpairs_mocks.so
#0 0x00007fffaf77a3a6 in countpairs_countpairs_s_mu_mocks ()
from /home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/Corrfunc/_countpairs_mocks.so
#1 0x00007ffff7cfdbad in ext_do_call (f=
Inferior 1 [process 15243] will be killed.
Quit anyway? (y or n) y
Figure I will start commenting here. In the crashing-script.py, is it possible to pass the isa='fallback' through to Corrfunc via SurveyData2PCF? Perhaps more of a question for @rainwoodman, given that nbodykit wraps Corrfunc.
@FedeDK , could you add
kws['isa'] = fallback
to your corresponding file to
https://github.com/bccp/nbodykit/blob/master/nbodykit/algorithms/pair_counters/corrfunc/base.py#L201
The file should be at
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/pair_counters/corrfunc/base.py
The line number may be a bit off from 201.
I suppose this should pass isa='fallback'?
I got an error that says "fallback" is not defined, so I asume you meant that I write it as a string.
Doing so doesn't seem to have changed anything, I still get the "Illegal instruction" message.
If the Corrfunc being used is the one built with pip, the illegal instruction message definitely shouldn't appear! So maybe there are multiple versions of Corrfunc installed now. Might need to repeatedly pip uninstall corrfunc and conda remove corrfunc until you get to a clean slate.
Just to double check: you're pip-installing corrfunc on the same machine that you're running the tests on, right? Some clusters have login nodes that don't support AVX but compute nodes that do, for example.
Just to double check: you're pip-installing corrfunc on the same machine that you're running the tests on, right? Some clusters have login nodes that don't support AVX but compute nodes that do, for example.
Yeah, I'm trying everything in the cluster, where I have my virtual environment.
If the Corrfunc being used is the one built with pip, the illegal instruction message definitely shouldn't appear! So maybe there are multiple versions of Corrfunc installed now. Might need to repeatedly
pip uninstall corrfuncandconda remove corrfuncuntil you get to a clean slate.
I did pip uninstall corrfunc and conda remove corrfunc, then pip install -U -I --no-deps CorrFunc. Now I get: ImportError: No module named nbodykit.lab
Doesn't seem to be a path problem.
What happens if you now do import Corrfunc.tests; Corrfunc.tests.tests()
? This should tell us whether Corrfunc is working in this state. nbodykit
is probably complaining because it can't find its version of Corrfunc.
On Thu, Nov 15, 2018 at 9:28 AM FedeDK [email protected] wrote:
Just to double check: you're pip-installing corrfunc on the same machine that you're running the tests on, right? Some clusters have login nodes that don't support AVX but compute nodes that do, for example.
Yeah, I'm trying everything in the cluster, where I have my virtual environment.
If the Corrfunc being used is the one built with pip, the illegal instruction message definitely shouldn't appear! So maybe there are multiple versions of Corrfunc installed now. Might need to repeatedly pip uninstall corrfunc and conda remove corrfunc until you get to a clean slate.
I did pip uninstall corrfunc and conda remove corrfunc, then pip install -U -I --no-deps CorrFunc. Now I get: ImportError: No module named nbodykit.lab
Doesn't seem to be a path problem.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bccp/nbodykit/issues/534#issuecomment-439058350, or mute the thread https://github.com/notifications/unsubscribe-auth/AFYPy3TavczHZiS_fIeDj8sA1Kp5EhiHks5uvXn3gaJpZM4YcVEt .
When you did conda remove corrfunc nbodykit got removed too.
conda should have asked you to confirm before you pressed yes.
I agree we shall first confirm corrfunc is working properly.
Ok, I got nbodykit back to working shape, but CorrFunc isn't installed.
Trying pip install -U -I --no-deps CorrFunc didn't seem to work, it acts as if it's still not installed. I think this explains why I still had the "Illegal instruction" message before, the pip install wasn't affecting anything I was working with.
Any ideas on how to use the package from the pip installation? Is this a path problem?
What does which pip give you? Hopefully it will be pip from your conda
installation.
On Thu, Nov 15, 2018 at 12:32 PM FedeDK [email protected] wrote:
Ok, I got nbodykit back to working shape, but CorrFunc isn't installed.
Trying pip install -U -I --no-deps CorrFunc didn't seem to work, it acts as if it's still not installed. I think this explains why I still had the "Illegal instruction" message before, the pip install wasn't affecting anything I was working with.
Any ideas on how to use the package from the pip installation? Is this a path problem?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bccp/nbodykit/issues/534#issuecomment-439124377, or mute the thread https://github.com/notifications/unsubscribe-auth/AFYPy1SXJduHrzUCLSDVV92r5gUGmB2fks5uvaUxgaJpZM4YcVEt .
Something's funny with your Python environment then! pip should work fine inside conda. Can you send the output of the pip install command, along with which python, which pip, which conda, and python -c 'import Corrfunc; print(Corrfunc.__file__)'? Might be tough to diagnose remotely; a local expert might be able to help you faster.
You may wish to conda remove corrfunc again even if it uninstalls nbodykit, then confirm that you can get corrfunc working through pip. I guess the next step would be to get this version working with nbodykit, which hopefully is supported; @rainwoodman would know.
Ok, I got it to work. I bumped into another problem, but I'll get to that in the end. This is what i did:
I reinstalled gsl=2.4.
I pip installed Corrfunc with pip install git+git://github.com/nickhand/Corrfunc, this is the output
Collecting git+git://github.com/nickhand/Corrfunc Cloning git://github.com/nickhand/Corrfunc to /tmp/pip-21sDpM-build Requirement already satisfied: numpy>=1.7 in ./.conda/envs/fdavilakurban/lib/python2.7/site-packages (from Corrfunc==2.0.1) Requirement already satisfied: future in ./.conda/envs/fdavilakurban/lib/python2.7/site-packages (from Corrfunc==2.0.1) Installing collected packages: Corrfunc Found existing installation: Corrfunc 2.2.0 Uninstalling Corrfunc-2.2.0: Successfully uninstalled Corrfunc-2.2.0 Running setup.py install for Corrfunc ... done Successfully installed Corrfunc-2.0.1 You are using pip version 9.0.1, however version 18.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command.
So Corrfunc was indeed installed, I'm not sure what the problem was. Maybe the version posed an issue. I think this was all I did (I was trying a bunch of things in parallel), if I see anything else I'll edit the comment.
Just in case this isn't actually finally solved, I'll post the outputs of what you asked for:
which python, which pip, which conda yield:
~/.conda/envs/fdavilakurban/bin/python ~/.conda/envs/fdavilakurban/bin/pip ~/.conda/envs/fdavilakurban/bin/conda
So no issue there.
Then python -c 'import Corrfunc; print(Corrfunc.__file__)'
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/Corrfunc/init.pyc
Finally, the test import Corrfunc.tests; Corrfunc.tests.tests():
Done reading the data - time taken = 1.1 seconds Beginning Theory Correlation functions calculations Running 3-D correlation function DD(r) Running with points in [xmin,xmax] = 0.000400,419.999817 with periodic wrapping = 420.000000 Running with points in [ymin,ymax] = 0.000100,419.999512 with periodic wrapping = 420.000000 Running with points in [zmin,zmax] = 0.000300,419.999207 with periodic wrapping = 420.000000 In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,17. Time taken = 0.109 sec Using SSE kernel 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% done. Time taken = 5.346 secs
DD(r): first 5 bins rmin rmax rpavg npairs weightavg 0.1675 0.2388 0.0000 325832 1.0000 0.2388 0.3403 0.0000 498880 1.0000 0.3403 0.4849 0.0000 770944 1.0000 0.4849 0.6910 0.0000 1173536 1.0000 0.6910 0.9848 0.0000 1709944 1.0000
Running 2-D correlation function DD(rp,pi) Running with points in [xmin,xmax] = 0.000400,419.999817 with periodic wrapping = 420.000000 Running with points in [ymin,ymax] = 0.000100,419.999512 with periodic wrapping = 420.000000 Running with points in [zmin,zmax] = 0.000300,419.999207 with periodic wrapping = 420.000000 In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,10. Time taken = 0.108 sec Using SSE kernel 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% done. Time taken = 12.109 secs
****** DD(rp,pi): first 5 bins ******* rmin rmax rpavg pi_upper npairs weightavg 0.1675 0.2388 0.0000 1.0 488624 1.0000 0.1675 0.2388 0.0000 2.0 41796 1.0000 0.1675 0.2388 0.0000 3.0 16322 1.0000 0.1675 0.2388 0.0000 4.0 10674 1.0000 0.1675 0.2388 0.0000 5.0 8496 1.0000
Running 2-D correlation function DD(s,mu) Running with points in [xmin,xmax] = 0.000400,419.999817 with periodic wrapping = 420.000000 Running with points in [ymin,ymax] = 0.000100,419.999512 with periodic wrapping = 420.000000 Running with points in [zmin,zmax] = 0.000300,419.999207 with periodic wrapping = 420.000000 In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,35. Time taken = 0.137 sec Using SSE kernel 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% done. Time taken = 4.963 secs
****** DD(s,mu): first 5 bins ******* smin smax savg mu_max npairs weightavg 0.1675 0.2388 0.2034 0.1 16696 1.0000 0.1675 0.2388 0.2033 0.1 16488 1.0000 0.1675 0.2388 0.2037 0.2 16020 1.0000 0.1675 0.2388 0.2036 0.2 16060 1.0000 0.1675 0.2388 0.2035 0.2 15934 1.0000
Running 2-D projected correlation function wp(rp) In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,10. Time taken = 0.108 sec Using SSE kernel 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% done. Time taken = 10.966 secs
****** wp: first 5 bins ******* rmin rmax rpavg wp npairs weightavg 0.1675 0.2388 0.0000 303.6 718966 1.0000 0.2388 0.3403 0.0000 223.8 1156266 1.0000 0.3403 0.4849 0.0000 164.5 1890416 1.0000 0.4849 0.6910 0.0000 118.5 3116582 1.0000 0.6910 0.9848 0.0000 83.5 5212852 1.0000
Running 3-D auto-correlation function xi(r) In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,17. Time taken = 0.094 sec Using SSE kernel 0%Segmentation fault
This is all correct, yes?
The problem I'm facing now is that doing SurveyData2PCF.run() (I defined fc=SurveyData2PCF) yields an attribute error:
AttributeError Traceback (most recent call last)
in () ----> 1 fc.run()
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/paircount_tpcf/tpcf.pyc in run(self) 462 """ 463 # this does most of the work --> 464 BasePairCount2PCF.run(self) 465 466 # compute wp(rp) if we computed xi(rp, pi)
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/paircount_tpcf/tpcf.pyc in run(self) 125 result = LandySzalayEstimator(pair_counter, self.data1, self.data2, 126 self.randoms1, self.randoms2, R1R2=self.R1R2, --> 127 logger=self.logger, **attrs) 128 self.D1D2, self.D1R2, self.D2R1, self.R1R2, self.corr = result 129
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/paircount_tpcf/estimators.pyc in LandySzalayEstimator(pair_counter, data1, data2, randoms1, randoms2, R1R2, logger, **kwargs) 204 fRD = R1R2.attrs['total_wnpairs']/D2R1.attrs['total_wnpairs'] 205 --> 206 nonzero = R1R2.pairs['npairs'] > 0 207 208 # init
AttributeError: 'BinnedStatistic' object has no attribute 'pairs'
So checking my variable fc.R1R2 I find that indeed there is no attribute "pairs", and I'm unsure as to whether there should be or not. And whether this is related or warrants another thread.
Thanks for all the help so far!!
Hmm, the "Segmentation fault" at the end of the Corrfunc tests indicates an error. But github.com/nickhand/Corrfunc is a fork of an older version of Corrfunc; it's possible this error is fixed in the latest version from github.com/manodeep/Corrfunc. (Or the latest version straight from pip).
On Thu, Nov 15, 2018 at 2:10 PM FedeDK [email protected] wrote:
Ok, I got it to work. I bumped into another problem, but I'll get to that in the end. This is what i did:
I reinstalled gsl=2.4. I pip installed Corrfunc with pip install git+git:// github.com/nickhand/Corrfunc, this is the output
Collecting git+git://github.com/nickhand/Corrfunc Cloning git://github.com/nickhand/Corrfunc to /tmp/pip-21sDpM-build Requirement already satisfied: numpy>=1.7 in ./.conda/envs/fdavilakurban/lib/python2.7/site-packages (from Corrfunc==2.0.1) Requirement already satisfied: future in ./.conda/envs/fdavilakurban/lib/python2.7/site-packages (from Corrfunc==2.0.1) Installing collected packages: Corrfunc Found existing installation: Corrfunc 2.2.0 Uninstalling Corrfunc-2.2.0: Successfully uninstalled Corrfunc-2.2.0 Running setup.py install for Corrfunc ... done Successfully installed Corrfunc-2.0.1 You are using pip version 9.0.1, however version 18.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command.
So Corrfunc was indeed installed, I'm not sure what the problem was. Maybe the version posed an issue. I think this was all I did (I was trying a bunch of things in parallel), if I see anything else I'll edit the comment.
Just in case this isn't actually finally solved, I'll post the outputs of what you asked for: which python, which pip, which conda yield:
~/.conda/envs/fdavilakurban/bin/python ~/.conda/envs/fdavilakurban/bin/pip ~/.conda/envs/fdavilakurban/bin/conda
So no issue there.
Then python -c 'import Corrfunc; print(Corrfunc.file)'
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/Corrfunc/ init.pyc
Finally, the test import Corrfunc.tests; Corrfunc.tests.tests():
Done reading the data - time taken = 1.1 seconds Beginning Theory Correlation functions calculations Running 3-D correlation function DD(r) Running with points in [xmin,xmax] = 0.000400,419.999817 with periodic wrapping = 420.000000 Running with points in [ymin,ymax] = 0.000100,419.999512 with periodic wrapping = 420.000000 Running with points in [zmin,zmax] = 0.000300,419.999207 with periodic wrapping = 420.000000 In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,17. Time taken = 0.109 sec Using SSE kernel 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% done. Time taken = 5.346 secs
DD(r): first 5 bins rmin rmax rpavg npairs weightavg 0.1675 0.2388 0.0000 325832 1.0000 0.2388 0.3403 0.0000 498880 1.0000 0.3403 0.4849 0.0000 770944 1.0000 0.4849 0.6910 0.0000 1173536 1.0000 0.6910 0.9848 0.0000 1709944 1.0000
Running 2-D correlation function DD(rp,pi) Running with points in [xmin,xmax] = 0.000400,419.999817 with periodic wrapping = 420.000000 Running with points in [ymin,ymax] = 0.000100,419.999512 with periodic wrapping = 420.000000 Running with points in [zmin,zmax] = 0.000300,419.999207 with periodic wrapping = 420.000000 In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,10. Time taken = 0.108 sec Using SSE kernel 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% done. Time taken = 12.109 secs
****** DD(rp,pi): first 5 bins *******rmin rmax rpavg pi_upper npairs weightavg 0.1675 0.2388 0.0000 1.0 488624 1.0000 0.1675 0.2388 0.0000 2.0 41796 1.0000 0.1675 0.2388 0.0000 3.0 16322 1.0000 0.1675 0.2388 0.0000 4.0 10674 1.0000 0.1675 0.2388 0.0000 5.0 8496 1.0000
Running 2-D correlation function DD(s,mu) Running with points in [xmin,xmax] = 0.000400,419.999817 with periodic wrapping = 420.000000 Running with points in [ymin,ymax] = 0.000100,419.999512 with periodic wrapping = 420.000000 Running with points in [zmin,zmax] = 0.000300,419.999207 with periodic wrapping = 420.000000 In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,35. Time taken = 0.137 sec Using SSE kernel 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% done. Time taken = 4.963 secs
****** DD(s,mu): first 5 bins *******smin smax savg mu_max npairs weightavg 0.1675 0.2388 0.2034 0.1 16696 1.0000 0.1675 0.2388 0.2033 0.1 16488 1.0000 0.1675 0.2388 0.2037 0.2 16020 1.0000 0.1675 0.2388 0.2036 0.2 16060 1.0000 0.1675 0.2388 0.2035 0.2 15934 1.0000
Running 2-D projected correlation function wp(rp) In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,10. Time taken = 0.108 sec Using SSE kernel 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% done. Time taken = 10.966 secs
****** wp: first 5 bins *******rmin rmax rpavg wp npairs weightavg 0.1675 0.2388 0.0000 303.6 718966 1.0000 0.2388 0.3403 0.0000 223.8 1156266 1.0000 0.3403 0.4849 0.0000 164.5 1890416 1.0000 0.4849 0.6910 0.0000 118.5 3116582 1.0000 0.6910 0.9848 0.0000 83.5 5212852 1.0000
Running 3-D auto-correlation function xi(r) In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,17. Time taken = 0.094 sec Using SSE kernel 0%Segmentation fault
This is all correct, yes?
The problem I'm facing now is that doing SurveyData2PCF.run() (I defined fc=SurveyData2PCF) yields an attribute error:
AttributeError Traceback (most recent call last) in () ----> 1 fc.run()
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/paircount_tpcf/tpcf.pyc in run(self) 462 """ 463 # this does most of the work --> 464 BasePairCount2PCF.run(self) 465 466 # compute wp(rp) if we computed xi(rp, pi)
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/paircount_tpcf/tpcf.pyc in run(self) 125 result = LandySzalayEstimator(pair_counter, self.data1, self.data2, 126 self.randoms1, self.randoms2, R1R2=self.R1R2, --> 127 logger=self.logger, **attrs) 128 self.D1D2, self.D1R2, self.D2R1, self.R1R2, self.corr = result 129
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/paircount_tpcf/estimators.pyc in LandySzalayEstimator(pair_counter, data1, data2, randoms1, randoms2, R1R2, logger, **kwargs) 204 fRD = R1R2.attrs['total_wnpairs']/D2R1.attrs['total_wnpairs'] 205 --> 206 nonzero = R1R2.pairs['npairs'] > 0 207 208 # init
AttributeError: 'BinnedStatistic' object has no attribute 'pairs'
So checking my variable fc.R1R2 I find that indeed there is no attribute "pairs", and I'm unsure as to whether there should be or not. And whether this is related or warrants another thread.
Thanks for all the help so far!!
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bccp/nbodykit/issues/534#issuecomment-439155399, or mute the thread https://github.com/notifications/unsubscribe-auth/AFYPy0LQJYtm_OtLHilTw0wOjlZfHHy0ks5uvbwSgaJpZM4YcVEt .
I installed the version you mentioned:
pip install git+git://github.com/manodeep/Corrfunc
Collecting git+git://github.com/manodeep/Corrfunc Cloning git://github.com/manodeep/Corrfunc to /tmp/pip-__r1HV-build Requirement already satisfied: numpy>=1.7 in ./.conda/envs/fdavilakurban/lib/python2.7/site-packages (from Corrfunc==2.2.0) Requirement already satisfied: future in ./.conda/envs/fdavilakurban/lib/python2.7/site-packages (from Corrfunc==2.2.0) Collecting wurlitzer (from Corrfunc==2.2.0) Downloading https://files.pythonhosted.org/packages/60/17/de2820542c755f4630a58d295daad86bfa981fbf48b48e5f9e1f2ed806cc/wurlitzer-1.0.2-py2.py3-none-any.whl Installing collected packages: wurlitzer, Corrfunc Found existing installation: Corrfunc 2.0.1 Uninstalling Corrfunc-2.0.1: Successfully uninstalled Corrfunc-2.0.1 Running setup.py install for Corrfunc ... done Successfully installed Corrfunc-2.2.0 wurlitzer-1.0.2
But still get a similar output
import Corrfunc.tests; Corrfunc.tests.tests()
Done reading the data - time taken = 1.2 seconds Beginning Theory Correlation functions calculations Running 3-D correlation function DD(r) Running with points in [xmin,xmax] = 0.000400,419.999817 with periodic wrapping = 420.000000 Running with points in [ymin,ymax] = 0.000100,419.999512 with periodic wrapping = 420.000000 Running with points in [zmin,zmax] = 0.000300,419.999207 with periodic wrapping = 420.000000 In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,17. Time taken = 0.118 sec Using SSE kernel 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% done. Time taken = 5.338 secs
**** DD(r): first 5 bins ******* rmin rmax rpavg npairs weightavg 0.1675 0.2388 0.0000 325832 1.0000 0.2388 0.3403 0.0000 498880 1.0000 0.3403 0.4849 0.0000 770944 1.0000 0.4849 0.6910 0.0000 1173536 1.0000 0.6910 0.9848 0.0000 1709944 1.0000
Running 2-D correlation function DD(rp,pi) Running with points in [xmin,xmax] = 0.000400,419.999817 with periodic wrapping = 420.000000 Running with points in [ymin,ymax] = 0.000100,419.999512 with periodic wrapping = 420.000000 Running with points in [zmin,zmax] = 0.000300,419.999207 with periodic wrapping = 420.000000 In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,10. Time taken = 0.093 sec Using SSE kernel 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% done. Time taken = 12.005 secs
****** DD(rp,pi): first 5 bins ******* rmin rmax rpavg pi_upper npairs weightavg 0.1675 0.2388 0.0000 1.0 488624 1.0000 0.1675 0.2388 0.0000 2.0 41796 1.0000 0.1675 0.2388 0.0000 3.0 16322 1.0000 0.1675 0.2388 0.0000 4.0 10674 1.0000 0.1675 0.2388 0.0000 5.0 8496 1.0000
Running 2-D correlation function DD(s,mu) Running with points in [xmin,xmax] = 0.000400,419.999817 with periodic wrapping = 420.000000 Running with points in [ymin,ymax] = 0.000100,419.999512 with periodic wrapping = 420.000000 Running with points in [zmin,zmax] = 0.000300,419.999207 with periodic wrapping = 420.000000 In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,35. Time taken = 0.130 sec Using SSE kernel 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% done. Time taken = 4.847 secs
****** DD(s,mu): first 5 bins ******* smin smax savg mu_max npairs weightavg 0.1675 0.2388 0.2034 0.1 16696 1.0000 0.1675 0.2388 0.2033 0.1 16488 1.0000 0.1675 0.2388 0.2037 0.2 16020 1.0000 0.1675 0.2388 0.2036 0.2 16060 1.0000 0.1675 0.2388 0.2035 0.2 15934 1.0000
Running 2-D projected correlation function wp(rp) In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,10. Time taken = 0.099 sec Using SSE kernel 0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100% done. Time taken = 11.044 secs
****** wp: first 5 bins ******* rmin rmax rpavg wp npairs weightavg 0.1675 0.2388 0.0000 303.6 718966 1.0000 0.2388 0.3403 0.0000 223.8 1156266 1.0000 0.3403 0.4849 0.0000 164.5 1890416 1.0000 0.4849 0.6910 0.0000 118.5 3116582 1.0000 0.6910 0.9848 0.0000 83.5 5212852 1.0000
Running 3-D auto-correlation function xi(r) In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y, nmesh_z] = 35,35,17. Time taken = 0.097 sec Using SSE kernel 0%Segmentation fault