vaex icon indicating copy to clipboard operation
vaex copied to clipboard

[BUG-REPORT] Printing vaex df after sort running out of memory

Open meta-ks opened this issue 8 months ago • 0 comments

Description Environment: IPython 8.15.0 + Python 3.10.13 I am encountering an issue where sorting a Vaex DataFrame and then attempting to print it results in running out of memory. I am able to print(just a sample in ipython env) the original df without any issues.

Code:

vdf = vaex.open('large_8GB_arrow_file.arrow', progress=True)
vdf  #works no issue
vdf2 = vdf.sort(['col1', 'col2', ....])   #5cols
vdf2  #hangs and crashes

Software information

  • Vaex version:
'vaex-core': '4.17.1',
'vaex-viz': '0.5.4',
'vaex-hdf5': '0.14.1',
'vaex-server': '0.9.0',
'vaex-astro': '0.9.3',
'vaex-jupyter': '0.8.2',
'vaex-ml': '0.18.3'
  • Vaex was installed via: pip
  • OS: Ubuntu 22

meta-ks avatar Oct 30 '23 07:10 meta-ks