thredds
thredds copied to clipboard
Performance Issue with high variable and dimension count
I try to use CDF format to store my data so that each piece of data is using one dimension and two variables. I noticed that the loading speed of the CDF file is increasing significantly with the dimension and variable count. I also run a simple test on this issue. The dimension length is randomized from 1 to 10000 for each data, and the data size is from 1000 to 10000 with a step of 1000 (for example, a data size of 1000 will give dimension count of 1000 and variable count of 2000). Here is the result: Size: 1000 Time: 48 ms Size: 2000 Time: 133 ms Size: 3000 Time: 150 ms Size: 4000 Time: 404 ms Size: 5000 Time: 622 ms Size: 6000 Time: 1426 ms Size: 7000 Time: 2374 ms Size: 8000 Time: 2773 ms Size: 9000 Time: 4044 ms Size: 10000 Time: 5159 ms Maybe you can find a way to fix this? Thank you.
The file format is netcdf3.