Support 64-bit zonelists

[aowen87]

Be sure to see Wiki article, 32 and 64 bit indexing in Silo Jeff Grandy requested an examination of this issue. He is considering supporting 64 bit node/zone ids in things like material lists, zonelists, polyhedral zonelists, etc. Currently, VTK uses the type vtkIdType for this. However, it is also typedef'd to be an int. Also, VisIt takes an automatic 2x memory hit if all of the suddent we switched to handling all this as 64bit. Thats worse if most of the time, we never actually need the extra size. Certainly, during problemsized I/O operations, if we read 64bit data, we can discover if its actually even needed and if not, convert it to 32bit data. However, what happens if we read 64bit data and actually need that number of bits for the ucd zonelist? On a large shared memory system, it might be possible for us to decompose a large block (one requiring 64bit ids) on the fly into smaller blocks such that no block actually needs 64-bit and then process it in parallel (perhaps with threads) instead.

-----------------------REDMINE MIGRATION----------------------- This ticket was migrated from Redmine. As such, not all information was able to be captured in the transition. Below is a complete record of the original redmine ticket.

Ticket number: 1935 Status: Pending Project: VisIt Tracker: Feature Priority: Normal Subject: Support 64-bit zonelists Assigned to: - Category: - Target version: - Author: Mark Miller Start: 07/30/2014 Due date: % Done: 0% Estimated time: Created: 07/30/2014 01:56 pm Updated: 10/16/2017 07:34 pm Likelihood: Severity: Found in version: Impact: 3 - Medium Expected Use: 3 - Occasional OS: All Support Group: Any Description: Be sure to see Wiki article, 32 and 64 bit indexing in Silo Jeff Grandy requested an examination of this issue. He is considering supporting 64 bit node/zone ids in things like material lists, zonelists, polyhedral zonelists, etc. Currently, VTK uses the type vtkIdType for this. However, it is also typedef'd to be an int. Also, VisIt takes an automatic 2x memory hit if all of the suddent we switched to handling all this as 64bit. Thats worse if most of the time, we never actually need the extra size. Certainly, during problemsized I/O operations, if we read 64bit data, we can discover if its actually even needed and if not, convert it to 32bit data. However, what happens if we read 64bit data and actually need that number of bits for the ucd zonelist? On a large shared memory system, it might be possible for us to decompose a large block (one requiring 64bit ids) on the fly into smaller blocks such that no block actually needs 64-bit and then process it in parallel (perhaps with threads) instead.

Comments: Jeff and I met and discussed this today.One big issue for me is what is the driver for this. Typically, although we handle meshes containing billions of zones, we do so in multi-block fashion so that any single, monlothic, coherent piece of mesh is relatively small and can be handled using 32 bit integers.One of the drivers for this is the possibility that future machines may need to be configured such that each MPI task is a big, fat large memory node with a slew of threads. In that case, it may be more efficient to run with a large, monolithic, coherent single mesh object containing billions of zones than it is to treat that same mesh as decomposed into smaller pieces containing only a few hundred millions of zones.Possible solutions for SiloPlan for work. Over the next year