pyvmomi icon indicating copy to clipboard operation
pyvmomi copied to clipboard

Pickling pyvmomi class using multiprocessing

Open welibekov opened this issue 6 years ago • 2 comments

Hi folk, I try do use python multiprocessing module to parallelize some data gathering from vcenter. Unfortunately i have faced with following problem during starting parallel processes:

_pickle.PicklingError: Can't pickle <class 'pyVmomi.VmomiSupport.vim.VirtualMachine'>: attribute lookup vim.VirtualMachine on pyVmomi.VmomiSupport failed

So the question is, it's that true, that pymomi doesn't support "right pickling"? Thanks, and appreciate.

welibekov avatar Jul 23 '19 20:07 welibekov

I ran into this same issue but found out that there is a JSON encoder that can be used to move these objects between processes.

from pyVmomi.VmomiSupport import VmomiJSONEncoder

Use this to create your list: jsonSerialized= json.dumps(pfVmomiObj, cls=VmomiJSONEncoder)

Then in the mapped function, use this to recover the object: pfVmomiObj = json.loads(jsonSerialized)

george-gwu avatar Jul 28 '19 19:07 george-gwu

VmomiJSONEncoder solved the problem and I can pass a serialized vim.VirtualMachine object to the mapped function.

But how do I do the opposite conversion (from json to vim.VirtualMachine object)? This: pfVmomiObj = json.loads(jsonSerialized) returns a dict, so I cannot use pfVmomiObj with APIs that take a ManagedEntity as argument (for example InitiateFileTransferToGuest).

Thanks

cube0x8 avatar Oct 19 '20 17:10 cube0x8