Pickle.jl icon indicating copy to clipboard operation
Pickle.jl copied to clipboard

`keras.src.saving.pickle_utils.deserialize_model_from_bytecode` not found, results in Defer

Open IanButterworth opened this issue 1 year ago • 2 comments

I'm guessing this means some keras support needs to be added.

julia> params = Pickle.Torch.THload("foo.pth")
Defer(:reduce, Defer(:keras.src.saving.pickle_utils.deserialize_model_from_bytecode), UInt8[0x50, 0x4b, 0x03, 0x04, 0x14, 0x00, 0x00, 0x00, 0x00, 0x00  …  0xb2, 0x00, 0x00, 0x00, 0xc6, 0xa5, 0xee, 0x02, 0x00, 0x00])

If so, can you provide guidance. I'm happy to PR

IanButterworth avatar Dec 04 '23 21:12 IanButterworth

You would need to implement a julia function that perform keras.src.saving.pickle_utils.deserialize_model_from_bytecode on UInt8[0x50, 0x4b, 0x03, 0x04, 0x14, 0x00, 0x00, 0x00, 0x00, 0x00 … 0xb2, 0x00, 0x00, 0x00, 0xc6, 0xa5, 0xee, 0x02, 0x00, 0x00] and register that to the unpickler (TorchPickler().mt["keras.src.saving.pickle_utils.deserialize_model_from_bytecode"] = julia_impl_deserialize_model_from_bytecode). Though I'm not sure what exactly keras.src.saving.pickle_utils.deserialize_model_from_bytecodedo.

chengchingwen avatar Dec 05 '23 02:12 chengchingwen

https://github.com/keras-team/keras/blob/601488fd4c1468ae7872e132e0f1c9843df54182/keras/saving/pickle_utils.py#L46

https://github.com/keras-team/keras/blob/601488fd4c1468ae7872e132e0f1c9843df54182/keras/saving/saving_lib.py#L233

The bytes seem to be raw bytes of a zipfile, which contains the configurations and weights stored in either H5 or Npz. The load_model function would directly construct the keras model from the configurations and weights.

chengchingwen avatar Dec 05 '23 16:12 chengchingwen