python-javaobj
python-javaobj copied to clipboard
tcalmant
Handle the SC_WRITE_METHOD class description flag
>>> val
'\xac\xed\x00\x05sr\x00-net.centersight.model.datapoint.DatapointBean\x00\x00\x00\x00\x00\x00\x00\x01\x02\x00\x11L\x00\x06accesst\x00\x13Ljava/lang/Integer;L\x00\x07addresst\x00\x12Ljava/lang/String;L\x00\x08categoryq\x00~\x00\x01L\x00\x0fcounterTimeBaseq\x00~\x00\x01L\x00\x08dataTypeq\x00~\x00\x01L\x00\rdatapointTypeq\x00~\x00\x01L\x00\x0cdisplayLevelq\x00~\x00\x01L\x00\nidentifiert\x00\x10Ljava/lang/Long;L\x00\tisCountert\x00\x13Ljava/lang/Boolean;L\x00\nlastUpdatet\x00\x10Ljava/util/Date;L\x00\x0clastValueStrq\x00~\x00\x02L\x00\rprevLastValuet\x004Lnet/centersight/model/datapoint/DatapointValueBean;L\x00\x10prevLastValueStrq\x00~\x00\x02L\x00\x0bstorageTypeq\x00~\x00\x01L\x00\x10transformDisplayq\x00~\x00\x02L\x00\x0btransformInq\x00~\x00\x02L\x00\x04unitq\x00~\x00\x02xr\x00\x1enet.centersight.model.NodeBean\x00\x00\x00\x00\x00\x00\x00\x01\x02\x00\x12I\x00\x05levelI\x00\x08nodeTypeI\x00\tsortIndexL\x00\ralarmSeverityq\x00~\x00\x01L\x00\x10customNodeTypeIdq\x00~\x00\x03L\x00\x0bdescriptionq\x00~\x00\x02L\x00\x02idq\x00~\x00\x03L\x00\x0blastUpdatedq\x00~\x00\x05L\x00\rlastUpdatedByq\x00~\x00\x02L\x00\nlinkedNodet\x00 Lnet/centersight/model/NodeBean;L\x00\x0bmapLatitudet\x00\x12Ljava/lang/Double;L\x00\x0cmapLongitudeq\x00~\x00\tL\x00\x07mapZoomq\x00~\x00\x01L\x00\x04nameq\x00~\x00\x02L\x00\x08parentIdq\x00~\x00\x03L\x00\x04pathq\x00~\x00\x02L\x00\x08subNodest\x00\x10Ljava/util/List;L\x00\x08timeZonet\x00\x14Ljava/util/TimeZone;xp\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00pppsr\x00\x0ejava.lang.Long;\x8b\xe4\x90\xcc\x8f#\xdf\x02\x00\x01J\x00\x05valuexr\x00\x10java.lang.Number\x86\xac\x95\x1d\x0b\x94\xe0\x8b\x02\x00\x00xp\x00\x00\x00\x00\x007Oxsr\x00\x12java.sql.Timestamp&\x18\xd5\xc8\x01S\xbfe\x02\x00\x01I\x00\x05nanosxr\x00\x0ejava.util.Datehj\x81\x01KYt\x19\x03\x00\x00xpw\x08\x00\x00\x01I\xcd`\xfdpx!\xf9\x82\x80t\x00\x06_ROOT_ppppt\x00\rMESSAGE_ERRORsq\x00~\x00\r\x00\x00\x00\x00\x00+\xb3(t\x00\x1b625/2863909/2863912/3624824ppppppsr\x00\x11java.lang.Integer\x12\xe2\xa0\xa4\xf7\x81\x878\x02\x00\x01I\x00\x05valuexq\x00~\x00\x0e\x00\x00\x00\x04sq\x00~\x00\x17\x00\x00\x00\nppsr\x00\x11java.lang.Boolean\xcd r\x80\xd5\x9c\xfa\xee\x02\x00\x01Z\x00\x05valuexp\x00ppppppt\x00\rMESSAGE_ERRORp'
>>> val_deserilized=javaobj.loads(val)
No handlers could be found for logger "javaobj"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.6/dist-packages/javaobj.py", line 180, in loads
return marshaller.readObject()
File "/usr/local/lib/python2.6/dist-packages/javaobj.py", line 383, in readObject
_, res = self._read_and_exec_opcode(ident=0)
File "/usr/local/lib/python2.6/dist-packages/javaobj.py", line 448, in _read_and_exec_opcode
return (opid, handler(ident=ident))
File "/usr/local/lib/python2.6/dist-packages/javaobj.py", line 674, in do_object
res = self._read_value(field_type, ident, name=field_name)
File "/usr/local/lib/python2.6/dist-packages/javaobj.py", line 862, in _read_value
_, res = self._read_and_exec_opcode(ident=ident + 1)
File "/usr/local/lib/python2.6/dist-packages/javaobj.py", line 446, in _read_and_exec_opcode
.format(opid))
RuntimeError: Unknown OpCode in the stream: 0x1
I've just made a small test and reproduced this error.
==Oops state dump=============
References: [[net.centersight.model.datapoint.DatapointBean:0x1],
'Ljava/lang/Integer;', 'Ljava/lang/String;', 'Ljava/lang/Long;',
'Ljava/lang/Boolean;', 'Ljava/util/Date;',
'Lnet/centersight/model/datapoint/DatapointValueBean;',
[net.centersight.model.NodeBean:0x1], 'Lnet/centersight/model/NodeBean;',
'Ljava/lang/Double;', 'Ljava/util/List;', 'Ljava/util/TimeZone;',
<javaobj:net.centersight.model.datapoint.DatapointBean>,
[java.lang.Long:0x3B8BE490CC8F23DF], [java.lang.Number:0x-79536AE2F46B1F75],
<javaobj:java.lang.Long>, [java.sql.Timestamp:0x2618D5C80153BF65],
[java.util.Date:0x686A81014B597419], <javaobj:java.sql.Timestamp>]
Stream seeking back at -16 byte (2nd line is an actual position!):
Warning!!!!: Stream still has 233 bytes left.
0000 81 01 4B 59 74 19 03 00 00 78 70 77 08 00 00 01 ..KYt....xpw....
0010 49 CD 60 FD 70 78 21 F9 82 80 74 00 06 5F 52 4F IÍ`ýpx!ù..t.._RO
0020 4F 54 5F 70 70 70 70 74 00 0D 4D 45 53 53 41 47 OT_ppppt..MESSAG
0030 45 5F 45 52 52 4F 52 73 71 00 7E 00 0D 00 00 00 E_ERRORsq.~.....
0040 00 00 2B B3 28 74 00 1B 36 32 35 2F 32 38 36 33 ..+³(t..625/2863
0050 39 30 39 2F 32 38 36 33 39 31 32 2F 33 36 32 34 909/2863912/3624
0060 38 32 34 70 70 70 70 70 70 73 72 00 11 6A 61 76 824ppppppsr..jav
0070 61 2E 6C 61 6E 67 2E 49 6E 74 65 67 65 72 12 E2 a.lang.Integer.â
0080 A0 A4 F7 81 87 38 02 00 01 49 00 05 76 61 6C 75 .¤÷..8...I..valu
0090 65 78 71 00 7E 00 0E 00 00 00 04 73 71 00 7E 00 exq.~......sq.~.
00A0 17 00 00 00 0A 70 70 73 72 00 11 6A 61 76 61 2E .....ppsr..java.
00B0 6C 61 6E 67 2E 42 6F 6F 6C 65 61 6E CD 20 72 80 lang.BooleanÍ r.
00C0 D5 9C FA EE 02 00 01 5A 00 05 76 61 6C 75 65 78 Õ.úî...Z..valuex
00D0 70 00 70 70 70 70 70 70 74 00 0D 4D 45 53 53 41 p.ppppppt..MESSA
00E0 47 45 5F 45 52 52 4F 52 70 GE_ERRORp
==============================
Traceback (most recent call last):
File "C:\Python34\lib\site-packages\javaobj.py", line 443, in _read_and_exec_opcode
handler = self.opmap[opid]
KeyError: 1
#... (after that, its the same trace as yours)
I can't work on it today, but I'll take a look when I'll be able.
Just a comment to tell you I haven't forget about this issue ;)
So far, I've found that this might be due to the way java.sql.Timestamp is serialized.
Its parent class java.util.Date has a customized serialization: it just writes an integer timestamp instead of associating its fields with a value.
This seems to cause an inconsistent reading of the class by Javaobj and to miss the next object, and to raise the error you see.
In Java, this is not problem, as the class of the loaded object is available and the reader can call the Date.readObject(InputObjectStream) method.
I'm still looking for a workaround for javaobj :/
I believe the same thing is happening for ImmutableListMultiMap (maybe just the version from Guava.) The serializer seems to write the number of key/value sets as block data, then for each set, writes: 1.) the key, 2.) blockdata with the count of the values, 3.) the list of values. In most cases, this seems to get sucked into the annotations and at least the stream is consistent (though the keys/values are not accessible in any convenient way.) But for an EmptyImmuntableListMultimap, the block data saying there are 0 k/v sets becomes the value for the following object.
I think one thing that might help is checking not just the classdesc flags, but the collapsed flags for all superclasses, for SC_WRITE_METHOD. It seems any time a parent class uses SC_WRITE_METHOD, there's a chance (nay, a guarantee) that one part of the object has used a blockdata--though it may be in the middle of a three-class hierarchy! The current code doesn't seem to OR the flags together, but instead uses the values from the lowest level class. It's also not clear how to interpret this when a class is referenced and that class has SC_WRITE_METHOD, or how to interpret it in the context of an object (where the classdec has already been read--it seems that it's being used for object data, not one-time class description data.)
This case is readable by the version 2 of the package, should this issue be closed?
I believe I had the issue in version 2. Can you give me the commit id you used for testing and I'll re-check?
I tested the @noragen binary with the current master state and it works.
`import javaobj.v2 as javaobj
test = b'\xac\xed\x00\x05sr\x00-net.centersight.model.datapoint.DatapointBean\x00\x00\x00\x00\x00\x00\x00\x01\x02\x00\x11L\x00\x06accesst\x00\x13Ljava/lang/Integer;L\x00\x07addresst\x00\x12Ljava/lang/String;L\x00\x08categoryq\x00~\x00\x01L\x00\x0fcounterTimeBaseq\x00~\x00\x01L\x00\x08dataTypeq\x00~\x00\x01L\x00\rdatapointTypeq\x00~\x00\x01L\x00\x0cdisplayLevelq\x00~\x00\x01L\x00\nidentifiert\x00\x10Ljava/lang/Long;L\x00\tisCountert\x00\x13Ljava/lang/Boolean;L\x00\nlastUpdatet\x00\x10Ljava/util/Date;L\x00\x0clastValueStrq\x00~\x00\x02L\x00\rprevLastValuet\x004Lnet/centersight/model/datapoint/DatapointValueBean;L\x00\x10prevLastValueStrq\x00~\x00\x02L\x00\x0bstorageTypeq\x00~\x00\x01L\x00\x10transformDisplayq\x00~\x00\x02L\x00\x0btransformInq\x00~\x00\x02L\x00\x04unitq\x00~\x00\x02xr\x00\x1enet.centersight.model.NodeBean\x00\x00\x00\x00\x00\x00\x00\x01\x02\x00\x12I\x00\x05levelI\x00\x08nodeTypeI\x00\tsortIndexL\x00\ralarmSeverityq\x00~\x00\x01L\x00\x10customNodeTypeIdq\x00~\x00\x03L\x00\x0bdescriptionq\x00~\x00\x02L\x00\x02idq\x00~\x00\x03L\x00\x0blastUpdatedq\x00~\x00\x05L\x00\rlastUpdatedByq\x00~\x00\x02L\x00\nlinkedNodet\x00 Lnet/centersight/model/NodeBean;L\x00\x0bmapLatitudet\x00\x12Ljava/lang/Double;L\x00\x0cmapLongitudeq\x00~\x00\tL\x00\x07mapZoomq\x00~\x00\x01L\x00\x04nameq\x00~\x00\x02L\x00\x08parentIdq\x00~\x00\x03L\x00\x04pathq\x00~\x00\x02L\x00\x08subNodest\x00\x10Ljava/util/List;L\x00\x08timeZonet\x00\x14Ljava/util/TimeZone;xp\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00pppsr\x00\x0ejava.lang.Long;\x8b\xe4\x90\xcc\x8f#\xdf\x02\x00\x01J\x00\x05valuexr\x00\x10java.lang.Number\x86\xac\x95\x1d\x0b\x94\xe0\x8b\x02\x00\x00xp\x00\x00\x00\x00\x007Oxsr\x00\x12java.sql.Timestamp&\x18\xd5\xc8\x01S\xbfe\x02\x00\x01I\x00\x05nanosxr\x00\x0ejava.util.Datehj\x81\x01KYt\x19\x03\x00\x00xpw\x08\x00\x00\x01I\xcd\xfdpx!\xf9\x82\x80t\x00\x06_ROOT_ppppt\x00\rMESSAGE_ERRORsq\x00~\x00\r\x00\x00\x00\x00\x00+\xb3(t\x00\x1b625/2863909/2863912/3624824ppppppsr\x00\x11java.lang.Integer\x12\xe2\xa0\xa4\xf7\x81\x878\x02\x00\x01I\x00\x05valuexq\x00~\x00\x0e\x00\x00\x00\x04sq\x00~\x00\x17\x00\x00\x00\nppsr\x00\x11java.lang.Boolean\xcd r\x80\xd5\x9c\xfa\xee\x02\x00\x01Z\x00\x05valuexp\x00ppppppt\x00\rMESSAGE_ERRORp' pobj = javaobj.loads(test)
- About add support to SC_WRITE_METHOD, I added the following PR => https://github.com/tcalmant/python-javaobj/pull/38 which should be able to process any custom writing