java-opentimestamps
java-opentimestamps copied to clipboard
opentimestamps
Unknown Attestation in OTS proof
Hi, one of the timestamps made on 2022-08-19 has verify Unknown Attestation in it's info. Timestamping was done using the multistamp method with a total of 23 document hashes
The proof : 004f70656e54696d657374616d7073000050726f6f6600bf89e2e884e892940108a0733aadc1dbe15d74410ecace34b8f6decc357c5c99825b23421ae458d9a94af010ca7d36ebc0fe1498eecb71047080dba808f1203bad873856885bae567a2458ccb7caabfe909e0a64d7f522f48006672468954108f12078ba28a25bb603ce9e047c4f9ce339b944ff2b84a8015fe27525c483636894b008f02032b27027e360ae98e1145071d9c9916c99dfad7fc3538c9c2acd43aa4af5490508f020440bb1c7f15a276d68710fabf3ad0492f872789bcf17ab92e8395990f9a7d08c08f020d79a5bc4c75e5e8661d9e70ac18b7f51f2463accee92966e0cb010e682e016a20800
Document Hash : a0733aadc1dbe15d74410ecace34b8f6decc357c5c99825b23421ae458d9a94a
Weird. Looks like you have a corrupted proof. Running ots info on that proof gives the error message: Tried to read 8 bytes but got only 0 bytes
Is this error repeatable?
Hi @petertodd , we have multistamped more than 25,000 hashes so far using Opentimestamps and this is the first time we got such an error. Have you seen this error before? If so, kindly let us know if there is any fix for this.
On Tue, Oct 18, 2022 at 03:38:33AM -0700, Rebekkal wrote:
Hi @petertodd , we have multistamped more than 25,000 hashes so far using Opentimestamps and this is the first time we got such an error. Have you seen this error before? If so, kindly let us know if there is any fix for this.
Sorry, I've never seen this error before. It's quite possible there was some kind of hardware or other error that resulted in the timestamp being truncated. Since it was truncated, there's probably no way to recover that particular timestamp.
If this happens again, please let us know!
On October 27, 2022 12:25:20 PM GMT+02:00, Rebekkal @.***> wrote:
Thank you for your response.
-- Reply to this email directly or view it on GitHub: https://github.com/opentimestamps/java-opentimestamps/issues/58#issuecomment-1293316291 You are receiving this because you were mentioned.
Message ID: @.***>
Hi @petertodd, the above issue seems to have reoccurred. I apologize for any inconvenience this may cause and would greatly appreciate any assistance you can provide in resolving this issue.
Following are the dates in which it has happened.
2022-11-25 2022-11-27 2022-12-16 2022-12-21 2022-12-22 2023-01-01 2023-01-02 2023-01-10 2023-01-11 2023-01-18
An example ots-proof received with Unknown attestation.
Ots Proof : 004f70656e54696d657374616d7073000050726f6f6600bf89e2e884e892940108006875e61d982e8541e73c9c0ea913ea473dcaae812934e0e54ea27ecbe4338ff010d8aafae63599379d0bda403859b6c3f508f020446f05f572070fe0334b03a8750392f31c3e682495e4e22decae23292e0213d408f120fc11847023ad3bb0344586243bda3bd16eb470bddbae72a9a0ada0e0b6597fb608f120dff1e4bcb08bcfbd107dda9f346de735a54a73a3a8a4f9e2cde0e760efb31afb08f0203c1bc4af0dbf1587873f55f1538f4e79520104e4eb40db738012dfe27c8fd08a0800
Document Hash : 006875e61d982e8541e73c9c0ea913ea473dcaae812934e0e54ea27ecbe4338f
Can you post some of the code used to make timestamps on your end? I have another thing that makes tens of thousands of timestamps regularly with python-opentimestamps and it hasn't seen a similar issue. So I suspect the issue has something to do with your specific setup.
Something that could help here is to try to deserialize every timestamp you receive, and retry the timestamp if deserialization fails.
On January 23, 2023 6:53:48 AM EST, Rebekkal @.***> wrote:
Reopened #58.
-- Reply to this email directly or view it on GitHub: https://github.com/opentimestamps/java-opentimestamps/issues/58#event-8333424959 You are receiving this because you were mentioned.
Message ID: @.***>
Thank you for your quick response, @petertodd
public static List<OtsTimestamp> stampHashes(List<DocHash> pendingHashData) throws AppException {
List<OtsTimestamp> otsTimestamps = new ArrayList<OtsTimestamp>();
HashMap<String, String> privateUrls = new HashMap<>();
List<String> calendarsUrl = new ArrayList<>();
HashMap<DocHash, DetachedTimestampFile> mapFiles = new HashMap<>();
Integer m = 0;
// convert hash to DetachedTimestampFile
for (DocHash pendingHash : pendingHashData) {
try {
String hashValue = pendingHash.getDocumentHash();
byte[] shasum = Utils.hexToBytes(hashValue);
mapFiles.put(pendingHash, DetachedTimestampFile.from(new Hash(shasum, OpSHA256._TAG)));
} catch (Exception e) {
LOGGER.log(Level.SEVERE, "Crypto Error", e);
}
}
// Stamping
Timestamp stampResult;
String merkleRoot = "";
try {
List<DetachedTimestampFile> detaches = new ArrayList<DetachedTimestampFile>(mapFiles.values());
stampResult = OpenTimestamps.stamp(detaches, calendarsUrl, m, privateUrls);
merkleRoot = Utils.bytesToHex(stampResult.msg);
} catch (IOException e) {
LOGGER.log(Level.SEVERE, "Stamp Error", e);
}
// Generate ots proofs
for (Map.Entry<DocHash, DetachedTimestampFile> entry : mapFiles.entrySet()) {
DocHash timestampData = entry.getKey();
DetachedTimestampFile detached = entry.getValue();
try {
// convert the bytes to hex
byte[] otsBytes = detached.serialize();
LOGGER.info("CHECKING FOR INVALID ATTESTATION IN THE RECEIVED PROOF");
DetachedTimestampFile.deserialize(otsBytes); // Temp fix for the attestation issue
LOGGER.info("NO INVALID ATTESTATION FOUND");
String otsHex = Utils.bytesToHex(otsBytes);
java.sql.Timestamp submittedTime = new java.sql.Timestamp(timestampData.getSubmittedTime());
otsTimestamps.add(new OtsTimestamp(timestampData, otsHex, submittedTime));
} catch (NullPointerException npe) {
LOGGER.log(Level.INFO, npe.getMessage());
throw npe;
} catch (Exception e) {
LOGGER.log(Level.SEVERE, e.getLocalizedMessage(), e);
}
}
return otsTimestamps;
}
So I decoded your invalid timestamp by hand:
004f70656e54696d657374616d7073000050726f6f6600bf89e2e884e8929401 # header
08 # File hash type: OpSHA256
006875e61d982e8541e73c9c0ea913ea473dcaae812934e0e54ea27ecbe4338f # file digest
f0 # OpAppend
10 # 16 bytes
d8aafae63599379d0bda403859b6c3f5
08 # OpSHA256
f0 # OpAppend
20 # 32 bytes
446f05f572070fe0334b03a8750392f31c3e682495e4e22decae23292e0213d4
08 # OpSHA256
f1 # OpPrepend
20 # 32 bytes
fc11847023ad3bb0344586243bda3bd16eb470bddbae72a9a0ada0e0b6597fb6
08 # OpSha256
f1 # OpPrepend
20 # 32 bytes
dff1e4bcb08bcfbd107dda9f346de735a54a73a3a8a4f9e2cde0e760efb31afb
08 # OpSHA256
f0 # OpAppend
20 # 32 bytes
3c1bc4af0dbf1587873f55f1538f4e79520104e4eb40db738012dfe27c8fd08a
08 # OpSha256
00 # Attestation
# End-of-file, when there should be an attestation!
I'm not familiar with the java-opentimestamps code base. But I suspect the problem has something to do with a lack of attestations.
@RCasatta It seems that the codebase allows a timestamp to have zero attestations: https://github.com/opentimestamps/java-opentimestamps/blob/81863915ea218354e9c84cae0c522b85d1dc9f91/src/main/java/com/eternitywall/ots/Timestamp.java#L118 This is incorrect: a valid timestamp must have at least one attestation at all times.
So maybe the issue is that the OpenTimestamps.stamp function isn't raising an exception properly when the calendar fails?
@RebekkalPangras I assume you're only timestamping with a single calendar right?
Hi @petertodd , Yes timestamping was done with single calendar before. Now I have changed it to two.
On February 2, 2023 5:21:17 AM GMT+01:00, Rebekkal @.***> wrote:
Hi @petertodd , Yes timestamping was done with single calendar before. Now I have changed it to two.
Are you saving calendar data locally, eg by upgrading the timestamps when the transactions confirm?
Yes, I save the ots proof locally.
Good! So my suggestion to you for now would be to continue timestamping with multiple calendars, and use the above code to verify it.
Long term we should fix java-opentimestamps to deal with errors better to avoid others making this mistake. I don't know any Java though, so hopefully one of the people who wrote this library can help. You're also welcome to come up with a pull-req. :)
Thank you for your suggestion @petertodd . As for fixing the java-opentimestamps library, I will look into contributing a pull request or reaching out to the developers for assistance.
On Mon, Feb 06, 2023 at 09:25:36PM -0800, Rebekkal wrote:
Thank you for your suggestion @petertodd . As for fixing the java-opentimestamps library, I will look into contributing a pull request or reaching out to the developers for assistance.
Thanks!