Wrong start/endTimeMillis?

[piratinskii]

Describe the Bug

Duration of the experiment doesn't match in the website and in the API. I got 00:08:13 in the website, but only 4 milliseconds on the API.

Expected behavior

endTimeMillis - startTimeMillis = Duration in milliseconds

Where is the issue?

• [ ] Comet Python SDK
• [ ] Comet UI
• [ ] Third Party Integrations (Huggingface, TensorboardX, Pytorch Lighting etc)

To Reproduce

Steps to reproduce the behavior:

1. Check the duration in the website
2. Check endTimeMillis - startTimeMillis in the API for the same experiment

Stack Trace

If possible please include the full stack trace of your issue here

# Paste stack trace here

Comet Debug Log

If possible, please follow the instructions here to run Comet in debug mode and attach the resulting log file.

Screenshots or GIFs

Link to Comet Project/Experiment

If applicable, please provide a link to your Comet Project or Experiment.

Additional context

Add any other context about the problem here.

[dsblank]

Perhaps the experiment was in the middle of still logging its data? I looked at the metadata for the experiment, and it looks correct now:

{
  "experimentKey": "807b0f20ab534d2bb4c57d0aa9929cf7",
  "experimentName": "entitled_parapet_204",
  "optimizationId": null,
  "userName": "piratinskii",
  "projectId": "c6964e6e81af4e03bf211172e4c4f8b1",
  "projectName": "comparison-yolov10",
  "workspaceName": "piratinskii",
  "filePath": "/home/sceuser/AthenaSky/yolo_train.py",
  "fileName": "yolo_train.py",
  "throttle": false,
  "throttleMessage": "",
  "throttlingReasons": [],
  "durationMillis": 493974,
  "startTimeMillis": 1718543791405,
  "endTimeMillis": 1718544285379,
  "running": false,
  "error": "Unexpected error detected in the user script: KeyboardInterrupt()",
  "hasCrashed": true,
  "archived": false,
  "tags": [],
}

And the times checkout:

>>> ((endTimeMillis - startTimeMillis) / 1000) / 60
8.232899999999999

[piratinskii]

Perhaps the experiment was in the middle of still logging its data? I looked at the metadata for the experiment, and it looks correct now:

{
  "experimentKey": "807b0f20ab534d2bb4c57d0aa9929cf7",
  "experimentName": "entitled_parapet_204",
  "optimizationId": null,
  "userName": "piratinskii",
  "projectId": "c6964e6e81af4e03bf211172e4c4f8b1",
  "projectName": "comparison-yolov10",
  "workspaceName": "piratinskii",
  "filePath": "/home/sceuser/AthenaSky/yolo_train.py",
  "fileName": "yolo_train.py",
  "throttle": false,
  "throttleMessage": "",
  "throttlingReasons": [],
  "durationMillis": 493974,
  "startTimeMillis": 1718543791405,
  "endTimeMillis": 1718544285379,
  "running": false,
  "error": "Unexpected error detected in the user script: KeyboardInterrupt()",
  "hasCrashed": true,
  "archived": false,
  "tags": [],
}

And the times checkout:

>>> ((endTimeMillis - startTimeMillis) / 1000) / 60
8.232899999999999

Hmm..it's seems like problem with experiments list: /api/rest/v2/experiments?projectId=ID&archived=false . I get wrong time there anyway. I don't know why I didn't use metadata endpoint... Thank you for right solution! But, anyway, it's seems like a bug in experiments endpoint

[dsblank]

Thanks for the additional information. I'll check that endpoint and will report any issues to the engineering team.

[dsblank]

I was able to replicate. Reported as CM-10771. Thanks again for reporting!

[dsblank]

The fix for this is scheduled to be made soon.

[dsblank]

Now being tested...

[dsblank]

This fix is now available on comet.com and will be available in on-prem installations in the next release.

Thanks for reporting!