parca icon indicating copy to clipboard operation
parca copied to clipboard

Report error "failed to insert profile: timestamp must not be zero" on v0.11.1

Open mangox opened this issue 3 years ago • 4 comments

Error: level=error name=parca ts=2022-05-24T07:26:54.961974115Z caller=scrape.go:500 ... msg="WriteRaw failed for scraped profile" err="rpc error: code = Internal desc = failed to insert profile: timestamp must not be zero"

profiling config in parca.yml.

profiling_config:
      pprof_config:
        fgprof:
          enabled: true
          path: /debug/fgprof
          delta: true

mangox avatar May 24 '22 07:05 mangox

I am not sure but I think I found the root cause. This line is adding seconds but subtracting 1 from it, due to which scraper is not picking it up correctly. As mentioned in parca.yaml it should be in ?seconds=<scrape-interval> format but instead it is in ?seconds=<scrape-interval> - 1.

I don't think this would solve the problem, doing this even removes /debug/pprof/profile WDYT cc @brancz?

knrt10 avatar May 24 '22 09:05 knrt10

Yeah unfortunately we realized that fgprof doesn't set several fields that we require to be set, I already opened several PRs to fix it: https://github.com/felixge/fgprof/pulls?q=is%3Apr+author%3Abrancz+

brancz avatar May 24 '22 10:05 brancz

@brancz Cool. Looks those PRs could solve my issue. Hope they could be merged ASAP. I'll try to build it locally using your commits.

mangox avatar May 24 '22 15:05 mangox

I tried reaching out to Felix via other channels as well, I know he’s super busy with $dayjob.

brancz avatar May 24 '22 18:05 brancz