UnifiedMetrics icon indicating copy to clipboard operation
UnifiedMetrics copied to clipboard
verified publisher icon Cubxity

Error on ingesting samples with different value but same timestamp

Open SamB440 opened this issue 11 months ago • 2 comments

Is there an existing issue for this bug?

  • [X] I have searched the existing issues before opening this bug report.

Current Behavior

Prometheus 3.1.0 is reporting the following problem:

time=2025-01-10T13:51:05.826Z level=WARN source=scrape.go:1885 msg="Error on ingesting samples with different value but same timestamp" component="scrape manager" scrape_pool=development target=http://e5528e9d-866f-48c2-b5ba-ee65da85c1d9:9100/metrics num_dropped=114

On finding https://github.com/prometheus/prometheus/issues/14089#issuecomment-2211842643, running that command gave:

      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="0.001",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="0.005",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="0.01",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="0.02",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="0.03",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="0.04",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="0.05",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="0.075",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="0.1",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="0.25",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="0.5",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="0.75",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="1.0",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="10.0",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="2.5",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="5.0",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="60.0",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="7.5",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Concurrent GC",le="+Inf",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="0.001",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="0.005",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="0.01",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="0.02",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="0.03",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="0.04",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="0.05",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="0.075",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="0.1",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="0.25",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="0.5",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="0.75",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="1.0",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="10.0",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="2.5",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="5.0",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="60.0",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="7.5",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Old Generation",le="+Inf",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="0.001",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="0.005",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="0.01",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="0.02",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="0.03",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="0.04",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="0.05",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="0.075",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="0.1",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="0.25",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="0.5",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="0.75",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="1.0",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="10.0",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="2.5",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="5.0",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="60.0",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="7.5",}
      2 jvm_gc_duration_seconds_bucket{gc="G1 Young Generation",le="+Inf",}
      2 jvm_gc_duration_seconds_count{gc="G1 Concurrent GC",}
      2 jvm_gc_duration_seconds_count{gc="G1 Old Generation",}
      2 jvm_gc_duration_seconds_count{gc="G1 Young Generation",}
      2 jvm_gc_duration_seconds_sum{gc="G1 Concurrent GC",}
      2 jvm_gc_duration_seconds_sum{gc="G1 Old Generation",}
      2 jvm_gc_duration_seconds_sum{gc="G1 Young Generation",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Concurrent GC",le="1.0E8",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Concurrent GC",le="1.0E9",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Concurrent GC",le="2.0E9",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Concurrent GC",le="2.5E7",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Concurrent GC",le="2.5E8",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Concurrent GC",le="3.0E9",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Concurrent GC",le="5.0E7",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Concurrent GC",le="5.0E8",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Concurrent GC",le="5.0E9",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Concurrent GC",le="+Inf",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Old Generation",le="1.0E8",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Old Generation",le="1.0E9",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Old Generation",le="2.0E9",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Old Generation",le="2.5E7",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Old Generation",le="2.5E8",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Old Generation",le="3.0E9",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Old Generation",le="5.0E7",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Old Generation",le="5.0E8",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Old Generation",le="5.0E9",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Old Generation",le="+Inf",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Young Generation",le="1.0E8",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Young Generation",le="1.0E9",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Young Generation",le="2.0E9",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Young Generation",le="2.5E7",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Young Generation",le="2.5E8",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Young Generation",le="3.0E9",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Young Generation",le="5.0E7",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Young Generation",le="5.0E8",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Young Generation",le="5.0E9",}
      2 jvm_gc_freed_bytes_bucket{gc="G1 Young Generation",le="+Inf",}
      2 jvm_gc_freed_bytes_count{gc="G1 Concurrent GC",}
      2 jvm_gc_freed_bytes_count{gc="G1 Old Generation",}
      2 jvm_gc_freed_bytes_count{gc="G1 Young Generation",}
      2 jvm_gc_freed_bytes_sum{gc="G1 Concurrent GC",}
      2 jvm_gc_freed_bytes_sum{gc="G1 Old Generation",}
      2 jvm_gc_freed_bytes_sum{gc="G1 Young Generation",}
      2 jvm_memory_bytes_committed{area="heap",}
      2 jvm_memory_bytes_committed{area="nonheap",}
      2 jvm_memory_bytes_init{area="heap",}
      2 jvm_memory_bytes_init{area="nonheap",}
      2 jvm_memory_bytes_max{area="heap",}
      2 jvm_memory_bytes_max{area="nonheap",}
      2 jvm_memory_bytes_used{area="heap",}
      2 jvm_memory_bytes_used{area="nonheap",}
      2 jvm_threads_current_count
      2 jvm_threads_daemon_count
      2 jvm_threads_peak
      2 jvm_threads_started_total
      2 process_cpu_load_ratio
      2 process_cpu_seconds_total
      2 process_start_time_seconds

Expected Behavior

Not to error

Server type

Spigot (includes Spigot-based forks)

Server Version

1.21.1

Other Mods / Plugins in your server

N/A

Error log

No response

More information

No response

SamB440 avatar Jan 10 '25 14:01 SamB440

Are you using any other plugins that are using the Prometheus client library?

Cubxity avatar Jan 19 '25 10:01 Cubxity

No, all the plugins we use are custom as well except WorldEdit, nuvotifier, spark, and UnifiedMetrics

SamB440 avatar Jan 21 '25 18:01 SamB440