bug: Only the Last Declared Histogram is Collected When Using Multiple Histograms

[takhyun12]

Describe the bug

I'm experiencing an issue where only the last declared Histogram metric is being collected when using multiple histograms in my BentoML service. The previously declared histograms do not collect any data, even though they are being called in the code.

To reproduce

Steps to Reproduce:

Here is a minimal reproducible example:

from bentoml.metrics import Counter, Histogram

test_runner = bentoml.Runner(TestRunnable, name="test_runner")
service = bentoml.Service(name="test_service", runners=[test_runner, (...)])

REQUEST_COUNT = Counter(name="request_count", documentation="Total number of requests")
REQUEST_LATENCY = Histogram(name="request_latency_seconds", documentation="Request latency in seconds")
REQUEST_IMAGE_WIDTH = Histogram(name="request_image_width", documentation="Width of input images in pixels")
REQUEST_IMAGE_HEIGHT = Histogram(name="request_image_height", documentation="Height of input images in pixels")


async def api(data: BytesIO):
    REQUEST_COUNT.inc()
    with REQUEST_LATENCY.time():
        image = PILImage.open(BytesIO(data.read())).convert("RGB")
        image = exif_transpose(image=image)

        original_size = image.size
        REQUEST_IMAGE_WIDTH.observe(original_size[0])
        REQUEST_IMAGE_HEIGHT.observe(original_size[1])

When I run the service and make requests, only the last declared histogram (REQUEST_IMAGE_HEIGHT) collects data. The other histograms (REQUEST_LATENCY and REQUEST_IMAGE_WIDTH) do not show any collected data in the metrics endpoint.

If I declare only one histogram (regardless of which one), it collects data as expected. Declaring two or more histograms results in only the last one collecting data.

I have tried changing the order of the histogram declarations, and it's always the last one that collects data. No errors or warnings are logged during runtime.

Could you please help investigate this issue? Is there a known limitation or a specific way to declare multiple histograms in BentoML so that they all collect data correctly?

Thank you for your assistance.

Expected behavior

All declared Histogram metrics should collect and display data when their observe() method is called, regardless of the number of histograms declared.

Environment

bentoml: 1.2.16 Python: 3.10 OS: Ubuntu 20.04.6 LTS

[frostming]

Can you upgrade to the latest bentoml version and see if the issue exists? Reporting against an old version is not helpful

[takhyun12]

@frostming

I've upgraded to the latest version of BentoML (1.3.10), but I'm still encountering the same issue.

[frostming]

Run this code on the latest BentoML and it reports all metrics correctly:

from pathlib import Path

import bentoml
import PIL.Image as PILImage
from bentoml.metrics import Counter, Histogram

REQUEST_COUNT = Counter(name="request_count", documentation="Total number of requests")
REQUEST_LATENCY = Histogram(
    name="request_latency_seconds", documentation="Request latency in seconds"
)
REQUEST_IMAGE_WIDTH = Histogram(
    name="request_image_width", documentation="Width of input images in pixels"
)
REQUEST_IMAGE_HEIGHT = Histogram(
    name="request_image_height", documentation="Height of input images in pixels"
)


@bentoml.service
class MyService:
    @bentoml.api
    def generate(self, src_image: Path) -> Path:
        REQUEST_COUNT.inc()
        with REQUEST_LATENCY.time():
            image = PILImage.open(src_image).convert("RGB")

            original_size = image.size
            REQUEST_IMAGE_WIDTH.observe(original_size[0])
            REQUEST_IMAGE_HEIGHT.observe(original_size[1])
        return src_image