BentoML icon indicating copy to clipboard operation
BentoML copied to clipboard

Documentation for custom metrics with Prometheus is outdated

Open ahallermed opened this issue 2 years ago • 6 comments

Describe the bug In the file docs/source/guides/monitoring.rst the following passage is outdated, links don't work anymore and the api has changed.


NOTE Users can also create custom metrics for BentoService making use of |prom_client|_, which can be later scraped by Prometheus.

       from bentoml.configuratoin.containers import BentoMLContainer

       metrics_client = BentoMLContainer.metircs_client.get()

       REQUEST_TIME = metrics_clint.Summary('request_processing_time', 'Time spend processing request')

       @artifacts([KerasModelArtifact('model'), PickleArtifact('tokenizer')])
       class TensorflowService(BentoService):

           @REQUEST_TIME.time()
           @api(input=JsonInput())
           def predict(self, parsed_json):
               raw = self.preprocessing(parsed_json['text'])
               input_data = [raw[: n + 1] for n in range(len(raw))]
               input_data = pad_sequences(input_data, maxlen=100, padding="post")
               return self.artifacts.model.predict(input_data)

ahallermed avatar Nov 04 '21 17:11 ahallermed

Gets fixed by this pull request #1961 and can be closed after it is merged.

ahallermed avatar Nov 04 '21 17:11 ahallermed

@ahallermed thank you for reporting this, you are right the sample code is outdated and we will look into that. cc @bojiang

parano avatar Nov 04 '21 17:11 parano

There is a typo in the doc:

from bentoml.configuratoin.containers import BentoMLContainer

It should be:

from bentoml.configuration.containers import BentoMLContainer

bojiang avatar Nov 05 '21 02:11 bojiang

https://github.com/bentoml/BentoML/pull/1963

bojiang avatar Nov 05 '21 02:11 bojiang

Update: this doc is actually only available for the master branch, not the 0.13.1 release, the document got updated before the release.

For version 0.13.1, the suggested change in #1961 should work.

parano avatar Nov 05 '21 06:11 parano

The docs are outdated again, I have tried the following code:

from bentoml._internal.configuration.containers import DeploymentContainer

metrics_client = DeploymentContainer.metrics_client.get()
REQUEST_TIME = metrics_client.Counter(
    name="foo",
    documentation="bar",
    labelnames=["endpoint"],
)

However I get an assertion error from the PrometheusClient class, "Invalid prometheus multiproc directory"

vikramjit-sidhu avatar Jun 29 '22 12:06 vikramjit-sidhu

Hi there, can you try out our recent documentation on the new metrics API? https://docs.bentoml.org/en/latest/guides/metrics.html

aarnphm avatar Nov 09 '22 05:11 aarnphm