Add message counts grouped by lang tag to system stats query

[andreaskoepf]

Add counts of (non-deleted) messages grouped by their lang-tag to the system stats query. The current SystemStats response model can be found here.

[dslvt]

Can I try to do this task?

[dslvt]

I am trying to run a backend. Can you figure out with with error?

~/Documents/Open-Assistant/backend ~/Documents/Open-Assistant/scripts/backend-development INFO: Will watch for changes in these directories: ['/Users/slvt/Documents/Open-Assistant/backend'] INFO: Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit) INFO: Started reloader process [91601] using StatReload Process SpawnProcess-1: Traceback (most recent call last): File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/uvicorn/_subprocess.py", line 76, in subprocess_started target(sockets=sockets) File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/uvicorn/server.py", line 60, in run return asyncio.run(self.serve(sockets=sockets)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/asyncio/runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/asyncio/base_events.py", line 650, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/uvicorn/server.py", line 67, in serve config.load() File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/uvicorn/config.py", line 477, in load self.loaded_app = import_from_string(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/uvicorn/importer.py", line 21, in import_from_string module = importlib.import_module(module_str) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1206, in _gcd_import File "<frozen importlib._bootstrap>", line 1178, in _find_and_load File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 690, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 940, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/Users/slvt/Documents/Open-Assistant/backend/main.py", line 9, in <module> import fastapi File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/fastapi/__init__.py", line 7, in <module> from .applications import FastAPI as FastAPI File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/fastapi/applications.py", line 15, in <module> from fastapi import routing File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/fastapi/routing.py", line 24, in <module> from fastapi.dependencies.models import Dependant File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/fastapi/dependencies/models.py", line 3, in <module> from fastapi.security.base import SecurityBase File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/fastapi/security/__init__.py", line 1, in <module> from .api_key import APIKeyCookie as APIKeyCookie File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/fastapi/security/api_key.py", line 3, in <module> from fastapi.openapi.models import APIKey, APIKeyIn File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/fastapi/openapi/models.py", line 103, in <module> class Schema(BaseModel): File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/pydantic/main.py", line 292, in __new__ cls.__signature__ = ClassAttribute('__signature__', generate_model_signature(cls.__init__, fields, config)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/site-packages/pydantic/utils.py", line 258, in generate_model_signature merged_params[param_name] = Parameter( ^^^^^^^^^^ File "/opt/homebrew/Caskroom/miniforge/base/envs/open_assistant/lib/python3.11/inspect.py", line 2715, in __init__ raise ValueError('{!r} is not a valid parameter name'.format(name)) ValueError: 'not' is not a valid parameter name

[dslvt]

And is it correct DATABASE_URI?

DATABASE_URI="postgresql://postgres:postgres@db:5432/postgres"

[dslvt]

@andreaskoepf

[hemangjoshi37a]

Thanks for bringing up this issue! I can understand how this could be frustrating.

Regarding the issue you're experiencing, it sounds like the problem may be related to the configuration of your environment. In order to properly troubleshoot the issue, it would be helpful to have more information about your setup, such as the version of the library you're using and the version of Python you're running. Additionally, it would be helpful to see the full traceback of the error message you're receiving.

One solution to this issue could be to try updating to the latest version of the library. This may resolve any compatibility issues that may be causing the problem. Another option could be to try running the code in a different environment, such as a virtual environment, to isolate any potential issues with your current setup.

Additionally, I would like to suggest that you could try to check the logging and see whether it will throw any error message which could be helpful in debugging.

Lastly, if you're still experiencing issues after trying these solutions, please don't hesitate to reach out for further assistance. We're here to help, and we'll do our best to help you resolve the problem.

Please let me know if these suggestions help or if there's anything else I can do to assist.

Also, I would like to suggest that you could also try to reproduce the issue in a minimal example, and share that, it could help other contributors to quickly reproduce and understand the problem.

[kingychiu]

Hi, I am picking this up. I have got the local dev server running. Based on the @andreaskoepf comment above, I think the API is http://localhost:8080/api/v1/stats/

I am trying to call that API from swagger but am encountering an auth error.

Please advise. How can I authenticate myself via swagger? Or should I be able to do this ticket by only unit test (if any)?

Thank you

[kingychiu]

I am able to call the API with X-API-Key=1234, It is defined under the Setting:

class Settings(BaseSettings):
    PROJECT_NAME: str = "open-assistant backend"
    API_V1_STR: str = "/api/v1"
    OFFICIAL_WEB_API_KEY: str = "1234"

[kingychiu]

PR created https://github.com/LAION-AI/Open-Assistant/pull/1101 please review.

[kingychiu]

Hi, @andreaskoepf any feedback on this PR? It has been here for 2 weeks. Please let me know if it fits the requirements.

[olliestanley]

Closing as #1101 is merged