DB-GPT icon indicating copy to clipboard operation
DB-GPT copied to clipboard

Published 20 hours ago • verified publisher icon eosphoros-ai

RuntimeError: cannot schedule new futures after interpreter shutdown

Open ahbon123 opened this issue 11 months ago • 8 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

Operating system information

MacOS(M1, M2...)

Python version information

=3.11

DB-GPT version

latest release

Related scenes

  • [ ] Chat Data
  • [ ] Chat Excel
  • [X] Chat DB
  • [ ] Chat Knowledge
  • [ ] Model Management
  • [ ] Dashboard
  • [ ] Plugins

Installation Information

Device information

Macbook Pro M3

Models information

I have two files on the folders of models: text2vec-large-chinese and vicuna-13b-v1.5.

What happened

RuntimeError: cannot schedule new futures after interpreter shutdown

What you expected to happen

I executed the following command on the Mac command line:

conda activate dbgpt_env
python dbgpt/app/dbgpt_server.py

I expected to be able to access it through , but the following error occurred.

How to reproduce

(dbgpt_env) mac@bogon DB-GPT % python dbgpt/app/dbgpt_server.py                                 

=========================== WebServerParameters ===========================

host: 0.0.0.0
port: 5000
daemon: False
controller_addr: None
model_name: vicuna-13b-v1.5
share: False
remote_embedding: False
log_level: INFO
light: False
log_file: dbgpt_webserver.log
tracer_file: dbgpt_webserver_tracer.jsonl
tracer_storage_cls: None
disable_alembic_upgrade: False
awel_dirs: None
default_thread_pool_size: None

======================================================================


2024-03-19 21:33:55 bogon dbgpt.component[93860] INFO Register component with name dbgpt_unified_metadata_db_manager_factory and instance: <dbgpt.storage.metadata.db_factory.UnifiedDBManagerFactory object at 0x176401390>
2024-03-19 21:33:56 bogon dbgpt.component[93860] INFO Register component with name dbgpt_thread_pool_default and instance: <dbgpt.util.executor_utils.DefaultExecutorFactory object at 0x2b27fa080>
2024-03-19 21:33:56 bogon dbgpt.component[93860] INFO Register component with name dbgpt_default_scheduler and instance: <dbgpt.app.initialization.scheduler.DefaultScheduler object at 0x2b2834c40>
2024-03-19 21:33:56 bogon dbgpt.component[93860] INFO Register component with name dbgpt_model_controller and instance: <dbgpt.model.cluster.controller.controller.ModelControllerAdapter object at 0x17606a680>
2024-03-19 21:33:56 bogon dbgpt.component[93860] INFO Register component with name dbgpt_plugin_hub and instance: <dbgpt.serve.agent.hub.controller.ModulePlugin object at 0x28b23aa70>
2024-03-19 21:33:56 bogon dbgpt.component[93860] INFO Register component with name dbgpt_multi_agents and instance: <dbgpt.serve.agent.agents.controller.MultiAgents object at 0x28bbf4280>
2024-03-19 21:33:56 bogon dbgpt.app.initialization.embedding_component[93860] INFO Register local LocalEmbeddingFactory
2024-03-19 21:33:56 bogon dbgpt.model.cluster.worker.embedding_worker[93860] INFO [EmbeddingsModelWorker] Parameters of device is None, use mps
2024-03-19 21:33:56 bogon dbgpt.app.initialization.embedding_component[93860] INFO 

=========================== EmbeddingModelParameters ===========================

model_name: text2vec
model_path: /Users/mac/DB-GPT/models/text2vec-large-chinese
device: mps
normalize_embeddings: None

======================================================================


/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/langchain/embeddings/__init__.py:29: LangChainDeprecationWarning: Importing embeddings from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.embeddings import HuggingFaceEmbeddings`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
2024-03-19 21:33:57 bogon sentence_transformers.SentenceTransformer[93860] INFO Load pretrained SentenceTransformer: /Users/mac/DB-GPT/models/text2vec-large-chinese
2024-03-19 21:33:57 bogon sentence_transformers.SentenceTransformer[93860] WARNING No sentence-transformers model found with name /Users/mac/DB-GPT/models/text2vec-large-chinese. Creating a new one with MEAN pooling.
2024-03-19 21:33:59 bogon dbgpt.component[93860] INFO Register component with name embedding_factory and instance: <dbgpt.app.initialization.embedding_component.LocalEmbeddingFactory object at 0x2b2834eb0>
2024-03-19 21:33:59 bogon dbgpt.component[93860] INFO Register component with name dbgpt_model_cache_manager and instance: <dbgpt.storage.cache.manager.LocalCacheManager object at 0x2cf343190>
2024-03-19 21:33:59 bogon dbgpt.component[93860] INFO Register component with name dbgpt_awel_trigger_manager and instance: <dbgpt.core.awel.trigger.trigger_manager.DefaultTriggerManager object at 0x2cf410340>
2024-03-19 21:33:59 bogon dbgpt.component[93860] INFO Register component with name dbgpt_awel_dag_manager and instance: <dbgpt.core.awel.dag.dag_manager.DAGManager object at 0x2cf410af0>
2024-03-19 21:33:59 bogon dbgpt.component[93860] INFO Register component with name dbgpt_app_editor_service and instance: <dbgpt.app.openapi.api_v1.editor.service.EditorService object at 0x2cf410c10>
2024-03-19 21:33:59 bogon dbgpt.component[93860] INFO Register component with name dbgpt_serve_prompt and instance: <dbgpt.serve.prompt.serve.Serve object at 0x2cf410c70>
2024-03-19 21:33:59 bogon dbgpt.component[93860] INFO Register component with name dbgpt_serve_prompt_service and instance: <dbgpt.serve.prompt.service.service.Service object at 0x2cf410ca0>
2024-03-19 21:33:59 bogon dbgpt.component[93860] INFO Register component with name dbgpt_serve_conversation and instance: <dbgpt.serve.conversation.serve.Serve object at 0x28bb5e6b0>
2024-03-19 21:33:59 bogon dbgpt.component[93860] INFO Register component with name dbgpt_serve_conversation_service and instance: <dbgpt.serve.conversation.service.service.Service object at 0x28bb5e740>
2024-03-19 21:33:59 bogon dbgpt.component[93860] INFO Register component with name dbgpt_serve_flow and instance: <dbgpt.serve.flow.serve.Serve object at 0x2cf3d9630>
2024-03-19 21:33:59 bogon dbgpt.component[93860] INFO Register component with name dbgpt_dbgpts_loader and instance: <dbgpt.util.dbgpts.loader.DBGPTsLoader object at 0x2cf4fc520>
2024-03-19 21:33:59 bogon dbgpt.component[93860] INFO Register component with name dbgpt_serve_flow_service and instance: <dbgpt.serve.flow.service.service.Service object at 0x2cf3d96c0>
2024-03-19 21:33:59 bogon dbgpt.util._db_migration_utils[93860] WARNING Initialize and upgrade database metadata with alembic, just run this in your development environment, if you deploy this in production environment, please run webserver with --disable_alembic_upgrade(`python dbgpt/app/dbgpt_server.py --disable_alembic_upgrade`).
we suggest you to use `dbgpt db migration` to initialize and upgrade database metadata with alembic, your can run `dbgpt db migration --help` to get more information.
2024-03-19 21:33:59 bogon alembic.runtime.migration[93860] INFO Context impl SQLiteImpl.
2024-03-19 21:33:59 bogon alembic.runtime.migration[93860] INFO Will assume non-transactional DDL.
2024-03-19 21:33:59 bogon dbgpt.util._db_migration_utils[93860] INFO Migration versions and their file paths:
========================================Migration versions========================================

3ed3251d42b2 (current): New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/3ed3251d42b2_new_migration.py)
a97e0270e22a : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/a97e0270e22a_new_migration.py)
01065a264f4e : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/01065a264f4e_new_migration.py)
527dff78863a : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/527dff78863a_new_migration.py)
71ba6e0f49ab : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/71ba6e0f49ab_new_migration.py)
ec79bec3957e : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/ec79bec3957e_new_migration.py)
4d7311c48c8f : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/4d7311c48c8f_new_migration.py)
e3c187c43814 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/e3c187c43814_new_migration.py)
f6d6ef43be3e : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/f6d6ef43be3e_new_migration.py)
0e1a06aa60c1 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/0e1a06aa60c1_new_migration.py)
8cc51efce6a6 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/8cc51efce6a6_new_migration.py)
4107504e1bd2 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/4107504e1bd2_new_migration.py)
fea0b593936c : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/fea0b593936c_new_migration.py)
3fe7807e6b3f : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/3fe7807e6b3f_new_migration.py)
757290b390fa : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/757290b390fa_new_migration.py)
3251cabd398d : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/3251cabd398d_new_migration.py)
8103c22aedbc : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/8103c22aedbc_new_migration.py)
df4fa22f23f2 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/df4fa22f23f2_new_migration.py)
edf1abe898ff : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/edf1abe898ff_new_migration.py)
f350d48665ab : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/f350d48665ab_new_migration.py)
6e740641ba46 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/6e740641ba46_new_migration.py)
c76c81eadc26 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/c76c81eadc26_new_migration.py)
==========================================================================================
2024-03-19 21:33:59 bogon alembic.runtime.migration[93860] INFO Context impl SQLiteImpl.
2024-03-19 21:33:59 bogon alembic.runtime.migration[93860] INFO Will assume non-transactional DDL.
2024-03-19 21:33:59 bogon alembic.runtime.migration[93860] INFO Context impl SQLiteImpl.
2024-03-19 21:33:59 bogon alembic.runtime.migration[93860] INFO Will assume non-transactional DDL.
2024-03-19 21:33:59 bogon dbgpt.util._db_migration_utils[93860] INFO alembic migration current revision: 3ed3251d42b2, latest revision: 3ed3251d42b2
2024-03-19 21:33:59 bogon alembic.runtime.migration[93860] INFO Context impl SQLiteImpl.
2024-03-19 21:33:59 bogon alembic.runtime.migration[93860] INFO Will assume non-transactional DDL.
  Generating /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/542211b93c27_new_migration.py ...  done
2024-03-19 21:33:59 bogon alembic.runtime.migration[93860] INFO Context impl SQLiteImpl.
2024-03-19 21:33:59 bogon alembic.runtime.migration[93860] INFO Will assume non-transactional DDL.
2024-03-19 21:33:59 bogon alembic.runtime.migration[93860] INFO Running upgrade 3ed3251d42b2 -> 542211b93c27, New migration
Model Unified Deployment Mode!
2024-03-19 21:33:59 bogon dbgpt.model.cluster.worker.manager[93860] INFO Worker params: 

=========================== ModelWorkerParameters ===========================

model_name: vicuna-13b-v1.5
model_path: /Users/mac/DB-GPT/models/vicuna-13b-v1.5
worker_type: None
model_alias: None
worker_class: None
model_type: huggingface
host: 0.0.0.0
port: 5000
daemon: False
limit_model_concurrency: 5
standalone: True
register: True
worker_register_host: None
controller_addr: None
send_heartbeat: True
heartbeat_interval: 20
log_level: None
log_file: dbgpt_model_worker_manager.log
tracer_file: dbgpt_model_worker_manager_tracer.jsonl
tracer_storage_cls: None

======================================================================


2024-03-19 21:33:59 bogon dbgpt.model.cluster.worker.manager[93860] INFO Run WorkerManager with standalone mode, controller_addr: http://127.0.0.1:5000
2024-03-19 21:33:59 bogon dbgpt.model.adapter.model_adapter[93860] INFO Use fastcat adapter
Warning: Loading base MPT model with `zero_shot` conversation configuration.  If this is not desired, inspect model configurations and names.
2024-03-19 21:33:59 bogon dbgpt.model.adapter.fschat_adapter[93860] INFO Found llm model adapter with model name: vicuna-13b-v1.5, <fastchat.model.model_adapter.VicunaAdapter object at 0x2b5707df0>
2024-03-19 21:33:59 bogon dbgpt.model.cluster.worker.default_worker[93860] INFO model_name: vicuna-13b-v1.5, model_path: /Users/mac/DB-GPT/models/vicuna-13b-v1.5, model_param_class: <class 'dbgpt.model.parameter.ModelParameters'>
2024-03-19 21:33:59 bogon dbgpt.model.cluster.worker.default_worker[93860] INFO [DefaultModelWorker] Parameters of device is None, use mps
2024-03-19 21:33:59 bogon dbgpt.model.cluster.worker.manager[93860] INFO Init empty instances list for vicuna-13b-v1.5@llm
2024-03-19 21:33:59 bogon dbgpt.component[93860] INFO Register component with name dbgpt_worker_manager_factory and instance: <dbgpt.model.cluster.worker.manager._DefaultWorkerManagerFactory object at 0x2b56c9930>
2024-03-19 21:33:59 bogon dbgpt.util.dbgpts.loader[93860] INFO Found 0 dbgpts packages from /Users/mac/.dbgpts/packages/be6ccebca05e56cbc6376891fa5bde08
INFO:     Started server process [93860]
INFO:     Waiting for application startup.
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_rag_rewrite_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Found dag DAG(dag_id=dbgpt_awel_simple_rag_rewrite_example) from mod <module 'unusual_prefix_05eb0f06d6bf45e5e4db9df22468546eca159645_simple_rag_rewrite_example' from '/Users/mac/DB-GPT/examples/awel/simple_rag_rewrite_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_rag_rewrite_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Found dag DAG(dag_id=simple_nl_schema_sql_chart_example) from mod <module 'unusual_prefix_3b7f8b6d07e23e1dc8539846ebfcef24da736d3e_simple_nl_schema_sql_chart_example' from '/Users/mac/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_chat_dag_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Found dag DAG(dag_id=dbgpt_awel_simple_dag_example) from mod <module 'unusual_prefix_0bf46b05e5219d64d0b7f09daf5ba07ef5312bfb_simple_chat_dag_example' from '/Users/mac/DB-GPT/examples/awel/simple_chat_dag_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_chat_dag_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_dag_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Found dag DAG(dag_id=simple_dag_example) from mod <module 'unusual_prefix_3ca65816aa3e0a5a3c89277cad0babff3626add2_simple_dag_example' from '/Users/mac/DB-GPT/examples/awel/simple_dag_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_dag_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_rag_summary_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Found dag DAG(dag_id=dbgpt_awel_simple_rag_summary_example) from mod <module 'unusual_prefix_c049f97a0c99ff4dd86982980daa40c02fd8f747_simple_rag_summary_example' from '/Users/mac/DB-GPT/examples/awel/simple_rag_summary_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_rag_summary_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_chat_history_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Found dag DAG(dag_id=dbgpt_awel_simple_chat_history) from mod <module 'unusual_prefix_54a4381cd27e4033b94ed9a9c4d606fd5a0da530_simple_chat_history_example' from '/Users/mac/DB-GPT/examples/awel/simple_chat_history_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_chat_history_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_llm_client_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Found dag DAG(dag_id=dbgpt_awel_simple_llm_client_generate) from mod <module 'unusual_prefix_4a0a98a7a84f1ba3faefba5929ad567a532f8d19_simple_llm_client_example' from '/Users/mac/DB-GPT/examples/awel/simple_llm_client_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_llm_client_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Found dag DAG(dag_id=dbgpt_awel_simple_llm_client_count_token) from mod <module 'unusual_prefix_4a0a98a7a84f1ba3faefba5929ad567a532f8d19_simple_llm_client_example' from '/Users/mac/DB-GPT/examples/awel/simple_llm_client_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_llm_client_example.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Importing /Users/mac/DB-GPT/examples/awel/data_analyst_assistant.py
2024-03-19 21:33:59 bogon dbgpt.core.interface.operators.message_operator[93860] INFO Storage is not set, use the InMemoryStorage as the conversation storage.
2024-03-19 21:33:59 bogon dbgpt.core.interface.operators.message_operator[93860] INFO Message storage is not set, use the InMemoryStorage as the message 
2024-03-19 21:33:59 bogon dbgpt.core.awel.dag.loader[93860] INFO Found dag DAG(dag_id=dbgpt_awel_data_analyst_assistant) from mod <module 'unusual_prefix_443fd604a8960e2154c906db820757ec146a0e94_data_analyst_assistant' from '/Users/mac/DB-GPT/examples/awel/data_analyst_assistant.py'> and model file /Users/mac/DB-GPT/examples/awel/data_analyst_assistant.py
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Register trigger HttpTrigger(node_id=ba740c34-f0b7-40ff-93f6-010045db90b4)
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2b8f4c790>(AWEL_trigger_route__examples_rag_rewrite), endpoint: /examples/rag/rewrite, methods: ['POST']
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/rag/rewrite
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Include router <fastapi.routing.APIRouter object at 0x2cf4104f0> to prefix path /api/v1/awel/trigger
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Register trigger HttpTrigger(node_id=c1bc4a12-8516-4209-af97-4a2ffb1eee08)
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2b8f4cb80>(AWEL_trigger_route__examples_rag_schema_linking), endpoint: /examples/rag/schema_linking, methods: ['POST']
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/rag/schema_linking
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Include router <fastapi.routing.APIRouter object at 0x2cf4104f0> to prefix path /api/v1/awel/trigger
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Register trigger HttpTrigger(node_id=e8f548a2-c28b-4948-b4c3-c6d959a18bd9)
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2b8f4d120>(AWEL_trigger_route__examples_simple_chat), endpoint: /examples/simple_chat, methods: ['POST']
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_chat
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Include router <fastapi.routing.APIRouter object at 0x2cf4104f0> to prefix path /api/v1/awel/trigger
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Register trigger HttpTrigger(node_id=3a0d9db9-213b-4560-b80b-507b5eb8c955)
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function_get at 0x2b8f4d7e0>(AWEL_trigger_route__examples_hello), endpoint: /examples/hello, methods: ['GET']
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/hello
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Include router <fastapi.routing.APIRouter object at 0x2cf4104f0> to prefix path /api/v1/awel/trigger
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Register trigger HttpTrigger(node_id=16ef4a5c-46f1-4b46-8900-48439f7d04f3)
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2b8f4dfc0>(AWEL_trigger_route__examples_rag_summary), endpoint: /examples/rag/summary, methods: ['POST']
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/rag/summary
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Include router <fastapi.routing.APIRouter object at 0x2cf4104f0> to prefix path /api/v1/awel/trigger
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Register trigger HttpTrigger(node_id=1aa7d9e7-a4b2-4702-b66a-50e30f6fca34)
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2b8f4e950>(AWEL_trigger_route__examples_simple_history_multi_round_chat_completions), endpoint: /examples/simple_history/multi_round/chat/completions, methods: ['POST']
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_history/multi_round/chat/completions
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Include router <fastapi.routing.APIRouter object at 0x2cf4104f0> to prefix path /api/v1/awel/trigger
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Register trigger HttpTrigger(node_id=f01bb523-698b-4e52-8dea-8c07598375c3)
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2b8f4f400>(AWEL_trigger_route__examples_simple_client_chat_completions), endpoint: /examples/simple_client/chat/completions, methods: ['POST']
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_client/chat/completions
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Include router <fastapi.routing.APIRouter object at 0x2cf4104f0> to prefix path /api/v1/awel/trigger
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Register trigger HttpTrigger(node_id=7c282070-b102-467d-bb7d-82d813249f4b)
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2b8f94040>(AWEL_trigger_route__examples_simple_client_count_token), endpoint: /examples/simple_client/count_token, methods: ['POST']
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_client/count_token
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Include router <fastapi.routing.APIRouter object at 0x2cf4104f0> to prefix path /api/v1/awel/trigger
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Register trigger HttpTrigger(node_id=a016ce90-38eb-48ec-9c5e-131f784fcc36)
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2b8f94dc0>(AWEL_trigger_route__examples_data_analyst_copilot), endpoint: /examples/data_analyst/copilot, methods: ['POST']
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.http_trigger[93860] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/data_analyst/copilot
2024-03-19 21:33:59 bogon dbgpt.core.awel.trigger.trigger_manager[93860] INFO Include router <fastapi.routing.APIRouter object at 0x2cf4104f0> to prefix path /api/v1/awel/trigger
2024-03-19 21:33:59 bogon dbgpt.model.cluster.worker.manager[93860] INFO Begin start all worker, apply_req: None
2024-03-19 21:33:59 bogon dbgpt.model.cluster.worker.manager[93860] INFO Apply req: None, apply_func: <function LocalWorkerManager._start_all_worker.<locals>._start_worker at 0x2b8f96680>
2024-03-19 21:33:59 bogon dbgpt.model.cluster.worker.manager[93860] INFO Apply to all workers
INFO:     Application startup complete.
2024-03-19 21:33:59 bogon dbgpt.model.cluster.worker.default_worker[93860] INFO Begin load model, model params: 

=========================== ModelParameters ===========================

model_name: vicuna-13b-v1.5
model_path: /Users/mac/DB-GPT/models/vicuna-13b-v1.5
device: mps
model_type: huggingface
prompt_template: None
max_context_size: 4096
num_gpus: None
max_gpu_memory: None
cpu_offloading: False
load_8bit: True
load_4bit: False
quant_type: nf4
use_double_quant: True
compute_dtype: None
trust_remote_code: True
verbose: False

======================================================================


ERROR:    [Errno 48] error while attempting to bind on address ('0.0.0.0', 5000): address already in use
INFO:     Waiting for application shutdown.
2024-03-19 21:33:59 bogon dbgpt.model.cluster.worker.manager[93860] INFO Stop all workers
2024-03-19 21:33:59 bogon dbgpt.model.cluster.worker.manager[93860] INFO Apply req: None, apply_func: <function LocalWorkerManager._stop_all_worker.<locals>._stop_worker at 0x2b8f96d40>
2024-03-19 21:33:59 bogon dbgpt.model.cluster.worker.manager[93860] INFO Apply to all workers
2024-03-19 21:33:59 bogon dbgpt.model.adapter.loader[93860] WARNING 8-bit quantization and 4-bit quantization just supported by cuda
2024-03-19 21:34:00 bogon dbgpt.model.cluster.worker.default_worker[93860] WARNING Model has been stopped!!
2024-03-19 21:34:00 bogon dbgpt.model.cluster.worker.manager[93860] ERROR Error starting worker manager: model vicuna-13b-v1.5@huggingface(192.168.1.6:5000) start failed, Traceback (most recent call last):
  File "/Users/mac/DB-GPT/dbgpt/model/cluster/worker/manager.py", line 506, in _start_worker
    await self.run_blocking_func(
  File "/Users/mac/DB-GPT/dbgpt/model/cluster/worker/manager.py", line 104, in run_blocking_func
    return await loop.run_in_executor(self.executor, func, *args)
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/mac/DB-GPT/dbgpt/model/cluster/worker/default_worker.py", line 116, in start
    self.model, self.tokenizer = self.ml.loader_with_params(
  File "/Users/mac/DB-GPT/dbgpt/model/adapter/loader.py", line 125, in loader_with_params
    return huggingface_loader(llm_adapter, model_params)
  File "/Users/mac/DB-GPT/dbgpt/model/adapter/loader.py", line 207, in huggingface_loader
    model, tokenizer = llm_adapter.load(model_params.model_path, kwargs)
  File "/Users/mac/DB-GPT/dbgpt/model/adapter/fschat_adapter.py", line 112, in load
    return self._adapter.load_model(model_path, from_pretrained_kwargs)
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/fastchat/model/model_adapter.py", line 686, in load_model
    tokenizer = AutoTokenizer.from_pretrained(
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 825, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2048, in from_pretrained
    return cls._from_pretrained(
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2287, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama.py", line 182, in __init__
    self.sp_model = self.get_spm_processor(kwargs.pop("from_slow", False))
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama.py", line 215, in get_spm_processor
    model = model_pb2.ModelProto.FromString(sp_model)
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/google/protobuf/internal/python_message.py", line 801, in FromString
    message.MergeFromString(s)
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/google/protobuf/internal/python_message.py", line 1128, in MergeFromString
    if self._InternalParse(serialized, 0, length) != length:
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/google/protobuf/internal/python_message.py", line 1181, in InternalParse
    (data, new_pos) = decoder._DecodeUnknownField(
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/google/protobuf/internal/decoder.py", line 965, in _DecodeUnknownField
    raise _DecodeError('Wrong wire type in tag.')
google.protobuf.message.DecodeError: Wrong wire type in tag.

2024-03-19 21:34:00 bogon dbgpt.model.cluster.worker.manager[93860] WARNING Stop worker, ignored exception from deregister_func: Remote request error, error code: 403, error msg: 
2024-03-19 21:34:00 bogon dbgpt.model.cluster.worker.manager[93860] WARNING Stop worker, ignored exception from deregister_func: Remote request error, error code: 403, error msg: 
INFO:     Application shutdown complete.
Exception in thread Thread-1 (_flush_to_storages):
Traceback (most recent call last):
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/mac/DB-GPT/dbgpt/util/tracer/span_storage.py", line 97, in _flush_to_storages
    self.executor.submit(append_and_ignore_error, s, spans_to_write)
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/concurrent/futures/thread.py", line 169, in submit
    raise RuntimeError('cannot schedule new futures after '
RuntimeError: cannot schedule new futures after interpreter shutdown

Additional context

No response

Are you willing to submit PR?

  • [X] Yes I am willing to submit a PR!

ahbon123 avatar Mar 19 '24 13:03 ahbon123

ERROR: [Errno 48] error while attempting to bind on address ('0.0.0.0', 5000): address already in use INFO: Waiting for application shutdown.

Please close the program occupying port 5000 or use the following command to change the port:

python dbgpt/app/dbgpt_server.py --port 5555

Same https://github.com/eosphoros-ai/DB-GPT/issues/1204

fangyinc avatar Mar 19 '24 16:03 fangyinc

Thanks for your help. It generates a new errror:

(dbgpt_env) mac@bogon DB-GPT % python dbgpt/app/dbgpt_server.py --port 5555


=========================== WebServerParameters ===========================

host: 0.0.0.0
port: 5555
daemon: False
controller_addr: None
model_name: vicuna-13b-v1.5
share: False
remote_embedding: False
log_level: INFO
light: False
log_file: dbgpt_webserver.log
tracer_file: dbgpt_webserver_tracer.jsonl
tracer_storage_cls: None
disable_alembic_upgrade: False
awel_dirs: None
default_thread_pool_size: None

======================================================================


2024-03-20 11:47:23 bogon dbgpt.component[6681] INFO Register component with name dbgpt_unified_metadata_db_manager_factory and instance: <dbgpt.storage.metadata.db_factory.UnifiedDBManagerFactory object at 0x1770d53c0>
2024-03-20 11:47:24 bogon dbgpt.component[6681] INFO Register component with name dbgpt_thread_pool_default and instance: <dbgpt.util.executor_utils.DefaultExecutorFactory object at 0x282e623b0>
2024-03-20 11:47:24 bogon dbgpt.component[6681] INFO Register component with name dbgpt_default_scheduler and instance: <dbgpt.app.initialization.scheduler.DefaultScheduler object at 0x282ea0f70>
2024-03-20 11:47:24 bogon dbgpt.component[6681] INFO Register component with name dbgpt_model_controller and instance: <dbgpt.model.cluster.controller.controller.ModelControllerAdapter object at 0x16aa826b0>
2024-03-20 11:47:24 bogon dbgpt.component[6681] INFO Register component with name dbgpt_plugin_hub and instance: <dbgpt.serve.agent.hub.controller.ModulePlugin object at 0x28241eda0>
2024-03-20 11:47:24 bogon dbgpt.component[6681] INFO Register component with name dbgpt_multi_agents and instance: <dbgpt.serve.agent.agents.controller.MultiAgents object at 0x282cf05b0>
2024-03-20 11:47:24 bogon dbgpt.app.initialization.embedding_component[6681] INFO Register local LocalEmbeddingFactory
2024-03-20 11:47:24 bogon dbgpt.model.cluster.worker.embedding_worker[6681] INFO [EmbeddingsModelWorker] Parameters of device is None, use mps
2024-03-20 11:47:24 bogon dbgpt.app.initialization.embedding_component[6681] INFO 

=========================== EmbeddingModelParameters ===========================

model_name: text2vec
model_path: /Users/mac/DB-GPT/models/text2vec-large-chinese
device: mps
normalize_embeddings: None

======================================================================


/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/langchain/embeddings/__init__.py:29: LangChainDeprecationWarning: Importing embeddings from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

`from langchain_community.embeddings import HuggingFaceEmbeddings`.

To install langchain-community run `pip install -U langchain-community`.
  warnings.warn(
2024-03-20 11:47:24 bogon sentence_transformers.SentenceTransformer[6681] INFO Load pretrained SentenceTransformer: /Users/mac/DB-GPT/models/text2vec-large-chinese
2024-03-20 11:47:24 bogon sentence_transformers.SentenceTransformer[6681] WARNING No sentence-transformers model found with name /Users/mac/DB-GPT/models/text2vec-large-chinese. Creating a new one with MEAN pooling.
2024-03-20 11:47:25 bogon dbgpt.component[6681] INFO Register component with name embedding_factory and instance: <dbgpt.app.initialization.embedding_component.LocalEmbeddingFactory object at 0x282ea11e0>
2024-03-20 11:47:25 bogon dbgpt.component[6681] INFO Register component with name dbgpt_model_cache_manager and instance: <dbgpt.storage.cache.manager.LocalCacheManager object at 0x2cfea34c0>
2024-03-20 11:47:25 bogon dbgpt.component[6681] INFO Register component with name dbgpt_awel_trigger_manager and instance: <dbgpt.core.awel.trigger.trigger_manager.DefaultTriggerManager object at 0x2cff6c670>
2024-03-20 11:47:25 bogon dbgpt.component[6681] INFO Register component with name dbgpt_awel_dag_manager and instance: <dbgpt.core.awel.dag.dag_manager.DAGManager object at 0x2cff6ce20>
2024-03-20 11:47:25 bogon dbgpt.component[6681] INFO Register component with name dbgpt_app_editor_service and instance: <dbgpt.app.openapi.api_v1.editor.service.EditorService object at 0x2cff6cf40>
2024-03-20 11:47:25 bogon dbgpt.component[6681] INFO Register component with name dbgpt_serve_prompt and instance: <dbgpt.serve.prompt.serve.Serve object at 0x2cff6cfa0>
2024-03-20 11:47:25 bogon dbgpt.component[6681] INFO Register component with name dbgpt_serve_prompt_service and instance: <dbgpt.serve.prompt.service.service.Service object at 0x2cff6cfd0>
2024-03-20 11:47:25 bogon dbgpt.component[6681] INFO Register component with name dbgpt_serve_conversation and instance: <dbgpt.serve.conversation.serve.Serve object at 0x2cff1e9e0>
2024-03-20 11:47:25 bogon dbgpt.component[6681] INFO Register component with name dbgpt_serve_conversation_service and instance: <dbgpt.serve.conversation.service.service.Service object at 0x2cff1ea70>
2024-03-20 11:47:25 bogon dbgpt.component[6681] INFO Register component with name dbgpt_serve_flow and instance: <dbgpt.serve.flow.serve.Serve object at 0x2cff75960>
2024-03-20 11:47:25 bogon dbgpt.component[6681] INFO Register component with name dbgpt_dbgpts_loader and instance: <dbgpt.util.dbgpts.loader.DBGPTsLoader object at 0x30a0fc850>
2024-03-20 11:47:25 bogon dbgpt.component[6681] INFO Register component with name dbgpt_serve_flow_service and instance: <dbgpt.serve.flow.service.service.Service object at 0x2cff759f0>
2024-03-20 11:47:25 bogon dbgpt.util._db_migration_utils[6681] WARNING Initialize and upgrade database metadata with alembic, just run this in your development environment, if you deploy this in production environment, please run webserver with --disable_alembic_upgrade(`python dbgpt/app/dbgpt_server.py --disable_alembic_upgrade`).
we suggest you to use `dbgpt db migration` to initialize and upgrade database metadata with alembic, your can run `dbgpt db migration --help` to get more information.
2024-03-20 11:47:25 bogon alembic.runtime.migration[6681] INFO Context impl SQLiteImpl.
2024-03-20 11:47:25 bogon alembic.runtime.migration[6681] INFO Will assume non-transactional DDL.
2024-03-20 11:47:25 bogon dbgpt.util._db_migration_utils[6681] INFO Migration versions and their file paths:
========================================Migration versions========================================

363f238e9d27 (current): New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/363f238e9d27_new_migration.py)
542211b93c27 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/542211b93c27_new_migration.py)
3ed3251d42b2 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/3ed3251d42b2_new_migration.py)
a97e0270e22a : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/a97e0270e22a_new_migration.py)
01065a264f4e : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/01065a264f4e_new_migration.py)
527dff78863a : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/527dff78863a_new_migration.py)
71ba6e0f49ab : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/71ba6e0f49ab_new_migration.py)
ec79bec3957e : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/ec79bec3957e_new_migration.py)
4d7311c48c8f : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/4d7311c48c8f_new_migration.py)
e3c187c43814 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/e3c187c43814_new_migration.py)
f6d6ef43be3e : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/f6d6ef43be3e_new_migration.py)
0e1a06aa60c1 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/0e1a06aa60c1_new_migration.py)
8cc51efce6a6 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/8cc51efce6a6_new_migration.py)
4107504e1bd2 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/4107504e1bd2_new_migration.py)
fea0b593936c : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/fea0b593936c_new_migration.py)
3fe7807e6b3f : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/3fe7807e6b3f_new_migration.py)
757290b390fa : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/757290b390fa_new_migration.py)
3251cabd398d : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/3251cabd398d_new_migration.py)
8103c22aedbc : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/8103c22aedbc_new_migration.py)
df4fa22f23f2 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/df4fa22f23f2_new_migration.py)
edf1abe898ff : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/edf1abe898ff_new_migration.py)
f350d48665ab : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/f350d48665ab_new_migration.py)
6e740641ba46 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/6e740641ba46_new_migration.py)
c76c81eadc26 : New migration (Path: /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/c76c81eadc26_new_migration.py)
==========================================================================================
2024-03-20 11:47:25 bogon alembic.runtime.migration[6681] INFO Context impl SQLiteImpl.
2024-03-20 11:47:25 bogon alembic.runtime.migration[6681] INFO Will assume non-transactional DDL.
2024-03-20 11:47:25 bogon alembic.runtime.migration[6681] INFO Context impl SQLiteImpl.
2024-03-20 11:47:25 bogon alembic.runtime.migration[6681] INFO Will assume non-transactional DDL.
2024-03-20 11:47:25 bogon dbgpt.util._db_migration_utils[6681] INFO alembic migration current revision: 363f238e9d27, latest revision: 363f238e9d27
2024-03-20 11:47:25 bogon alembic.runtime.migration[6681] INFO Context impl SQLiteImpl.
2024-03-20 11:47:25 bogon alembic.runtime.migration[6681] INFO Will assume non-transactional DDL.
  Generating /Users/mac/DB-GPT/pilot/meta_data/alembic/versions/3ca5d5390cb8_new_migration.py ...  done
2024-03-20 11:47:25 bogon alembic.runtime.migration[6681] INFO Context impl SQLiteImpl.
2024-03-20 11:47:25 bogon alembic.runtime.migration[6681] INFO Will assume non-transactional DDL.
2024-03-20 11:47:25 bogon alembic.runtime.migration[6681] INFO Running upgrade 363f238e9d27 -> 3ca5d5390cb8, New migration
Model Unified Deployment Mode!
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.manager[6681] INFO Worker params: 

=========================== ModelWorkerParameters ===========================

model_name: vicuna-13b-v1.5
model_path: /Users/mac/DB-GPT/models/vicuna-13b-v1.5
worker_type: None
model_alias: None
worker_class: None
model_type: huggingface
host: 0.0.0.0
port: 5555
daemon: False
limit_model_concurrency: 5
standalone: True
register: True
worker_register_host: None
controller_addr: None
send_heartbeat: True
heartbeat_interval: 20
log_level: None
log_file: dbgpt_model_worker_manager.log
tracer_file: dbgpt_model_worker_manager_tracer.jsonl
tracer_storage_cls: None

======================================================================


2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.manager[6681] INFO Run WorkerManager with standalone mode, controller_addr: http://127.0.0.1:5555
2024-03-20 11:47:25 bogon dbgpt.model.adapter.model_adapter[6681] INFO Use fastcat adapter
Warning: Loading base MPT model with `zero_shot` conversation configuration.  If this is not desired, inspect model configurations and names.
2024-03-20 11:47:25 bogon dbgpt.model.adapter.fschat_adapter[6681] INFO Found llm model adapter with model name: vicuna-13b-v1.5, <fastchat.model.model_adapter.VicunaAdapter object at 0x2d07b7fa0>
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.default_worker[6681] INFO model_name: vicuna-13b-v1.5, model_path: /Users/mac/DB-GPT/models/vicuna-13b-v1.5, model_param_class: <class 'dbgpt.model.parameter.ModelParameters'>
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.default_worker[6681] INFO [DefaultModelWorker] Parameters of device is None, use mps
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.manager[6681] INFO Init empty instances list for vicuna-13b-v1.5@llm
2024-03-20 11:47:25 bogon dbgpt.component[6681] INFO Register component with name dbgpt_worker_manager_factory and instance: <dbgpt.model.cluster.worker.manager._DefaultWorkerManagerFactory object at 0x2d23a8280>
2024-03-20 11:47:25 bogon dbgpt.util.dbgpts.loader[6681] INFO Found 0 dbgpts packages from /Users/mac/.dbgpts/packages/be6ccebca05e56cbc6376891fa5bde08
INFO:     Started server process [6681]
INFO:     Waiting for application startup.
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_rag_rewrite_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Found dag DAG(dag_id=dbgpt_awel_simple_rag_rewrite_example) from mod <module 'unusual_prefix_05eb0f06d6bf45e5e4db9df22468546eca159645_simple_rag_rewrite_example' from '/Users/mac/DB-GPT/examples/awel/simple_rag_rewrite_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_rag_rewrite_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Found dag DAG(dag_id=simple_nl_schema_sql_chart_example) from mod <module 'unusual_prefix_3b7f8b6d07e23e1dc8539846ebfcef24da736d3e_simple_nl_schema_sql_chart_example' from '/Users/mac/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_chat_dag_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Found dag DAG(dag_id=dbgpt_awel_simple_dag_example) from mod <module 'unusual_prefix_0bf46b05e5219d64d0b7f09daf5ba07ef5312bfb_simple_chat_dag_example' from '/Users/mac/DB-GPT/examples/awel/simple_chat_dag_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_chat_dag_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_dag_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Found dag DAG(dag_id=simple_dag_example) from mod <module 'unusual_prefix_3ca65816aa3e0a5a3c89277cad0babff3626add2_simple_dag_example' from '/Users/mac/DB-GPT/examples/awel/simple_dag_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_dag_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_rag_summary_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Found dag DAG(dag_id=dbgpt_awel_simple_rag_summary_example) from mod <module 'unusual_prefix_c049f97a0c99ff4dd86982980daa40c02fd8f747_simple_rag_summary_example' from '/Users/mac/DB-GPT/examples/awel/simple_rag_summary_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_rag_summary_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_chat_history_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Found dag DAG(dag_id=dbgpt_awel_simple_chat_history) from mod <module 'unusual_prefix_54a4381cd27e4033b94ed9a9c4d606fd5a0da530_simple_chat_history_example' from '/Users/mac/DB-GPT/examples/awel/simple_chat_history_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_chat_history_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Importing /Users/mac/DB-GPT/examples/awel/simple_llm_client_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Found dag DAG(dag_id=dbgpt_awel_simple_llm_client_generate) from mod <module 'unusual_prefix_4a0a98a7a84f1ba3faefba5929ad567a532f8d19_simple_llm_client_example' from '/Users/mac/DB-GPT/examples/awel/simple_llm_client_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_llm_client_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Found dag DAG(dag_id=dbgpt_awel_simple_llm_client_count_token) from mod <module 'unusual_prefix_4a0a98a7a84f1ba3faefba5929ad567a532f8d19_simple_llm_client_example' from '/Users/mac/DB-GPT/examples/awel/simple_llm_client_example.py'> and model file /Users/mac/DB-GPT/examples/awel/simple_llm_client_example.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Importing /Users/mac/DB-GPT/examples/awel/data_analyst_assistant.py
2024-03-20 11:47:25 bogon dbgpt.core.interface.operators.message_operator[6681] INFO Storage is not set, use the InMemoryStorage as the conversation storage.
2024-03-20 11:47:25 bogon dbgpt.core.interface.operators.message_operator[6681] INFO Message storage is not set, use the InMemoryStorage as the message 
2024-03-20 11:47:25 bogon dbgpt.core.awel.dag.loader[6681] INFO Found dag DAG(dag_id=dbgpt_awel_data_analyst_assistant) from mod <module 'unusual_prefix_443fd604a8960e2154c906db820757ec146a0e94_data_analyst_assistant' from '/Users/mac/DB-GPT/examples/awel/data_analyst_assistant.py'> and model file /Users/mac/DB-GPT/examples/awel/data_analyst_assistant.py
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Register trigger HttpTrigger(node_id=6c662225-be8f-435f-92ac-ec42724a8265)
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2d43ec790>(AWEL_trigger_route__examples_rag_rewrite), endpoint: /examples/rag/rewrite, methods: ['POST']
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/rag/rewrite
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Include router <fastapi.routing.APIRouter object at 0x2cff6c820> to prefix path /api/v1/awel/trigger
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Register trigger HttpTrigger(node_id=369a26ca-1a81-44c0-a9e4-3584762315b0)
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2d43ecb80>(AWEL_trigger_route__examples_rag_schema_linking), endpoint: /examples/rag/schema_linking, methods: ['POST']
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/rag/schema_linking
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Include router <fastapi.routing.APIRouter object at 0x2cff6c820> to prefix path /api/v1/awel/trigger
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Register trigger HttpTrigger(node_id=f0edfb55-289e-4395-b8d5-1d241e78d61e)
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2d43ed120>(AWEL_trigger_route__examples_simple_chat), endpoint: /examples/simple_chat, methods: ['POST']
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_chat
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Include router <fastapi.routing.APIRouter object at 0x2cff6c820> to prefix path /api/v1/awel/trigger
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Register trigger HttpTrigger(node_id=d70af1da-502f-442e-a710-b71a9b889c9a)
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function_get at 0x2d43ed7e0>(AWEL_trigger_route__examples_hello), endpoint: /examples/hello, methods: ['GET']
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/hello
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Include router <fastapi.routing.APIRouter object at 0x2cff6c820> to prefix path /api/v1/awel/trigger
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Register trigger HttpTrigger(node_id=30aa488b-a9b1-4cef-a0e8-c9aead28ea56)
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2d43edfc0>(AWEL_trigger_route__examples_rag_summary), endpoint: /examples/rag/summary, methods: ['POST']
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/rag/summary
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Include router <fastapi.routing.APIRouter object at 0x2cff6c820> to prefix path /api/v1/awel/trigger
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Register trigger HttpTrigger(node_id=1ffb5111-7c0e-4de3-9f00-31d910aaddf4)
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2d43ee950>(AWEL_trigger_route__examples_simple_history_multi_round_chat_completions), endpoint: /examples/simple_history/multi_round/chat/completions, methods: ['POST']
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_history/multi_round/chat/completions
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Include router <fastapi.routing.APIRouter object at 0x2cff6c820> to prefix path /api/v1/awel/trigger
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Register trigger HttpTrigger(node_id=d3a55011-e798-4b17-b5a9-ec5b2b1ded25)
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2d43ef400>(AWEL_trigger_route__examples_simple_client_chat_completions), endpoint: /examples/simple_client/chat/completions, methods: ['POST']
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_client/chat/completions
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Include router <fastapi.routing.APIRouter object at 0x2cff6c820> to prefix path /api/v1/awel/trigger
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Register trigger HttpTrigger(node_id=20afb214-4e13-40d3-b83e-140efeaaa288)
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2d4634040>(AWEL_trigger_route__examples_simple_client_count_token), endpoint: /examples/simple_client/count_token, methods: ['POST']
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_client/count_token
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Include router <fastapi.routing.APIRouter object at 0x2cff6c820> to prefix path /api/v1/awel/trigger
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Register trigger HttpTrigger(node_id=4827d804-47c0-475c-8569-8b621a6a41be)
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO mount router function <function HttpTrigger._create_route_func.<locals>.create_route_function.<locals>.route_function at 0x2d4634dc0>(AWEL_trigger_route__examples_data_analyst_copilot), endpoint: /examples/data_analyst/copilot, methods: ['POST']
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.http_trigger[6681] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/data_analyst/copilot
2024-03-20 11:47:25 bogon dbgpt.core.awel.trigger.trigger_manager[6681] INFO Include router <fastapi.routing.APIRouter object at 0x2cff6c820> to prefix path /api/v1/awel/trigger
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.manager[6681] INFO Begin start all worker, apply_req: None
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.manager[6681] INFO Apply req: None, apply_func: <function LocalWorkerManager._start_all_worker.<locals>._start_worker at 0x2d4636680>
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.manager[6681] INFO Apply to all workers
INFO:     Application startup complete.
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.default_worker[6681] INFO Begin load model, model params: 

=========================== ModelParameters ===========================

model_name: vicuna-13b-v1.5
model_path: /Users/mac/DB-GPT/models/vicuna-13b-v1.5
device: mps
model_type: huggingface
prompt_template: None
max_context_size: 4096
num_gpus: None
max_gpu_memory: None
cpu_offloading: False
load_8bit: True
load_4bit: False
quant_type: nf4
use_double_quant: True
compute_dtype: None
trust_remote_code: True
verbose: False

======================================================================


INFO:     Uvicorn running on http://0.0.0.0:5555 (Press CTRL+C to quit)
2024-03-20 11:47:25 bogon dbgpt.model.adapter.loader[6681] WARNING 8-bit quantization and 4-bit quantization just supported by cuda
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.manager[6681] ERROR Error starting worker manager: model vicuna-13b-v1.5@huggingface(172.16.40.133:5555) start failed, Traceback (most recent call last):
  File "/Users/mac/DB-GPT/dbgpt/model/cluster/worker/manager.py", line 506, in _start_worker
    await self.run_blocking_func(
  File "/Users/mac/DB-GPT/dbgpt/model/cluster/worker/manager.py", line 104, in run_blocking_func
    return await loop.run_in_executor(self.executor, func, *args)
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/mac/DB-GPT/dbgpt/model/cluster/worker/default_worker.py", line 116, in start
    self.model, self.tokenizer = self.ml.loader_with_params(
  File "/Users/mac/DB-GPT/dbgpt/model/adapter/loader.py", line 125, in loader_with_params
    return huggingface_loader(llm_adapter, model_params)
  File "/Users/mac/DB-GPT/dbgpt/model/adapter/loader.py", line 207, in huggingface_loader
    model, tokenizer = llm_adapter.load(model_params.model_path, kwargs)
  File "/Users/mac/DB-GPT/dbgpt/model/adapter/fschat_adapter.py", line 112, in load
    return self._adapter.load_model(model_path, from_pretrained_kwargs)
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/fastchat/model/model_adapter.py", line 686, in load_model
    tokenizer = AutoTokenizer.from_pretrained(
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 825, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2048, in from_pretrained
    return cls._from_pretrained(
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2287, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama.py", line 182, in __init__
    self.sp_model = self.get_spm_processor(kwargs.pop("from_slow", False))
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama.py", line 215, in get_spm_processor
    model = model_pb2.ModelProto.FromString(sp_model)
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/google/protobuf/internal/python_message.py", line 801, in FromString
    message.MergeFromString(s)
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/google/protobuf/internal/python_message.py", line 1128, in MergeFromString
    if self._InternalParse(serialized, 0, length) != length:
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/google/protobuf/internal/python_message.py", line 1181, in InternalParse
    (data, new_pos) = decoder._DecodeUnknownField(
  File "/Users/mac/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/google/protobuf/internal/decoder.py", line 965, in _DecodeUnknownField
    raise _DecodeError('Wrong wire type in tag.')
google.protobuf.message.DecodeError: Wrong wire type in tag.

INFO:     Shutting down
INFO:     Waiting for application shutdown.
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.manager[6681] INFO Stop all workers
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.manager[6681] INFO Apply req: None, apply_func: <function LocalWorkerManager._stop_all_worker.<locals>._stop_worker at 0x2d4636710>
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.manager[6681] INFO Apply to all workers
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.default_worker[6681] WARNING Model has been stopped!!
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.manager[6681] WARNING Stop worker, ignored exception from deregister_func: Remote request error, error code: 503, error msg: Unable to round-trip http request to upstream: dial tcp 127.0.0.1:5555: connect: connection refused
2024-03-20 11:47:25 bogon dbgpt.model.cluster.worker.manager[6681] WARNING Stop worker, ignored exception from deregister_func: Remote request error, error code: 503, error msg: Unable to round-trip http request to upstream: dial tcp 127.0.0.1:5555: connect: connection refused
INFO:     Application shutdown complete.
INFO:     Finished server process [6681]

ahbon123 avatar Mar 20 '24 03:03 ahbon123

@ahbon123 Please make sure your model download is complete.

fangyinc avatar Mar 20 '24 04:03 fangyinc

I have downloaded the three models below, is there anything else I need to download?

(base) mac@bogon models % ls
ggml-gpt4all-j-v1.3-groovy.bin	text2vec-large-chinese		vicuna-13b-v1.5

ahbon123 avatar Mar 20 '24 05:03 ahbon123

Check whether the model download is complete:

ls -lh models/vicuna-13b-v1.5/

fangyinc avatar Mar 21 '24 07:03 fangyinc

Here is the result:

(base) mac@bogon DB-GPT % ls -lh models/vicuna-13b-v1.5/
total 50843648
-rw-r--r--  1 mac  staff   1.9K  3 15 16:55 README.md
-rw-r--r--  1 mac  staff   638B  3 15 16:55 config.json
-rw-r--r--  1 mac  staff    73B  3 15 16:55 configuration.json
-rw-r--r--  1 mac  staff   192B  3 15 16:55 generation_config.json
-rw-r--r--@ 1 mac  staff   9.3G  3 15 22:13 pytorch_model-00001-of-00003.bin
-rw-r--r--@ 1 mac  staff   9.2G  3 15 22:13 pytorch_model-00002-of-00003.bin
-rw-r--r--@ 1 mac  staff   5.8G  3 15 19:10 pytorch_model-00003-of-00003.bin
-rw-r--r--  1 mac  staff    33K  3 15 16:55 pytorch_model.bin.index.json
-rw-r--r--  1 mac  staff   438B  3 15 16:55 special_tokens_map.json
-rw-r--r--  1 mac  staff   131B  3 15 16:55 tokenizer.model
-rw-r--r--  1 mac  staff   749B  3 15 16:55 tokenizer_config.json

(base) mac@bogon DB-GPT % ls -lh models/text2vec-large-chinese 
total 5087776
drwxr-xr-x@ 3 mac  staff    96B  3 17 12:22 Qwen
-rw-r--r--  1 mac  staff   901B  3 15 16:26 README.md
drwxr-xr-x@ 5 mac  staff   160B  3 15 19:40 archive
-rw-r--r--  1 mac  staff   821B  3 15 16:26 config.json
-rw-r--r--  1 mac  staff   215B  3 15 16:26 configuration.json
-rw-r--r--  1 mac  staff    69B  3 15 16:26 eval_results.txt
-rw-r--r--@ 1 mac  staff   1.2G  3 15 16:45 model.safetensors
-rw-r--r--@ 1 mac  staff   1.2G  3 15 16:46 pytorch_model.bin
-rw-r--r--  1 mac  staff   125B  3 15 16:26 special_tokens_map.json
-rw-r--r--  1 mac  staff   429K  3 15 16:26 tokenizer.json
-rw-r--r--  1 mac  staff   514B  3 15 16:26 tokenizer_config.json
-rw-r--r--  1 mac  staff   107K  3 15 16:26 vocab.txt

ahbon123 avatar Mar 22 '24 01:03 ahbon123

I got the same issue with local model 'vicuna-13b-v1.5'. I confirm it have nothing with embedding model as it's success to setup with openai proxy.

Environment: Macbook with M1

 vicuna-13b-v1.5 % ls -lh -a
-rw-r--r--   1 figodeng  staff   1.5K Mar 28 18:44 .gitattributes
-rw-r--r--   1 figodeng  staff   1.9K Mar 28 18:44 README.md
-rw-r--r--   1 figodeng  staff   638B Mar 28 18:44 config.json
-rw-r--r--   1 figodeng  staff   192B Mar 28 18:44 generation_config.json
-rw-r--r--@  1 figodeng  staff   9.3G Mar 28 18:53 pytorch_model-00001-of-00003.bin
-rw-r--r--@  1 figodeng  staff   9.2G Mar 28 18:52 pytorch_model-00002-of-00003.bin
-rw-r--r--@  1 figodeng  staff   5.8G Mar 28 18:50 pytorch_model-00003-of-00003.bin
-rw-r--r--   1 figodeng  staff    33K Mar 28 18:44 pytorch_model.bin.index.json
-rw-r--r--   1 figodeng  staff   438B Mar 28 18:44 special_tokens_map.json
-rw-r--r--   1 figodeng  staff   131B Mar 28 18:44 tokenizer.model
-rw-r--r--   1 figodeng  staff   749B Mar 28 18:44 tokenizer_config.json


INFO:     Uvicorn running on http://0.0.0.0:5000 (Press CTRL+C to quit)
2024-03-29 10:36:15 dbgpt.model.adapter.loader[13499] WARNING 8-bit quantization and 4-bit quantization just supported by cuda
2024-03-29 10:36:15 dbgpt.model.cluster.worker.manager[13499] ERROR Error starting worker manager: model vicuna-13b-v1.5@huggingface(192.168.219.144:5000) start failed, Traceback (most recent call last):
  File "/Users/figodeng/hkpc/DB-GPT/dbgpt/model/cluster/worker/manager.py", line 506, in _start_worker
    await self.run_blocking_func(
  File "/Users/figodeng/hkpc/DB-GPT/dbgpt/model/cluster/worker/manager.py", line 104, in run_blocking_func
    return await loop.run_in_executor(self.executor, func, *args)
  File "/Users/figodeng/anaconda3/envs/dbgpt_env/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/figodeng/hkpc/DB-GPT/dbgpt/model/cluster/worker/default_worker.py", line 116, in start
    self.model, self.tokenizer = self.ml.loader_with_params(
  File "/Users/figodeng/hkpc/DB-GPT/dbgpt/model/adapter/loader.py", line 125, in loader_with_params
    return huggingface_loader(llm_adapter, model_params)
  File "/Users/figodeng/hkpc/DB-GPT/dbgpt/model/adapter/loader.py", line 207, in huggingface_loader
    model, tokenizer = llm_adapter.load(model_params.model_path, kwargs)
  File "/Users/figodeng/hkpc/DB-GPT/dbgpt/model/adapter/fschat_adapter.py", line 112, in load
    return self._adapter.load_model(model_path, from_pretrained_kwargs)
  File "/Users/figodeng/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/fastchat/model/model_adapter.py", line 686, in load_model
    tokenizer = AutoTokenizer.from_pretrained(
  File "/Users/figodeng/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 837, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/Users/figodeng/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2086, in from_pretrained
    return cls._from_pretrained(
  File "/Users/figodeng/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2325, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/Users/figodeng/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama.py", line 182, in __init__
    self.sp_model = self.get_spm_processor(kwargs.pop("from_slow", False))
  File "/Users/figodeng/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/llama/tokenization_llama.py", line 215, in get_spm_processor
    model = model_pb2.ModelProto.FromString(sp_model)
  File "/Users/figodeng/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/google/protobuf/internal/python_message.py", line 801, in FromString
    message.MergeFromString(s)
  File "/Users/figodeng/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/google/protobuf/internal/python_message.py", line 1128, in MergeFromString
    if self._InternalParse(serialized, 0, length) != length:
  File "/Users/figodeng/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/google/protobuf/internal/python_message.py", line 1181, in InternalParse
    (data, new_pos) = decoder._DecodeUnknownField(
  File "/Users/figodeng/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/google/protobuf/internal/decoder.py", line 965, in _DecodeUnknownField
    raise _DecodeError('Wrong wire type in tag.')
google.protobuf.message.DecodeError: Wrong wire type in tag.

INFO:     Shutting down

Any update about this issue?

figodeng avatar Mar 29 '24 02:03 figodeng

This issue has been marked as stale, because it has been over 30 days without any activity.

github-actions[bot] avatar Apr 28 '24 21:04 github-actions[bot]

This issue bas been closed, because it has been marked as stale and there has been no activity for over 7 days.

github-actions[bot] avatar May 23 '24 21:05 github-actions[bot]