EXTREME WARNING: Not enough space to even fit in even the test example

[oan-J]

Hi, when I was running ./reproduce.sh ircot flan-t5-base hotpotqa, I faced a warning:

Token indices sequence length is longer than the specified maximum sequence length for this model (555 > 512). Running this sequence through the model will result in indexing errors Running inference on examples 0it [00:00, ?it/s]EXTREME WARNING: Not enough space to even fit in even the test example. EXTREME WARNING: Not enough space to even fit in even the test example. EXTREME WARNING: Not enough space to even fit in even the test example. EXTREME WARNING: Not enough space to even fit in even the test example. EXTREME WARNING: Not enough space to even fit in even the test example. EXTREME WARNING: Not enough space to even fit in even the test example. 1it [03:27, 207.97s/it] ...

I am not sure if this is right, please let me know if there's anything I need to fix.

These information might be relevant, so I put it here:

1. I changed retriever_server port Instead of uvicorn serve:app --port 8000 --app-dir retriever_server, I changed my port to 9201 since port 8000 was used I ran :uvicorn serve:app --port 9201 --app-dir retriever_server Also, I made these changes: In predict.py and run.py, I set the env_variables["RETRIEVER_PORT"] to 9201, since str(retriever_address["port"]) can't get the right port:

retriever_address = get_retriever_address() print("[here]retriever_address",retriever_address) env_variables["RETRIEVER_HOST"] = str(retriever_address["host"]) # env_variables["RETRIEVER_PORT"] = str(retriever_address["port"]) env_variables["RETRIEVER_PORT"] = str("9201") print("[here][env_variables['RETRIEVER_PORT']]",env_variables["RETRIEVER_PORT"])

2. I was using bf16 Since I got CUDA Out of Memory, I ran MODEL_NAME=flan-t5-base-bf16 RETRIEVER_PORT=9201 /mnt/.conda/envs/ircot/bin/uvicorn serve:app --port 8010 --app-dir llm_server. Also, I changed base_configs/ircot_flan_t5_base_hotpotqa.jsonnet:

"model_tokens_limit": 1000,

4. About the localhost (I feel like sth. is wrong about the outputs but I am not so sure) First, I started elasticsearch and I got these on http://127.0.0.1:9200

{ "name" : "dell-PowerEdge-T640", "cluster_name" : "elasticsearch", "cluster_uuid" : "i1NX0dODQ3qWEUBxhfl9Ig", "version" : { "number" : "7.10.2", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "747e1cc71def077253878a59143c1f785afa92b9", "build_date" : "2021-01-13T00:42:12.435326Z", "build_snapshot" : false, "lucene_version" : "8.7.0", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" }

Second, I started retriever_server, and I got these on http://127.0.0.1:9201

{"message":"Hello! This is a retriever server."} and on http://127.0.0.1:9201/retrieve/ {"detail":"Method Not Allowed"}

Third, I started MODEL_NAME=flan-t5-base-bf16 RETRIEVER_PORT=9201 /mnt/.conda/envs/ircot/bin/uvicorn serve:app --port 8010 --app-dir llm_server, and I got these on http://127.0.0.1:8010/

{"message":"Hello! This is a server for flan-t5-base-bf16. Go to /generate/ for generation requests."} and on http://127.0.0.1:8010/generate/ {"detail":[{"type":"missing","loc":["query","prompt"],"msg":"Field required","input":null,"url":"https://errors.pydantic.dev/2.7/v/missing"}]}

Thank you in advance!