any tutorial on how to use Infinity?

[abcnow]

trafficstars

Love the concept behind infinity! I wonder if you have a video tutorial or pdf about how to use Infinity? It will be great!

[michaelfeil]

Thanks for the creative suggestion. For now, there is no plan to create a video from it. As of the Readme.md, in what way is it not clear to you how to use infinity, what do you think should be improved?

[Jawn78]

@michaelfeil, some examples of how to load a model would be nice. Examples in general, are nice in the readme.

[michaelfeil]

@Jawn78 Knowing all components inside out doesnt make it easy to me to see which examples are missing?

Are startup commands missing? / more commands? Should I host the Swagger UI on github.io?

[BSVogler]

Just some feedback regarding documentation/newbiews on the project. I installed it today and had three difficulties:

• How do I add new models to the web server? When I use something other than the default model I get a 404

underlying_embeddings: Embeddings = InfinityEmbeddings(model=model, infinity_api_url="http://0.0.0.0:7997/v1")

• What is the default port when launching via infinity_emb? The main page only at the end mentions 8080 but I figured it is indeed 7997.

• How to launch it via python. Running what I found in the docs

from infinity_emb import create_server
fastapi_app = create_server()

exits with status 0 immediately. Maybe something is missing.

[michaelfeil]

Love your feedback!

Port:

Should I switch the default port to 8080? Or is 7997 fine and just make it a bit more clear in the docs?

Python

The create_server util is really just a replacement for what the CLI does. It will launch a uvicorn server.

I would recommend using the following: Install with all extras.

pip install "infinity_emb[all]==0.0.x"

from infinity_emb import AsyncEmbeddingEngine, transformer

sentences = ["Embedded this is sentence via Infinity.", "Paris is in France."]
engine = AsyncEmbeddingEngine(engine="torch")
async with engine:
    embeddings, usage = await engine.embed(sentences)
    embeddings = np.array(embeddings)

You can find possible usage here: https://github.com/michaelfeil/infinity/blob/main/libs/infinity_emb/tests/unit_test/test_engine.py

[BSVogler]

I guess 7997 is fine when it is mentioned more prominently in the docs.

A uvicorn server should not exit immediately so I think something must be wrong in what I did.

[michaelfeil]

Added some chances to the docs.

[michaelwechner]

Unfortunately I also have trouble installing / running infinity on my M1 MacBook Pro.

I was assuming that I could access the Swagger documentation at http://localhost:7997, but that does not seem to be the case. Do I misunderstand something?

[michaelfeil]

@michaelwechner Sounds like youre doing the correct thing. What is the issue you’re seeing?

[michaelwechner]

The containter got created, resp. I can see it with "docker ps -a", but when I send a request to http://localhost:7997/, then it does not seem to be running at all.

[michaelfeil]

Are you forwarding port 7997 from docker, as in the readme?

docker run -it --gpus all -p 7997:7997 michaelf34/infinity:0.0.26 --model-name-or-path BAAI/bge-small-en-v1.5 --port 7997