llama-stack

llama-stack copied to clipboard

# Context - Dell TGI's endpoint ([registry.dell.huggingface.co/enterprise-dell-inference-meta-llama-meta-llama-3.1-8b-instruct](http://registry.dell.huggingface.co/enterprise-dell-inference-meta-llama-meta-llama-3.1-8b-instruct)) ``` {'model_id': '/model', 'model_sha': None, 'model_dtype': 'torch.float16', 'model_device_type': 'cuda', 'model_pipeline_tag': None, ...} ``` - Official TGI ([ghcr.io/huggingface/text-generation-inference:latest](http://ghcr.io/huggingface/text-generation-inference:latest)) ``` {'model_id': 'meta-llama/Llama-3.1-8B-Instruct', 'model_sha': '0e9e39f249a16976918f6564b8830bc894c89659', ...}...

yanxi0830

Feature/add platform to docker

2

Now you can specify ```platform``` param in the cmd args `llama stack build --template local --image-type docker --platform "linux/amd64" --name llama-stack` resolves #253

picografix

Cerebras Inference Integration

4

Adding Cerebras Inference as an API provider. It looks like the other providers use the [legacy OpenAI API ](https://platform.openai.com/docs/guides/completions) but we prefer the [new chat completion API](https://platform.openai.com/docs/guides/text-generation). As a result...

henrytwo

Added implementations for get_agents_session, delete_agents_session and delete_agents

1

Added implementations for not implemented methods for agents

cheesecake100201

Create a remote memory provider for pinecone

2

Follow the implementation of weaviate: https://github.com/meta-llama/llama-stack/tree/main/llama_stack/providers/adapters/memory/weaviate

raghotham

llama stack distributions / templates / docker refactor

2

# Test #### --list-templates  #### ollama docker ``` llama-stack/llama_stack/distribution/docker/ollama$ ls compose.yaml ollama-run.yaml llama-stack/llama_stack/distribution/docker/ollama$ docker compose up ```

yanxi0830

Rename .github/workflows/pre-commit.yml to pip install llama-stack

2

Kate457

Added tests for persistence

1

- Changed return for resolve_impls_for_test in resolver.py to get the persistence store object and had to make changes in other test files to avoid key_error. delete_agent_and _session and get_agent_turn_and_steps tests...

cheesecake100201

``` $ llama download --source meta --model-id Llama3.2-3B-Instruct --meta-url "https://llama3-2-lightweight.llamameta.net/*?some_stuff_I_do_not_know_if_it_is_safe_to_make_public_so_replacing_with_this_phrase&Download-Request-ID=1259948918685929" Downloading `checklist.chk`... Already downloaded `C:\Users\whiteSkar\.llama\checkpoints\Llama3.2-3B-Instruct\checklist.chk`, skipping... Downloading `tokenizer.model`... Already downloaded `C:\Users\whiteSkar\.llama\checkpoints\Llama3.2-3B-Instruct\tokenizer.model`, skipping... Downloading `params.json`... Already downloaded `C:\Users\whiteSkar\.llama\checkpoints\Llama3.2-3B-Instruct\params.json`, skipping... Downloading `consolidated.00.pth`......

whiteSkar

Getting built here for now: https://llama-stack.readthedocs.org/

raghotham