databricks-ml-examples
databricks-ml-examples copied to clipboard
databricks
In fine_tune_deepspeed.py, the first part of the load_training_dataset function looks like this: ``` def load_training_dataset( tokenizer, path_or_dataset: str = DEFAULT_TRAINING_DATASET, max_seq_len: int = 256, seed: int = DEFAULT_SEED, ) ->...
WIP: 1. Added troubleshooting guide 2. Added comments on all DBFS locations to introduce UC Volumes Note: We might need to wait until after MLR 14.3 to submit these PRs.
Hello, first, thanks a lot for this amazing repo, it's helping us a lot with our LLM infra. One thing though is that I'm not seeing much examples with Databricks...
Tested on: https://adb-7064161269814046.2.staging.azuredatabricks.net/?o=7064161269814046#notebook/94670986903573/command/94670986903574
I have a Standard_NC48ads_A100_v4 cluster (same runtime as the sample code: 13.2 GPU ML Runtime) with 2 A100-80G GPUs. I copied the sample code and changed device_map="cuda:0" to device_map="auto" for...
Currently, all storage directories are hardcoded. I’d like to change these to parameters to avoid collisions. I have it in work, are you accepting pull requests?
Hi, I have seen your example llm-models/llamav2/llamav2-7b/02_mlflow_logging_inference.py , when you create the endpoint, you add headers, is there a way to add CORS here? I have created an endpoint for...
Hello, I know this is a bit old: llm-models/mistral/mistral-7b/01_b_load_inference_vllm.py, but I'm looking for example code snippets with VLLM working on DBR/MLR. Your markdown refers to MLR 14.0 GPU, which is...
## Issue When calling the llama guard 2 model `import mlflow import mlflow.deployments import re def query_llamaguard(chat, unsafe_categories=DEFAULT_UNSAFE_CATEGORY): """ Queries LlamaGuard model to check if the chat contains unsafe content...
