cai
cai copied to clipboard
Published
20 hours ago •
aliasrobotics
aliasrobotics
OpenAI Integration with HuggingFace API
Hello 👋
I'm using your amazing tool, but for free with HuggingFace API, so according to HF's Docs, I will replace OPENAI_API_KEY with their API key, and for sure BASELINE too, here i didn't find any env var to assign my proxy router to HF inference but OLLAMA_API_BASE and also i added CAI_MODEL as HF's docs says, so now i got an error.
here is my .env file contents
OPENAI_API_KEY="hf_**********************"
ANTHROPIC_API_KEY=""
OLLAMA=""
PROMPT_TOOLKIT_NO_CPR=1
OLLAMA_API_BASE="https://router.huggingface.co/hf-inference/models/meta-llama/Llama-3.3-70B-Instruct/v1"
CAI_MODEL="meta-llama/Llama-3.3-70B-Instruct"
and the output
cai
Error importing bb_triage: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
Error importing red_team: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
WARNING:root:Could not read version from pyproject.toml: [Errno 2] No such file or directory: 'pyproject.toml'
CCCCCCCCCCCCC ++++++++ ++++++++ IIIIIIIIII
CCC::::::::::::C ++++++++++ ++++++++++ I::::::::I
CC:::::::::::::::C ++++++++++ ++++++++++ I::::::::I
C:::::CCCCCCCC::::C +++++++++ ++ +++++++++ II::::::II
C:::::C CCCCCC +++++++ +++++ +++++++ I::::I
C:::::C +++++ +++++++ +++++ I::::I
C:::::C ++++ ++++ I::::I
C:::::C ++ ++ I::::I
C:::::C + +++++++++++++++ + I::::I
C:::::C +++++++++++++++++++ I::::I
C:::::C +++++++++++++++++ I::::I
C:::::C CCCCCC +++++++++++++++ I::::I
C:::::CCCCCCCC::::C +++++++++++++ II::::::II
CC:::::::::::::::C +++++++++ I::::::::I
CCC::::::::::::C +++++ I::::::::I
CCCCCCCCCCCCC ++ IIIIIIIIII
Cybersecurity AI (CAI), vunknown
Bug bounty-ready AI
╭─────────────────────────────────────────────── 🚀 CAI defacto scaffolding for cybersecurity agents - Type /help for detailed documentation ────────────────────────────────────────────────╮
│ │
│ CAI Command Reference ╭──────────────── 🛡️ Alias0 - best model for cybersecurity ─────────────────╮ │
│ │ │ │
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │ 🔒 Security-Focused AI Framework │ │
│ AGENT MANAGEMENT (/a) │ │ │
│ CAI>/agent list - List all available agents │ For optimal cybersecurity AI performance, use │ │
│ CAI>/agent select [NAME] - Switch to specific agent │ alias0 - specifically designed for cybersecurity │ │
│ CAI>/agent info [NAME] - Show agent details │ tasks with superior domain knowledge. │ │
│ CAI>/parallel add [NAME] - Configure parallel agents │ │ │
│ │ alias0 outperforms general-purpose models in: │ │
│ MEMORY & HISTORY │ • Vulnerability assessment │ │
│ CAI>/memory list - List saved memories │ • Penetration testing and bug bounty │ │
│ CAI>/history - View conversation history │ • Security analysis │ │
│ CAI>/compact - AI-powered conversation summary │ • Threat detection │ │
│ CAI>/flush - Clear conversation history │ │ │
│ │ Learn more about alias0 and its privacy-first approach: │ │
│ ENVIRONMENT │ https://news.aliasrobotics.com/alias0-a-privacy-first-cybersecurity-ai/ │ │
│ CAI>/workspace set [NAME] - Set workspace directory │ │ │
│ CAI>/config - Manage environment variables ╰───────────────────────────────────────────────────────────────────────────╯ │
│ CAI>/virt run [IMAGE] - Run Docker containers │
│ │
│ TOOLS & INTEGRATION │
│ CAI>/mcp load [TYPE] [CONFIG] - Load MCP servers │
│ CAI>/shell [COMMAND] or $ - Execute shell commands │
│ CAI>/model [NAME] - Change AI model │
│ │
│ QUICK SHORTCUTS │
│ ESC + ENTER - Multi-line input │
│ TAB - Command completion │
│ ↑/↓ - Command history │
│ Ctrl+C - Interrupt/Exit │
│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │
│ │
│ Quick Start Workflows │
│ │
│ 🎯 CTF Challenge │
│ 1. CAI> /agent select redteam_agent │
│ 2. CAI> /workspace set ctf_name │
│ 3. CAI> Describe the challenge... │
│ │
│ 🐛 Bug Bounty │
│ 1. CAI> /agent select bug_bounter_agent │
│ 2. CAI> /model claude-3-7-sonnet │
│ 3. CAI> Test https://example.com │
│ │
│ CAI collects pseudonymized data to improve our research. │
│ Your privacy is protected in compliance with GDPR. │
│ Continue to start, or press Ctrl-C to exit. │
│ │
│ 🔍 Parallel Recon │
│ 1. CAI> /parallel add red_teamer │
│ 2. CAI> /parallel add network_traffic_analyzer │
│ 3. CAI> Scan 192.168.1.0/24 │
│ │
│ 🛠️ MCP Tools Integration │
│ 1. CAI> /mcp load sse http://localhost:3000 │
│ 2. CAI> /mcp add server_name agent_name │
│ 3. CAI> Use the new tools... │
│ │
│ Environment Variables: │
│ CAI_MODEL = alias0 │
│ CAI_AGENT_TYPE = one_tool_agent │
│ CAI_PARALLEL = 1 │
│ CAI_STREAM = true │
│ CAI_WORKSPACE = default │
│ │
│ 💡 Pro Tips: │
│ • Use /help for detailed command help │
│ • Use /help quick for this guide │
│ • Use /help commands for all commands │
│ • Use $ prefix for quick shell: $ ls │
│ │
│ │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
CAI> hello
Log file: logs/cai_e41c8b45-a3e9-447d-96de-3dce59f4a720_20250701_012352_beta_linux_6.6.87.2-microsoft-standard-WSL2_41_238_168_8.jsonl
ERROR:cai.cli:Error in main loop: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
Traceback (most recent call last):
File "/root/.pyenv/versions/3.12.3/envs/cai_env/lib/python3.12/site-packages/cai/cli.py", line 1533, in run_cai_cli
response = asyncio.run(Runner.run(agent, conversation_input))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.pyenv/versions/3.12.3/lib/python3.12/asyncio/runners.py", line 194, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/root/.pyenv/versions/3.12.3/lib/python3.12/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.pyenv/versions/3.12.3/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/root/.pyenv/versions/3.12.3/envs/cai_env/lib/python3.12/site-packages/cai/sdk/agents/run.py", line 239, in run
input_guardrail_results, turn_result = await asyncio.gather(
^^^^^^^^^^^^^^^^^^^^^
File "/root/.pyenv/versions/3.12.3/envs/cai_env/lib/python3.12/site-packages/cai/sdk/agents/run.py", line 847, in _run_single_turn
new_response = await cls._get_new_response(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.pyenv/versions/3.12.3/envs/cai_env/lib/python3.12/site-packages/cai/sdk/agents/run.py", line 1046, in _get_new_response
new_response = await model.get_response(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.pyenv/versions/3.12.3/envs/cai_env/lib/python3.12/site-packages/cai/sdk/agents/models/openai_chatcompletions.py", line 545, in get_response
| {"base_url": str(self._get_client().base_url)},
^^^^^^^^^^^^^^^^^^
File "/root/.pyenv/versions/3.12.3/envs/cai_env/lib/python3.12/site-packages/cai/sdk/agents/models/openai_chatcompletions.py", line 3506, in _get_client
self._client = AsyncOpenAI()
^^^^^^^^^^^^^
File "/root/.pyenv/versions/3.12.3/envs/cai_env/lib/python3.12/site-packages/openai/_client.py", line 349, in __init__
raise OpenAIError(
openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
CAI> /model
╭─────────────────────────────────────────────────────────────────────────────────────── Active Model ───────────────────────────────────────────────────────────────────────────────────────╮
│ Current model: Unknown │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Now after you see the log, please tell me 3 things:
- What makes this error happens
- How to correctly integrate HF's API through OPENAI Integration in your tool.
- If you don't mind, and if my way will not work to integrate HF's API, tell me what alternative free model to use.
Best Regards, Beta.
You can try hugginface_api_key and use huggingface/meta-flame/flame as your model name...
https://docs.litellm.ai/docs/providers/huggingface
Yes, if your PC is powerful enough, you can host your own model locally. To work with CAI, the model needs to be exposed through an endpoint compatible with the OpenAI API.
You could use open source inference servers to run Hugging Face models locally, or tools like Ollama and LM Studio for local hosting. Keep in mind that larger models require a GPU with sufficient VRAM, while smaller models can run on CPU but slower.
This approach allows you to experiment with free models.
Hope this helps!
