gptme icon indicating copy to clipboard operation
gptme copied to clipboard

fix: more detailed perplexity prompting guide

Open ErikBjare opened this issue 3 months ago • 3 comments

[!IMPORTANT] Adds a detailed user prompt guide for web search models in _browser_perplexity.py and integrates it into browser.py examples.

  • User Prompt Guide:
    • Adds USER_PROMPT in _browser_perplexity.py with detailed guidelines for effective web search model prompting.
    • Covers best practices like specificity, avoiding few-shot prompting, and thinking like a web search user.
    • Highlights pitfalls such as overly generic questions and complex multi-part requests.
  • Integration:
    • Imports USER_PROMPT in browser.py and includes it in the examples section for demonstration.
  • Misc:
    • Minor import reordering in _browser_perplexity.py and browser.py for clarity.

This description was created by Ellipsis for 9d489eeb6fb925e35463ec6960dea944f13040b0. You can customize this summary. It will automatically update as commits are pushed.

ErikBjare avatar Aug 27 '25 07:08 ErikBjare

:x: 1 Tests Failed:

Tests completed Failed Passed Skipped
252 1 251 10
View the top 1 failed test(s) by shortest run time
tests.test_prompts::test_get_prompt_full
Stack Traces | 3.51s run time
def test_get_prompt_full():
        prompt_msgs = get_prompt(get_tools(), prompt="full")
        # Combine all message contents for token counting
        combined_content = "\n\n".join(msg.content for msg in prompt_msgs)
        # TODO: lower this significantly by selectively removing examples from the full prompt
>       assert 500 < len_tokens(combined_content, "gpt-4") < 5000
E       assert 5343 < 5000
E        +  where 5343 = len_tokens('You are gptme v0.28.0+unknown, a general-purpose AI assistant powered by LLMs. \nYou are designed to help users with programming tasks, such as writing code, debugging, and learning new concepts.\nYou can run code, execute terminal commands, and access the filesystem on the local machine.\nYou will help the user with writing code, either from scratch or in existing projects.\nYou will think step by step when solving a problem, in `<thinking>` tags.\nBreak down complex tasks into smaller, manageable steps.\n\nYou have the ability to self-correct. If you receive feedback that your output or actions were incorrect, you should:\n- acknowledge the mistake\n- analyze what went wrong in `<thinking>` tags\n- provide a corrected response\n\nYou should learn about the context needed to provide the best help,\nsuch as exploring the current working directory and reading the code using terminal tools.\n\nWhen suggesting code changes, prefer applying patches over examples. Preserve comments, unless they are no longer relevant.\nUse the patch tool to edit existing files, or the save tool to overwrite.\nWhen the output of a command is of interest, end the code block and message, so that it can ...st_chats(max_results: int, include_summary: bool)\n- search_chats(query: str, max_results: int, sort: Literal["date", "count"])\n- read_chat(id: str, max_results: int)\n- screenshot(path: Union[Path, NoneType]) -> Path\n- view_image(image_path: Union[Path, str]) -> Message\n- get_transcript(video_id: str) -> str\n- summarize_transcript(transcript: str) -> str\n\n### Examples\n\n#### Result of the last expression will be returned\n\n> User: What is 2 + 2?\n> Assistant:\n```ipython\n2 + 2\n```\n> System: Executed code block.\n```result\n4\n```\n\n#### Write a function and call it\n\n> User: compute fib 10\n> Assistant: To compute the 10th Fibonacci number, we can run the following code:\n```ipython\ndef fib(n):\n    if n <= 1:\n        return n\n    return fib(n - 1) + fib(n - 2)\nfib(10)\n```\n> System: Executed code block.\n```result\n55\n```\n\n*End of Tools List.*\n\n\n\n# About User\n\nI am a curious human programmer.\n\n## User\'s Response Preferences\n\nBasic concepts don\'t need to be explained.\n\n\n## System Information\n\n**OS:** Ubuntu 24.04\n**Working Directory:** .../pytest-0/popen-gw13/test_symlink_handling0\n\n## Current Date\n\n**UTC:** 2025-08-27', 'gpt-4')

.../gptme/tests/test_prompts.py:17: AssertionError

To view more test analytics, go to the Test Analytics Dashboard 📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

codecov[bot] avatar Aug 27 '25 07:08 codecov[bot]

Looks like there are a few issues preventing this PR from being merged!

  • GitHub Actions are failing:
    • Test

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #628 at branch `dev/perplexity-detailed-prompt`

Feel free to include any additional details that might help me get this PR into a better state.

You can manage your notification settings

openhands-ai[bot] avatar Aug 27 '25 08:08 openhands-ai[bot]

This prompt may be an improvement, but it is overly verbose and seems like a better fit fot a lesson or something unless it can be significantly reduced in size. Maybe a short version in the browser prompt and a longer lesson for detailed usage (basically "query as a LLM", not sure how detailed you have to be, you can ask it hard things to challenge it to do a good job)

ErikBjare avatar Oct 16 '25 07:10 ErikBjare