Sporadic Failure When Sending Request to Dify API

[ottoradiologia]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.16

Cloud or Self Hosted

Cloud

Steps to reproduce

Steps to reproduce:

Initiate a request to the Dify API. Monitor the steps executed in the tracking console. Observe the sporadic failure in certain requests.

✔️ Expected Behavior

The request should be processed correctly, with all steps completed without showing any errors.

❌ Actual Behavior

In some cases, the request fails, returning an error. I have attached an example of a failure and a success for comparison.

[dosubot[bot]]

You are experiencing sporadic failures when sending requests to the Dify API likely because the Ollama service is not accessible from the Docker container. This is a common issue when localhost refers to the container itself rather than the host machine or other containers.

To resolve this issue, you need to expose the Ollama service to the network. Specifically, you should replace localhost with host.docker.internal in the service URL.

http://host.docker.internal:11434

Additionally, ensure that your API requests are correctly configured. For example, when using the DifyClient in a Node.js environment, make sure that the base URL and headers are properly set, as shown in the test cases:

import { DifyClient, BASE_URL, routes } from ".";

import axios from 'axios'

jest.mock('axios')

describe('Send Requests', () => {
  let difyClient

  beforeEach(() => {
    difyClient = new DifyClient('test')
  })

  afterEach(() => {
    jest.resetAllMocks()
  })

  it('should make a successful request to the application parameter', async () => {
    const method = 'GET'
    const endpoint = routes.application.url()
    const expectedResponse = { data: 'response' }
    axios.mockResolvedValue(expectedResponse)

    await difyClient.sendRequest(method, endpoint)

    expect(axios).toHaveBeenCalledWith({
      method,
      url: `${BASE_URL}${endpoint}`,
      params: null,
      headers: {
        Authorization: `Bearer ${difyClient.apiKey}`,
        'Content-Type': 'application/json',
      },
      responseType: 'json',
    })
  })

  it('should handle errors from the API', async () => {
    const method = 'GET'
    const endpoint = '/test-endpoint'
    const errorMessage = 'Request failed with status code 404'
    axios.mockRejectedValue(new Error(errorMessage))

    await expect(difyClient.sendRequest(method, endpoint)).rejects.toThrow(
      errorMessage
    )
  })
})

If you are using the DifyClient in a PHP environment, ensure that the base URL and headers are properly set as well. Here is an example configuration:

<?php

require 'vendor/autoload.php';

use GuzzleHttp\Client;

class DifyClient {
    protected $api_key;
    protected $base_url;
    protected $client;

    public function __construct($api_key) {
        $this->api_key = $api_key;
        $this->base_url = "https://api.dify.ai/v1/";
        $this->client = new Client([
            'base_uri' => $this->base_url,
            'headers' => [
                'Authorization' => 'Bearer ' . $this->api_key,
                'Content-Type' => 'application/json',
            ],
        ]);
    }

    protected function send_request($method, $endpoint, $data = null, $params = null, $stream = false) {
        $options = [
            'json' => $data,
            'query' => $params,
            'stream' => $stream,
        ];

        $response = $this->client->request($method, $endpoint, $options);
        return $response;
    }

    public function message_feedback($message_id, $rating, $user) {
        $data = [
            'rating' => $rating,
            'user' => $user,
        ];
        return $this->send_request('POST', "messages/{$message_id}/feedbacks", $data);
    }

    public function get_application_parameters($user) {
        $params = ['user' => $user];
        return $this->send_request('GET', 'parameters', null, $params);
    }
}

Additionally, ensure that you are not hitting any rate limits or quota issues with the API. According to the Dify documentation, errors such as "Rate limit reached" or "You exceeded your current quota" can occur if the official interface call rate limit has been reached or if the account balance has been used up. Check the official documentation for details on rate limits and billing.

This change should make the service accessible from the Docker container and resolve the connection issues ^[1][2][3][4].

To continue talking to Dosu, mention @dosu.

[crazywoola]

What LLM did you use in this picture?

[ottoradiologia]

Hi! Sonnet 3.5 direct from Anthropic.