AIEngine Demos: DeepSeek plugin sample for Smart CodeInsight not working (lacking some OTA integration?)

[marcobreveglieri]

The DeepSeek AI plugin example (but also the others in the same folder) do not work properly: when sending a command via the "AI Chat" window, the loader animation runs infinitely and the received response is not displayed.

I am referring to the demo stored in this folder: "Object Pascal/ToolsAPI/AIEngine Demos/DeepSeek_Plugin"

I've done a DeepSeek plugin myself during a live coding session, starting from the Cohere sample. I am having this issue then I confronted my work with the sample plugin and both of them give the same wrong behavior.

Doing some checks, it seems that the plugin is unable to pass the answer received from the LLM service since there are no "notifiers" (IOTAAIServicesNotifier) registered calling the TDeepSeekAIRestClient.AddNotifier() function.

When TDeepSeekAIRestClient.NotifyAnswer() is called in order to pass out the chat answer, then FNotifyList.Count is equal to zero; simply put, the received response is not passed to Delphi in any way.

Are there any missing registrations in the plugin's OTA services? Is it necessary to implement the IOTAAIServicesNotifier interface? If is it so, how?

I think it would be appropriate to fix the example by adding the missing part for those who would like to exploit this interesting opportunity of integrating Delphi with AI services, given that OTA documentation is very limited.

I am totally available to give additional feedback and hints about this.

Thanks a lot! 🤗

[marcobreveglieri]

Calling out @checkdigits to draw some attention to the issue, hoping it doesn't bother anyone. :)

[AliDehbansiahkarbon]

You are right, when you install the plugin for the first time, it is registered as a plugin but the TDeepSeekAIRestClient.AddNotifier() function won't be called until the next IDE execution, which means you need to restart the IDE once you installed a new plugin, then it will work properly.

You need to restart the IDE once you install a new plugin.

The AddNotifier() method will be called in IDE start time once, it means the plugin must be available in IDE starting time. In this case the plugin has been registered but it's not correctly wired to the IDE's main AI plugin (including the AIChat window, editor menus, etc)

It could be fixed in RAD Studio 13 or later probably via introducing a newer version of IOTAAIEngineService that has a function to refresh "Notifiers" or like that but for now just restart the IDE and you are fine.

About the implementation the IOTAAIServicesNotifier interface, normally you don't need to unless you want to have your personal Chat window instead of the official one. In that case you should implement a class like this:

    TMyNotifier = class(IOTAAIServicesNotifier)
      procedure Answer(const AMessage: string; const ARequestID: TGUID);
      procedure Error(const AMessage: string; const ARequestID: TGUID);
      procedure ModelsLoaded(const AModels: TArray<string>; const ARequestID: TGUID);
      procedure ImageLoaded(const AImage: TStream; const ARequestID: TGUID); overload;
      procedure ImageLoaded(const A64BitImages: TArray<string>; const ARequestID: TGUID); overload;
      procedure ImageURlLoaded(const AImageURls: TArray<string>; const ARequestID: TGUID);
      procedure SpeechLoaded(const ASpeech: TStream; const ARequestID: TGUID);
    end;

And add it manually like this in your plugin's register procedure, after registering the plugin to AIEngineService:

AIEngineService.GetPluginByName(cDeepSeekAI_name).AddNotifier(TOTAAIServicesNotifier.Create);

Question: What is the usage of ARequestID: TGUID ?

Answer: Each API call (for example _TDeepSeekPlugin.Chat()) must generate one GUID as a key for the question and get the exact answer that carries this GUID, so in the Answer method of TMyNotifier you will know which answer belongs to which question in case of delayed answer or partial answer.

Best regards, Ali.

[marcobreveglieri]

You are right, when you install the plugin for the first time [...] Best regards, Ali.

Thank you so much for all the valuable information. 🤗

I'll definitely delve deeper into this as soon as possible, taking the explanations you've provided to heart.

Best regards, Marco.