mercator icon indicating copy to clipboard operation
mercator copied to clipboard

Problem to retrieve 1-to-many links via API

Open ggs67 opened this issue 9 months ago • 3 comments

When creating a resource via the api one can link those using some attributes and a list of IDs. An example are logical-servers where you can pass the "servers" attribute with a list (array) of IDs of physical servers on which can run.

i.e. servers : [ 30 ] # 30 being the ID of the physical server

In the controller this is realized via:

if ($request->has('servers')) { $logicalServer->servers()->sync($request->input('servers', [])); }

When I retrieve now all logical servers, the linking attributes (like "servers") is missing from the returned json, While the show() method seems to honor it via:

$logicalServer->servers = $logicalServer->servers()->pluck('id');

ggs67 avatar Feb 14 '25 12:02 ggs67

By default, when retrieving a list of logical servers via the API, Eloquent does not automatically include relationships. This is why the "servers" attribute is missing from the returned JSON.

Is it possible to add these lines in the index to get all servers :

        $logicalServers->each(function ($server) {
            $server->servers = $server->servers()->pluck('id');
        });

or you may do it in your script and get all related objects.

However, this approach is not efficient because it generates one additional SQL query per logical server, leading to an N+1 query problem.

Due to the nature of SQL queries, I have not find a more efficient approach to this.

dbarzin avatar Feb 14 '25 14:02 dbarzin

The problem with modifying the code is obviously future updates, also it would be only for the servers not the other links. The problem with getting the servers one by one, beside the many calls themselves, is the DDOS protection which needs throttling of the calls. The only solution I found for the Mercator python library I wrote is a throttling function which is called after each call and after 50 (or so) calls it pauses for 60 seconds to avoid repeat calls errors.

Wouldn't it be possible to add code for links in each index and add a "retrieve-links" attribute or add an additional route like "logical-servers-with-links" (but not only for logical servers) to allow queries with and without the links ?

ggs67 avatar Feb 18 '25 08:02 ggs67

You may bypass the DDOS protection by putting this line in comment 'throttle:60,1', in app/Http/kernel.php source file.

dbarzin avatar Feb 18 '25 09:02 dbarzin