lemmy icon indicating copy to clipboard operation
lemmy copied to clipboard

Lots of redundant data returned by comment API

Open derivator opened this issue 1 year ago • 6 comments

Issue Summary

Lemmy API sends tons of redundant data. For each comment, the entire containing post and community are also returned, even if they are all the same.

Steps to Reproduce

Go to https://feddit.de/api/v3/comment/list?post_id=745959&type_=All

Technical details

CommentView contains these fields and they are not optional. They are of course needed if you are listing comments for a community, but when listing by post_id, they are redundant.

derivator avatar Jun 07 '23 22:06 derivator

Lots of reasons that the CommentView contains lots of extra data.

Take the profile page for instance: many comments from one person, but on different communities.

Screenshot_2023-06-07-20-42-29-518_net slions fulguris full fdroid

Or the home page: lots of posts / comments from different communities.

Having lots of different CommentViews, which change forms depending on where they're selected from, would be a burden to maintain.

dessalines avatar Jun 08 '23 00:06 dessalines

Having lots of different CommentViews, which change forms depending on where they're selected from, would be a burden to maintain.

I understand that, however "show me the comments for this one post" is an extremely common case and deleting the post and community fields reduced size by half on a random thread I just tried, so I thought I'd at least mention it.

derivator avatar Jun 08 '23 01:06 derivator

For sure. If anyone would like to add various CommentViews, I wouldn't be opposed, as long as they are well-typed.

dessalines avatar Jun 08 '23 16:06 dessalines

Should the less redundant API be at /post/comments?

dullbananas avatar Jun 18 '23 23:06 dullbananas

This will be a minor problem when we have compressed API responses (already merged) and optimized database queries.

This API should be made less redundant in /api/v4. The response should be an object with "users" and "posts" fields, which are both maps that are indexed with IDs. Each value in "posts" should have a "comments" field which is also indexed with IDs. Each value in "comments" should have its own "comments" field just like posts and only store the author's ID.

{
    "users": {
        1: {...}
    },
    "posts": {
        2: {
            ...,
            "creator_id": 1,
            "comments": {
                3: {
                    ...,
                    "creator_id": 1,
                    "comments": {}
                }
            }
        }
    }
}

dullbananas avatar Jun 26 '23 23:06 dullbananas

How realistic or helpful would it be to implement a ?fields=field1,field2 parameter that determines which fields are returned? I know in some cases this can reduce server load/bandwidth usage (especially with nested data), and it would prevent the need for additional endpoints.

silasabbott avatar Feb 18 '24 22:02 silasabbott

That sounds like a lot of effort to implement. And in the end clients would likely request all fields anyway to keep things simple.

Nutomic avatar Feb 19 '24 09:02 Nutomic