Shorten federated strings if they exceed database limits, instead of throwing error

[tateisu]

this instance is created at 8.10 and upgraded to 9.0.0. using docker image.

postgres_1  | 2021-01-26 03:53:52.811 UTC [30] STATEMENT:  INSERT INTO "user_" ("name", "preferred_username", "password_encrypted", "admin", "banned", "email", "avatar", "published", "updated", "show_nsfw", "theme", "default_sort_type", "default_listing_type", "lang", "show_avatars", "send_notifications_to_email", "matrix_user_id", "actor_id", "bio", "local", "private_key", "public_key", "last_refreshed_at", "banner") VALUES ($1, $2, $3, $4, DEFAULT, DEFAULT, $5, DEFAULT, DEFAULT, $6, $7, $8, $9, $10, $11, $12, DEFAULT, $13, $14, $15, DEFAULT, $16, $17, $18) ON CONFLICT ("actor_id") DO UPDATE SET "name" = $19, "preferred_username" = $20, "password_encrypted" = $21, "admin" = $22, "avatar" = $23, "show_nsfw" = $24, "theme" = $25, "default_sort_type" = $26, "default_listing_type" = $27, "lang" = $28, "show_avatars" = $29, "send_notifications_to_email" = $30, "actor_id" = $31, "bio" = $32, "local" = $33, "public_key" = $34, "last_refreshed_at" = $35, "banner" = $36 RETURNING "user_"."id", "user_"."name", "user_"."preferred_username", "user_"."password_encrypted", "user_"."email", "user_"."avatar", "user_"."admin", "user_"."banned", "user_"."published", "user_"."updated", "user_"."show_nsfw", "user_"."theme", "user_"."default_sort_type", "user_"."default_listing_type", "user_"."lang", "user_"."show_avatars", "user_"."send_notifications_to_email", "user_"."matrix_user_id", "user_"."actor_id", "user_"."bio", "user_"."local", "user_"."private_key", "user_"."public_key", "user_"."last_refreshed_at", "user_"."banner", "user_"."deleted"
lemmy_1     | [2021-01-26T03:53:52Z ERROR actix_http::response] Internal Server Error: LemmyError { inner: value too long for type character varying(20) }

Found a bug? Please fill out the sections below. 👍

For front end issues, use lemmy-ui

Issue Summary

A summary of the bug.

Steps to Reproduce

1. (for example) I clicked login, and an endless spinner show up.
2. I tried to install lemmy via this guide, and I'm getting this error.
3. ...

Technical details

• Please post your log: sudo docker-compose logs > lemmy_log.out.
• What OS are you trying to install lemmy on?
• Any browser console errors?

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[dessalines]

4 Fields with that constraint:

 name                        | character varying(20)       |           | not null |
 preferred_username          | character varying(20)       |           |          |
 theme                       | character varying(20)       |           | not null | 'darkly'::character varying
 lang                        | character varying(20)       |           | not null | 'browser'::character varying

Did you somehow create a user with more than that for a username length?

[tateisu]

the request did not sent from me, how to notify it to the user?

[dessalines]

It will tell them that error message, and it won't crash the server.

You not being able to log in is separate from this issue.

[tateisu]

this error is on server https://lemmy.juggler.jp/ , NOT https://lemmy2.juggler.jp/ . This server is already configured a few days ago.

This error keeps appearing so often that I suspect a federation-related issue rather than user creation from a web form.

[dessalines]

@Nutomic its possible this is a federation error that's trying to fetch a user and not being able to insert it due to the table constraints? That means that someone disabled the table constraints on another instance.

[Nutomic]

Yes thats possible, the federation code isnt doing any separate length checks, so it will error out on the database insert, and keep retrying when the user gets fetched again. Maybe we should log the @user@instance when a fetch/insert fails. And I suppose we might increase these limits for remote users?

[dessalines]

I'd recommend the logging be in the remote fetcher, to log the actor_id that it's remote fetching.

As far as a fix? Probably check the log and block that instance. Malicious instances could mess with all kinds of constraints that we shouldn't have to check twice in both the apub and DB code. The only way I can see that error happening is if an instances manually removed that DB constraint.

[Nutomic]

We are also gonna federate with other software sooner or later, and those might well have higher limits.

The logging is related to #1259.

[dessalines]

@Nutomic Is this still an issue?

[Nutomic]

Yes this hasnt been resolved. I think a better solution might be to remove length limits from the database (or increase them a lot), and only enforce limits in the API. Thats what Mastodon does and it allows longer content from remote instances while limits apply only locally.

[dessalines]

I spose I'd be fine with either approach, but the one that makes me wary about removing the DB limits, is that things like post titles from other platforms might accidentally not get truncated properly, and lemmy-ui would render some really long post titles.