server
server copied to clipboard
PayloadTooLargeError and graphql foreign key violation on server start
Description
I am seeing these errors when I run prefect server start
on 0.13.8
on a particular machine.
Link to error log gist
I was running an older version of prefect, either 0.13.4 or 0.13.6, ran a workflow over dask executor, then upgraded to 0.13.8 and restarted it. I've tried clearing all docker images and volumes, and ~/.prefect/flows, and I still see the same error.
Expected Behavior
No error.
Reproduction
I've tried reproducing this on 2 other machines but haven't been able to.
Environment
Ubuntu 18.04
This is my config.toml
backend = "server"
[server]
host = "http://172.18.1.3"
[server.ui]
apollo_url = "http://172.18.1.3:4200/graphql"
[logging]
# The logging level: NOTSET, DEBUG, INFO, WARNING, ERROR, or CRITICAL
level = "WARNING"
[context.secrets]
SLACK_WEBHOOK_URL = "<redacted>"
Hi @gitanshu - the error payloads you are seeing are:
-
PayloadTooLargeError
: this is caused by sending too large a payload to the API, probably on the order of tens of megabytes if I had to guess -
Foreign key violation. insert or update on table "log" violates foreign key constraint "log_flow_run_id_fkey"'
: this is being caused by an API request that is attempting to write a log for a flow run ID that is not present in the database. All logs must be associated with a valid flow run ID.
Both of these errors are only triggered when you make a request to the API, so could you describe more where these requests might be coming from?