Unable to Run Flow Through Local UI

[isaacanthony]

Description

A clear description of the bug

Any flow that I register and try to run through the UI can get Submitted for Execution but never Executed.

Expected Behavior

What did you expect to happen instead?

In the best case, I would hope the flow to run without an issue. At the minimum, I would hope that I could get a more informative error message of what I am missing. To my eyes, I have followed the documentation and tutorials exactly.

Reproduction

A minimal example that exhibits the behavior.

1. I set up an account with Prefect Cloud so that I could create TENANT and RUNNER tokens.
2. I added the TENANT token to my local ~/.prefect/config.toml.
3. I downloaded the newest version of prefect from PyPi. pip3 install prefect
4. I started the server. prefect server start
5. I started an agent. prefect agent start -t {{RUNNER_TOKEN}}
6. I registered the sample flow provided in a walkthrough. python3 hello_flow.py

import prefect
from prefect import task, Flow

@task
def hello_task():
    logger = prefect.context.get("logger")
    logger.info("Hello, Cloud!")

flow = Flow("hello-flow", tasks=[hello_task])
flow.register()

7. I go through my local UI on localhost:8080 and click "Run flow".
8. The flow transitions through the states: Flow Run Scheduled and Submitted for Execution then just hangs. The flow is never executed.
9. Here are the logs.

June 7th 2020,1:06:37pm 	agent	INFO	Submitted for execution: PID: 24731
June 7th 2020,1:06:38pm 	prefect.CloudFlowRunner	INFO	Beginning Flow run for 'hello-flow'
June 7th 2020,1:06:38pm 	prefect.CloudFlowRunner	DEBUG	Failed to retrieve flow state with error: BoxKeyError("'serialized_state'",)

10. There is nothing on Google or in the documentation about that error message.
11. Note, I also tried setting up a local dask cluster and specifying that as my DaskExecutor address in config.toml but that didn't work either.

Environment

Any additional information about your environment Optionally run prefect diagnostics from the command line and paste the information here

{
  "config_overrides": {
    "backend": true,
    "cloud": {
      "auth_token": true
    },
    "engine": {
      "executor": {
        "dask": {
          "address": true,
          "cluster_class": true
        },
        "default_class": true
      },
      "flow_runner": {
        "default_class": true
      },
      "task_runner": {
        "default_class": true
      }
    },
    "server": {
      "telemetry": {
        "enabled": true
      }
    }
  },
  "env_vars": [],
  "system_information": {
    "platform": "Linux-4.15.0-101-generic-x86_64-with-Ubuntu-18.04-bionic",
    "prefect_version": "0.11.5",
    "python_version": "3.6.9"
  }
}

[cicdw]

Hi @isaacanthony - you appear to be conflating the Cloud API and the open source API and this is causing some issues. I believe you are registering / attempting to run your Flow through the open source API but when your Agent submits your flow run some of your configuration is reset and your Flow Run is attempting to talk to the Cloud API instead (which is not aware of your Flow's existence).

I recommend auditing your configuration to use one or the other API. For reference, our Cloud API is our recommended production API that is scalable, secure, and has collaborative features whereas the open source API is currently best suited for local development.

We are aware that our documentation needs some clarification on this distinction and are working on rolling out improvements in the next few weeks!

[isaacanthony]

Hi Chris, thank you for your response. You are right. I am confused between the local open-source Prefect platform and the Prefect Cloud API. I was thinking possibly it was a requirement of Agents to be registered with a Runner token from Prefect Cloud to work even locally. Is there documentation or a walk-through I can reference that would show me how to create a flow and run it through the UI locally? Alternatively, what should my config.toml file contain to specify I want to run everything locally, the UI, the Agents, and the Executor for a first, simple "Hello world" flow run from the UI? If you would prefer I just wait for the updated documentation, I understand. Thank you for your help!

[cicdw]

For your first flow, I recommend clearing your local user configuration file completely (maybe just make a copy for future reference). Then I recommend running prefect backend server to ensure you are pointing at the local API.

Next I suggest following these steps: https://docs.prefect.io/orchestration/tutorial/configure.html You can skip any section that has a blue "Cloud" badge. Things get complicated when you want to involve multiple machines (because each machine will need API access to your local server), so I suggest shying away from that initially.

[isaacanthony]

Great, thanks. I'll give that a try. I'll close this issue for now. Thank you for your help, Chris!

[isaacanthony]

This is still not working for me. It seems like it defaults to a CloudFlowRunner when I want it to run locally? I tried the following steps with my ~/.prefect/config.toml file completely empty and separately containing only a single line backend = "server".

Steps to reproduce:

1. Start services locally.

pip3 install --upgrade prefect
prefect backend server
prefect server start
prefect agent start

2. Create first flow.

# hello_world.py
import prefect
from prefect import task, Flow

@task
def hello_task():
    logger = prefect.context.get("logger")
    logger.info("Hello, Cloud!")

flow = Flow("hello-flow", tasks=[hello_task])
flow.register()

3. Register flow.

python3 hello_world.py

4. Navigate to UI and click Flows > hello_world > Run.
5. Flow is submitted for execution but never ran.

June 20th 2020 at 2:07:02pm | agent
INFO 
Submitted for execution: PID: 17374
June 20th 2020 at 2:07:02pm | prefect.CloudFlowRunner

INFO 
Beginning Flow run for 'hello-flow'
June 20th 2020 at 2:07:02pm | prefect.CloudFlowRunner

DEBUG 
Failed to retrieve flow state with error: BoxKeyError("'serialized_state'",)

Diagnostics

Here were the diagnostics once I put in backend = "server" and debug = true.

> prefect diagnostics
{
  "config_overrides": {
    "backend": true,
    "debug": true
  },
  "env_vars": [],
  "system_information": {
    "platform": "Linux-4.15.0-106-generic-x86_64-with-Ubuntu-18.04-bionic",
    "prefect_version": "0.12.0",
    "python_version": "3.6.9"
  }
}