seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Bug] [env jars] third-party packages can't be loaded

Open Xuzhengz opened this issue 1 year ago • 7 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

third-party packages can't be loaded

SeaTunnel Version

dev-2.3.6

SeaTunnel Config

{
  "env": {
    "execution.parallelism": 1,
    "job.name": "udf测试",
    "job.mode": "batch",
    "jars": "file://Users/xuzhengzhou/IdeaProjects/starrocks-udf/target/starrocks-udf-1.0-SNAPSHOT-jar-with-dependencies.jar;"
  },
  "preHandler": [
  ],
  "source": [
    {
      "plugin_name": "FakeSource",
      "resourceType": "nosql",
      "row.num": 100,
      "schema": {
        "table": "ottomi",
        "fields": {
          "id": "string",
          "age": "int"
        }
      },
      "resources": [
        "ottomi"
      ],
      "result_table_name": "fake_source",
      "ottomi_node_type": "FakeSource"
    }
  ],
  "transform": [
  ],
  "sink": [
    {
      "plugin_name": "Void",
      "source_table_name": "fake_source"
    }
  ]
}

Running Command

bin/seatunnel.sh -c udf.json

Error Exception

null

Zeta or Flink or Spark Version

No response

Java or Scala Version

1.8

Screenshots

image image

Are you willing to submit PR?

  • [X] Yes I am willing to submit a PR!

Code of Conduct

Xuzhengz avatar Jun 20 '24 05:06 Xuzhengz

@Hisoka-X PTAL

EricJoy2048 avatar Jun 20 '24 10:06 EricJoy2048

you can put the jar to plugins dir

EricJoy2048 avatar Jun 20 '24 10:06 EricJoy2048

@Xuzhengz Could you try with #7035 ?

Hisoka-X avatar Jun 20 '24 13:06 Hisoka-X

@Xuzhengz Could you try with #7035 ?

Does env.jars only support local files? If the clustering mode fails, the jar package may not be on another node

Xuzhengz avatar Jun 20 '24 13:06 Xuzhengz

@Xuzhengz Could you try with #7035 ?

Does env.jars only support local files? If the clustering mode fails, the jar package may not be on another node

It should support HTTP/HTTPS, FTP, Local at now.

Hisoka-X avatar Jun 20 '24 13:06 Hisoka-X

@Xuzhengz Could you try with #7035 ?

Does env.jars only support local files? If the clustering mode fails, the jar package may not be on another node

It should support HTTP/HTTPS, FTP, Local at now.

Please tell me that the code you repaired has not been merged into the dev branch, how can I test it or wait for the dev branch

Xuzhengz avatar Jun 20 '24 13:06 Xuzhengz

@Xuzhengz Could you try with #7035 ?

Does env.jars only support local files? If the clustering mode fails, the jar package may not be on another node

It should support HTTP/HTTPS, FTP, Local at now.

Please tell me that the code you repaired has not been merged into the dev branch, how can I test it or wait for the dev branch

You can cherry pick to your local branch or waiting it merge into dev.

Hisoka-X avatar Jun 21 '24 02:06 Hisoka-X