instream icon indicating copy to clipboard operation
instream copied to clipboard

Flux Queries With V1 Client Against OSS 2.x

Open kbredemeier opened this issue 3 years ago • 4 comments

The org is not added to the URL parameters when using a connection configured for v1 against an InfluxDB OSS 2.x. The API documentation states that either one of those is required in the query parameters.

defmodule V1 do
  use Instream.Connection,
    config: [
      auth: [
        method: :token,
        token: System.fetch_env!("LB_INFLUXDB_LOCAL_TOKEN")
      ],
      org: "instream_test",
      version: :v1,
      database: "mapped_database"
    ]
end

V1.query(
  """
  import "influxdata/influxdb/schema"

  schema.measurements(bucket: "test_bucket")
  """,
  query_language: :flux
)

%{code: "invalid", message: "failed to decode request body: Please provide either orgID or org"}

I'm currently working on a fix.

kbredemeier avatar Nov 09 '22 13:11 kbredemeier

Well, the :v1 client is was never intended to work with an InfluxDB v2.x instance. And I would argue it never should.

As you are using a Flux query with an Org and querying a Bucket, what is missing to configure the connection as version: :v2?

mneudert avatar Nov 09 '22 15:11 mneudert

We recently migrated from the influx enterprise version to cloud 2 version. Almost all of our queries are written in InfluxQL, so our client is configured to use v1 as this required the least amount of changes to make the migration happen.

Unfortunately some InfluxQL features aren't available anymore in the cloud 2 version so that we had to re-implement some queries with flux.

To make it work with version: :v2 I would have to

  • migrate all queries to flux.
  • explicitly pass the query_language: :influxql option with all InfluxQL queries.
  • use a different connection for flux queries.

I'm fine with explicitly passing the query_language option, this just looked like a bug to me.

Btw. I noticed that cloud 2 and OSS are diverging in some parts of their API (e.g. parameterized flux queries are only supported with cloud 2).

kbredemeier avatar Nov 09 '22 15:11 kbredemeier

As the current behaviour is not really a bug but it would also be quite painful to add an option to however many function calls there are I would like to add another option: #80

This should allow you to keep your code as is and only require version: :v2, query_language: :influxql in your config. Unless I messed up somewhere of course.

mneudert avatar Nov 09 '22 18:11 mneudert

Hey, thanks for the effort. I just gave your branch a spin but I ran into another problem with that: We are still relying on the DBRP mapping for backwards compatibility. If I configure the client with v2 it doesn't use the compatibility API anymore which breaks our data ingestion.

We could add another option for that but I'm not sure if I'm a fan of that idea :sweat_smile:

kbredemeier avatar Nov 10 '22 12:11 kbredemeier