influxdb icon indicating copy to clipboard operation
influxdb copied to clipboard

Published 20 hours ago • verified publisher icon influxdata

Error: panic: runtime error: invalid memory address or nil pointer dereference

Open libindas opened this issue 4 years ago • 27 comments
trafficstars

I am getting below error and restarting influxDB very frequently while querying through grafana. Anyone can help or suggest solutions to solve this problem. Error: panic: runtime error: invalid memory address or nil pointer dereference Running influx with below parameter : InfluxDB Verion: 1.8.4 Kernel: 3.10.0-1127.19.1.el7.x86_64 [monitor] store-enabled = false

I was downloaded and installed above version from https://repos.influxdata.com/rhel/

Sample Log file output.

Jun 01 13:35:51 nv01-prod-rtb-influx01 influxd: panic: runtime error: invalid memory address or nil pointer dereference Jun 01 13:35:51 nv01-prod-rtb-influx01 influxd: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x961750]

Regards Libin

libindas avatar Jun 10 '21 08:06 libindas

This looks similar to what we are experiencing.

We use influxdb 1.8.6 on freebsd 13.

Jun 10 10:30:55 influx01 daemon[75372]: panic: runtime error: invalid memory address or nil pointer dereference
Jun 10 10:30:55 influx01 daemon[75372]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x9609b0]
Jun 10 10:30:55 influx01 daemon[75372]:
Jun 10 10:30:55 influx01 daemon[75372]: goroutine 37065 [running]:
Jun 10 10:30:55 influx01 daemon[75372]: github.com/influxdata/influxdb/query.Iterators.Close(0xc02bf4a3c0, 0xa, 0xf, 0xc011f2b9a0, 0x24e4298)
Jun 10 10:30:55 influx01 daemon[75372]: 	github.com/influxdata/influxdb/query/iterator.go:48 +0x50
Jun 10 10:30:55 influx01 daemon[75372]: github.com/influxdata/influxdb/coordinator.(*LocalShardMapping).CreateIterator(0xc0029449c0, 0x24d5f80, 0xc0e3c381e0, 0xc011f2b9a0, 0x24cc568, 0xc0e3c38300, 0x0, 0x0, 0x0, 0x0, ...)
Jun 10 10:30:55 influx01 daemon[75372]: 	github.com/influxdata/influxdb/coordinator/shard_mapper.go:202 +0x339
Jun 10 10:30:55 influx01 daemon[75372]: github.com/influxdata/influxdb/query.(*exprIteratorBuilder).callIterator.func1(0xc012b65940, 0x24d5f80, 0xc0e3c381e0, 0xc012b63c48, 0xc012b63c00, 0xc0e3c38300, 0x0, 0xc00a60de50)
Jun 10 10:30:55 influx01 daemon[75372]: 	github.com/influxdata/influxdb/query/select.go:583 +0x535
Jun 10 10:30:55 influx01 daemon[75372]: github.com/influxdata/influxdb/query.(*exprIteratorBuilder).callIterator(0xc0006fb940, 0x24d5f80, 0xc0e3c381e0, 0xc0e3c38300, 0x24cc568, 0xc0e3c38300, 0x0, 0x0, 0x0, 0x0, ...)
Jun 10 10:30:55 influx01 daemon[75372]: 	github.com/influxdata/influxdb/query/select.go:608 +0xe5
Jun 10 10:30:55 influx01 daemon[75372]: github.com/influxdata/influxdb/query.(*exprIteratorBuilder).buildCallIterator.func1(0xc0e3c38300, 0x24d5f80, 0xc0e3c381e0, 0xc0006fb940, 0xc0006fad20, 0x1, 0x3, 0x3, 0xc00c467a60)
Jun 10 10:30:55 influx01 daemon[75372]: 	github.com/influxdata/influxdb/query/select.go:515 +0xe5
Jun 10 10:30:55 influx01 daemon[75372]: github.com/influxdata/influxdb/query.(*exprIteratorBuilder).buildCallIterator(0xc012b65940, 0x24d5f80, 0xc0e3c381e0, 0xc0e3c38300, 0x0, 0x0, 0xc0e3c3a000, 0x14610e0)
Jun 10 10:30:55 influx01 daemon[75372]: 	github.com/influxdata/influxdb/query/select.go:559 +0x745
Jun 10 10:30:55 influx01 daemon[75372]: github.com/influxdata/influxdb/query.buildExprIterator(0x24d5f80, 0xc0e3c381e0, 0x24cc568, 0xc0e3c38300, 0xc9a6fb1c0, 0xc0029449c0, 0xc011f058a0, 0x1, 0x1, 0x24cc568, ...)
Jun 10 10:30:55 influx01 daemon[75372]: 	github.com/influxdata/influxdb/query/select.go:156 +0x285
Jun 10 10:30:55 influx01 daemon[75372]: github.com/influxdata/influxdb/query.buildFieldIterator(0x24d5f80, 0xc0e3c381e0, 0x24cc568, 0xc0e3c38300, 0xc9a6fb1c0, 0xc0029449c0, 0xc011f058a0, 0x1, 0x1, 0x0, ...)
Jun 10 10:30:55 influx01 daemon[75372]: 	github.com/influxdata/influxdb/query/select.go:870 +0x4b6
Jun 10 10:30:55 influx01 daemon[75372]: github.com/influxdata/influxdb/query.buildCursor.func1(0xc10ac03630, 0xc00306e280)
Jun 10 10:30:55 influx01 daemon[75372]: 	github.com/influxdata/influxdb/query/select.go:744 +0x12b
Jun 10 10:30:55 influx01 daemon[75372]: golang.org/x/sync/errgroup.(*Group).Go.func1(0xc0e3c38360, 0xc05330a540)
Jun 10 10:30:55 influx01 daemon[75372]: 	golang.org/x/[email protected]/errgroup/errgroup.go:57 +0x59
Jun 10 10:30:55 influx01 daemon[75372]: created by golang.org/x/sync/errgroup.(*Group).Go
Jun 10 10:30:55 influx01 daemon[75372]: 	golang.org/x/[email protected]/errgroup/errgroup.go:54 +0x66

mhalden avatar Jun 10 '21 10:06 mhalden

Hi, We have the same issue, does someone know how to fix that:

Jul 30 11:28:45 influxd: panic: runtime error: invalid memory address or nil pointer dereference Jul 30 11:28:45 influxd: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x9617f0] Jul 30 11:28:45 influxd: goroutine 250627521 [running]: Jul 30 11:28:45 influxd: github.com/influxdata/influxdb/query.Iterators.Close(0xc19846c1e0, 0x6, 0xf, 0xc5fff2ae60, 0x25aefa0) Jul 30 11:28:45 influxd: /go/src/github.com/influxdata/influxdb/query/iterator.go:48 +0x50 Jul 30 11:28:45 influxd: github.com/influxdata/influxdb/coordinator.(*LocalShardMapping).CreateIterator(0xc3a0fb1d00, 0x259d400, 0xc477820750, 0xc5fff2ae60, 0x2592380, 0xc477820870, 0x0, 0x0, 0x0, 0x0, ...) Jul 30 11:28:45 influxd: /go/src/github.com/influxdata/influxdb/coordinator/shard_mapper.go:202 +0x33a Jul 30 11:28:45 influxd: github.com/influxdata/influxdb/query.(*exprIteratorBuilder).callIterator.func1(0xc1986618f0, 0x259d400, 0xc477820750, 0xc19865fbe0, 0xc19865fb98, 0xc477820870, 0x7b5a34215b00, 0x7b5a343f3fff) Jul 30 11:28:45 influxd: /go/src/github.com/influxdata/influxdb/query/select.go:583 +0x15e Jul 30 11:28:45 influxd: github.com/influxdata/influxdb/query.(*exprIteratorBuilder).callIterator(0xc196f898f0, 0x259d400, 0xc477820750, 0xc477820870, 0x2592380, 0xc477820870, 0x0, 0x0, 0x0, 0x0, ...) Jul 30 11:28:45 influxd: /go/src/github.com/influxdata/influxdb/query/select.go:608 +0xdf Jul 30 11:28:45 influxd: github.com/influxdata/influxdb/query.(*exprIteratorBuilder).buildCallIterator.func1(0xc477820870, 0x259d400, 0xc477820750, 0xc196f898f0, 0xc196f88dd0, 0x2592480, 0xc5fff2adc0, 0x8, 0x7ff17255ff80) Jul 30 11:28:45 influxd: /go/src/github.com/influxdata/influxdb/query/select.go:515 +0xf5 Jul 30 11:28:45 influxd: github.com/influxdata/influxdb/query.(*exprIteratorBuilder).buildCallIterator(0xc1986618f0, 0x259d400, 0xc477820750, 0xc477820870, 0x29, 0xc58f03a300, 0xc45c6aab78, 0x60) Jul 30 11:28:45 influxd: /go/src/github.com/influxdata/influxdb/query/select.go:559 +0xd3e Jul 30 11:28:45 influxd: github.com/influxdata/influxdb/query.buildExprIterator(0x259d400, 0xc477820750, 0x2592380, 0xc477820870, 0x7460312fa2d0, 0xc3a0fb1d00, 0xc35cb24dd0, 0x1, 0x1, 0x2592380, ...) Jul 30 11:28:45 influxd: /go/src/github.com/influxdata/influxdb/query/select.go:156 +0x1f4 Jul 30 11:28:45 influxd: github.com/influxdata/influxdb/query.buildFieldIterator(0x259d400, 0xc477820750, 0x2592380, 0xc477820870, 0x7460312fa2d0, 0xc3a0fb1d00, 0xc35cb24dd0, 0x1, 0x1, 0x0, ...) Jul 30 11:28:45 influxd: /go/src/github.com/influxdata/influxdb/query/select.go:870 +0x489 Jul 30 11:28:45 influxd: github.com/influxdata/influxdb/query.buildCursor.func1(0xc5976cff68, 0x0) Jul 30 11:28:45 influxd: /go/src/github.com/influxdata/influxdb/query/select.go:744 +0x127 Jul 30 11:28:45 influxd: github.com/influxdata/influxdb/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1(0xc4778208a0, 0xc5c1d7e770) Jul 30 11:28:45 influxd: /go/src/github.com/influxdata/influxdb/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x64 Jul 30 11:28:45 influxd: created by github.com/influxdata/influxdb/vendor/golang.org/x/sync/errgroup.(*Group).Go Jul 30 11:28:45 influxd: /go/src/github.com/influxdata/influxdb/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66 `

janekhh avatar Jul 30 '21 12:07 janekhh

Hi,

We are getting the same error with v1.8.7. It is happening because of the iterator being nil. There are some cases where both iterator and error are returned as nil like here.

Can we merge this PR that adds a null check before adding the iterator to the list?

cc: @lianghongji

roobalimsab avatar Aug 16 '21 11:08 roobalimsab

Hi,

We are getting the same error with v1.8.7. It is happening because of the iterator being nil. There are some cases where both iterator and error are returned as nil like here.

Can we merge this PR that adds a null check before adding the iterator to the list?

cc: @lianghongji

Sorry for the late reply, Because I have something causing me every time I look at email Of course you can merge this PR I'm very happy it can help you

hongjijun233 avatar Sep 02 '21 02:09 hongjijun233

"version": "1.8.9", "branch": "1.8", "commit": "d9b56321d579"

ts=2021-09-16T22:08:02.670131Z lvl=warn msg="Error while freeing cold shard resources" log_id=0WalGdDW000 service=store error="engine is closed" db_shard_id=87 ts=2021-09-16T22:08:02.670136Z lvl=warn msg="Error while freeing cold shard resources" log_id=0WalGdDW000 service=store error="engine is closed" db_shard_id=88 ts=2021-09-16T22:08:02.670140Z lvl=warn msg="Error while freeing cold shard resources" log_id=0WalGdDW000 service=store error="engine is closed" db_shard_id=81 ts=2021-09-16T22:08:02.670144Z lvl=warn msg="Error while freeing cold shard resources" log_id=0WalGdDW000 service=store error="engine is closed" db_shard_id=83 panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xa7402f]

goroutine 60 [running]: github.com/influxdata/influxdb/tsdb.(*databaseState).removeIndexType(...) /go/src/github.com/influxdata/influxdb/tsdb/store.go:65 github.com/influxdata/influxdb/tsdb.(*Store).DeleteShard.func1(0xc00025c800, 0x49, 0xc00a0dab5f, 0x8, 0xc000117200) /go/src/github.com/influxdata/influxdb/tsdb/store.go:733 +0x12f github.com/influxdata/influxdb/tsdb.(*Store).DeleteShard(0xc00025c800, 0x49, 0x256d6a0, 0xc0000c77b0) /go/src/github.com/influxdata/influxdb/tsdb/store.go:739 +0x707 github.com/influxdata/influxdb/services/retention.(*Service).run(0xc0001a6320) /go/src/github.com/influxdata/influxdb/services/retention/service.go:134 +0x1ff github.com/influxdata/influxdb/services/retention.(*Service).Open.func1(0xc0001a6320) /go/src/github.com/influxdata/influxdb/services/retention/service.go:51 +0x61 created by github.com/influxdata/influxdb/services/retention.(*Service).Open /go/src/github.com/influxdata/influxdb/services/retention/service.go:51 +0x1a9

liuguoshun avatar Sep 17 '21 01:09 liuguoshun

InfluxDB 2.0.8 (git: e91d41810f) build_date: 2021-08-13T18:22:30Z

panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x1d2b86d]

goroutine 3936 [running]: github.com/influxdata/influxdb/v2/storage/flux.(*floatGroupTable).advance(0xc01701bad0, 0xc01ac816d0) /home/circleci/go/src/github.com/influxdata/influxdb/storage/flux/table.gen.go:733 +0xcd github.com/influxdata/influxdb/v2/storage/flux.(*table).do(0xc01701bad0, 0xc00f6f0210, 0xc0065a1e40, 0x0, 0x0) /home/circleci/go/src/github.com/influxdata/influxdb/storage/flux/table.go:95 +0x1f0 github.com/influxdata/influxdb/v2/storage/flux.(*floatGroupTable).Do(0xc01701bad0, 0xc00f6f0210, 0x1, 0xc0209d9fa0) /home/circleci/go/src/github.com/influxdata/influxdb/storage/flux/table.gen.go:707 +0x5c github.com/influxdata/flux/execute.(*consecutiveTransportTable).Do(0xc00f6f01c8, 0xc0209d9fa0, 0x7, 0x8) /home/circleci/go/pkg/mod/github.com/influxdata/[email protected]/execute/transport.go:449 +0x74 github.com/influxdata/flux/stdlib/universe.(*limitTransformation).limitTable(0xc0179426f0, 0x3d2a038, 0xc01ac8f6c0, 0xc01ac8f700, 0x3d306f8, 0xc00f6f01c8, 0xce2185, 0xc0022900b0) /home/circleci/go/pkg/mod/github.com/influxdata/[email protected]/stdlib/universe/limit.go:137 +0xcb github.com/influxdata/flux/stdlib/universe.(*limitTransformation).Process.func1(0x3d2a038, 0xc01ac8f6c0, 0xc01ac8f700, 0x7, 0xc00324a080) /home/circleci/go/pkg/mod/github.com/influxdata/[email protected]/stdlib/universe/limit.go:127 +0x5f github.com/influxdata/flux/internal/execute/table.StreamWithContext.func1(0xc00d676480, 0xc00d6763c0, 0xc00f6f01f8, 0x3d2a038, 0xc01ac8f6c0, 0xc01ac8f700) /home/circleci/go/pkg/mod/github.com/influxdata/[email protected]/internal/execute/table/stream.go:109 +0xb2 created by github.com/influxdata/flux/internal/execute/table.StreamWithContext /home/circleci/go/pkg/mod/github.com/influxdata/[email protected]/internal/execute/table/stream.go:106 +0x1ae

The error happens with following FLUX Query:

from(bucket:"<bucket_name>") |> range(start: v.timeRangeStart, stop:v.timeRangeStop) |> group(columns: ["_measurement"]) |> limit(n:10)

mauzigoe avatar Oct 06 '21 11:10 mauzigoe

Getting the same error, 100% reproducable, on Influxdb 1.8.10 with Grafana 8.5.

Query: from(bucket: "iobroker/global") |> range(start: v.timeRangeStart, stop:v.timeRangeStop) |> filter(fn: (r) => ( r._measurement == "speed" and r._field == "value" ) |> drop(columns: ["_measurement"]) |> sort(columns: ["_time"]) |> set(key: "_field", value: "Geschwindigkeit (km/h)") |> aggregateWindow(every: v.windowPeriod, fn: mean)

Crash message: ˋ ts=2021-10-31T18:26:53.182679Z lvl=error msg="[500] - "panic: runtime error: invalid memory address or nil pointer dereference"" log_id=0XSk4i5G000 service=httpd ˋ

This happens only when I add the "aggregateWindow()" function to the chain - anywhere below the "filter()" function. Data files as reported by influx_inspect are all "healthy" (is this sufficient to be sure?).

I would appreciate any help.

Update: This only happens when using "fn: mean" with "aggregateWindow()". It does not happen with "fn: max" or "fn: min".

jensb avatar Oct 31 '21 18:10 jensb

It seems that a similar issue is still unsolved in 1.8.10: panic in query.Iterators.Close() (I' m running influx 1.8.10 on 4 servers with Red Hat Enterprise Linux Server release 6.10 (Santiago) ) I have too many queries (lot of grafana users) to easily identify the culprit but the panic is the same and it happens often (sometimes more than once per day per server) eg:

influxd.log-20211103-[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x963de0]
influxd.log-20211103:github.com/influxdata/influxdb/query.Iterators.Close(0xc08d77c000, 0x6, 0x34, 0xc0f70d79f0, 0x25b6fe0)
influxd.log-20211103-   /go/src/github.com/influxdata/influxdb/query/iterator.go:48 +0x50
influxd.log-20211103-github.com/influxdata/influxdb/coordinator.(*LocalShardMapping).CreateIterator(0xc14149f300, 0x25a5440, 0xc12aa37ce0, 0xc0f70d79f0, 0x259a3c0, 0xc12aa37e00, 0x0, 0x0, 0x0, 0x0, ...)
influxd.log-20211103-   /go/src/github.com/influxdata/influxdb/coordinator/shard_mapper.go:202 +0x33a
influxd.log-20211104-[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x963de0]
influxd.log-20211104:github.com/influxdata/influxdb/query.Iterators.Close(0xc149104a80, 0x6, 0x34, 0xc1484ec780, 0x25b6fe0)
influxd.log-20211104-   /go/src/github.com/influxdata/influxdb/query/iterator.go:48 +0x50
influxd.log-20211104-github.com/influxdata/influxdb/coordinator.(*LocalShardMapping).CreateIterator(0xc145ac9bc0, 0x25a5440, 0xc274b7f590, 0xc1484ec780, 0x259a3c0, 0xc274b7f6b0, 0x0, 0x0, 0x0, 0x0, ...)
influxd.log-20211104-   /go/src/github.com/influxdata/influxdb/coordinator/shard_mapper.go:202 +0x33a
influxd.log-20211104-[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x963de0]
influxd.log-20211104:github.com/influxdata/influxdb/query.Iterators.Close(0xc0632e0380, 0x18, 0x34, 0xc061d03ea0, 0x25b6fe0)
influxd.log-20211104-   /go/src/github.com/influxdata/influxdb/query/iterator.go:48 +0x50
influxd.log-20211104-github.com/influxdata/influxdb/coordinator.(*LocalShardMapping).CreateIterator(0xc04e201d00, 0x25a5440, 0xc2d5f328d0, 0xc061d03ea0, 0x259a3c0, 0xc2d5f329f0, 0x0, 0x0, 0x0, 0x0, ...)
influxd.log-20211104-   /go/src/github.com/influxdata/influxdb/coordinator/shard_mapper.go:202 +0x33a

Let me know if you need more details. This bug seems to be around since (at least) 1.8.3 (quick search in issues). I guess it is time to fix it for good? Thank you for your help.

neoliv avatar Nov 09 '21 10:11 neoliv

Same here: Influxdb 1.8.10, grafana 8.2.3 fluxql: from(bucket: "domotica") |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r["_measurement"] == "youless" and r._field == "youless_power") |> aggregateWindow( every: 1h, fn: sum ) fails with the panic indeed @jensb min and max succeed

kbfifi avatar Nov 10 '21 18:11 kbfifi

Alternative also results in same panic: from(bucket: "domotica") |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r["_measurement"] == "youless" and r._field == "youless_power") |> window(every: 10m) |> sum() |> window(every: inf)

It doesn't like adding ;-)

kbfifi avatar Nov 10 '21 19:11 kbfifi

I have also this issue. bummer

Screenshot 2021-11-12 at 15 28 44

mazzy89 avatar Nov 12 '21 14:11 mazzy89

I have same issue on 1.8.10, arm64. my case is using aggregate() and mean(). in a combination of aggregate() and count(), panic is not happened.

miettal avatar Dec 15 '21 05:12 miettal

See the same with InfluxDB OSS 1.8.10 in docker.

Not sure why @roobalimsab's fix https://github.com/influxdata/influxdb/pull/21068 wasn't merged.

Also there is https://github.com/influxdata/influxdb/pull/21666 merged last june into master-1.x which was then also added to 1.9 That PR references 2 more duplicates to this issue:

  • https://github.com/influxdata/influxdb/issues/19579
  • https://github.com/influxdata/influxdb/issues/19476

Are there any plans to further support 1.8? Are there any plans to release 1.9?

Marc-Assmann avatar Mar 23 '22 09:03 Marc-Assmann

I am getting the same issue in armv7, v1.8.10

2opremio avatar Apr 04 '22 02:04 2opremio

Same here in InfluxDB OSS 1.8.10 in docker (alpine). We are using regexps for the measurement and explicit time windows in an influxql query.

arrlee avatar May 11 '22 09:05 arrlee

Did anyone find the way how to resolve/fix the issue ? I am running InfluxDB 1.8.10 with Grafana 9.1.6 and need to use

aggregateWindow(every: 1mo, fn: sum)

in the flux query. Unfortunately Grafana returns [runtime error: invalid memory address or nil pointer dereference]

When I remove aggregateWindow from query or change fn: sum to fn: min then query runs without problem.

AzurovyMisak avatar Dec 01 '22 12:12 AzurovyMisak

I have same issue on InfluxDB v2.5.0 (Windows Server 2022 Standard)

parumsancto avatar Dec 06 '22 11:12 parumsancto

Same issue on Rpi 3. For me it occurs as soon as I select a larger period (e.g. 2015 to 2020) and display the data in the web interface. For shorter periods it works without problems.

Raspbian Bullseye 64bit, InfluxDB v2.6.0 Docker latest arm64/v8

Influx Server: 24a2b621ea Influx Frontent: cbab7db

ts=2022-12-28T08:49:16.491816Z lvl=info msg="Execute source panic" log_id=0f1xAgnG000 service=storage-reads error="panic: runtime error: invalid memory address or nil pointer dereference" stacktrace="goroutine 3480767 [running]:
   runtime/debug.Stack()
   /go/src/runtime/debug/stack.go:24 +0x68
github.com/influxdata/flux/execute.(*executionState).recover(0x40074f4ea0)
   /go/pkg/mod/github.com/influxdata/[email protected]/execute/recover.go:32 +0x1e0
   panic({0x7fa4f37280, 0x7fa644e7f0})
   /go/src/runtime/panic.go:838 +0x204
github.com/influxdata/influxdb/v2/tsdb/cursors.(*StringArray).Len(...)
   /root/project/tsdb/cursors/arrayvalues.gen.go:621
github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*stringArrayAscendingCursor).reset(0x400f4b16d0, 0x13ecc30f5e65c800, 0x400fd3c060?, {0x0?, 0x0?, 0x0?}, 0x4011ae8900)
   /root/project/tsdb/engine/tsm1/array_cursor.gen.go:897 +0xf4
github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*arrayCursorIterator).buildStringArrayCursor(_, {_, _}, {_, _, _}, {_, _, _}, {_, ...}, ...)
   /root/project/tsdb/engine/tsm1/array_cursor_iterator.gen.go:86 +0x35c
github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*arrayCursorIterator).Next(0x400e9781c0, {0x7fa52f6878, 0x400fd3c060}, 0x400b8d3450)
   /root/project/tsdb/engine/tsm1/array_cursor_iterator.go:71 +0x448
github.com/influxdata/influxdb/v2/storage/reads.(*multiShardArrayCursors).createCursor(0x400b8d3440, {{0x0, 0x0, 0x0}, {0x7f76b50011, 0xb, 0x3fffef}, {0x0, 0x0, 0x0}, ...})
   /root/project/storage/reads/array_cursor.go:118 +0x158
github.com/influxdata/influxdb/v2/storage/reads.(*windowAggregateResultSet).createCursor(0x4009347590, {{0x0, 0x0, 0x0}, {0x7f76b50011, 0xb, 0x3fffef}, {0x0, 0x0, 0x0}, ...})
   /root/project/storage/reads/aggregate_resultset.go:100 +0x8c
github.com/influxdata/influxdb/v2/storage/reads.(*windowAggregateResultSet).Next(0x4009347590)
   /root/project/storage/reads/aggregate_resultset.go:84 +0xf4
github.com/influxdata/influxdb/v2/storage/flux.(*windowAggregateIterator).handleRead(0x4002d5e700, 0x7fa52f6878?, {0x7fa5302a78, 0x4009347590})
   /root/project/storage/flux/reader.go:742 +0x340
github.com/influxdata/influxdb/v2/storage/flux.(*windowAggregateIterator).Do(0x4002d5e700, 0x7fa51a3fe0?)
   /root/project/storage/flux/reader.go:688 +0x2b0
github.com/influxdata/influxdb/v2/query/stdlib/influxdata/influxdb.(*Source).processTables(0x4002d74f20, {0x7fa52f6878?, 0x400fd3c060}, {0x7fa52ea1a8, 0x4002d5e700}, 0x0?)
   /root/project/query/stdlib/influxdata/influxdb/source.go:69 +0xb8
github.com/influxdata/influxdb/v2/query/stdlib/influxdata/influxdb.(*readWindowAggregateSource).run(0x4002d74f20, {0x7fa52f6878, 0x400fd3c060})
   /root/project/query/stdlib/influxdata/influxdb/source.go:303 +0xd4
github.com/influxdata/influxdb/v2/query/stdlib/influxdata/influxdb.(*Source).Run(0x4002d74f20, {0x7fa52f6878, 0x400fd3c060})
   /root/project/query/stdlib/influxdata/influxdb/source.go:50 +0x7c
github.com/influxdata/flux/execute.(*executionState).do.func2({0x7fa52f88d8?, 0x4002d74f20})
   /go/pkg/mod/github.com/influxdata/[email protected]/execute/executor.go:535 +0x310
created by github.com/influxdata/flux/execute.(*executionState).do
   /go/pkg/mod/github.com/influxdata/[email protected]/execute/executor.go:515 +0x398"

image

felixoswald avatar Dec 28 '22 09:12 felixoswald

Same here on a raspi 3 32bit. 32bit is not supported by influx > v1.8.x. This is why I am stuck with 1.8.x.

When I change aggregateWindow(every: 30m, fn: mean) to fn: median or fn: min, the error is gone, but I actually want to use mean..

tpraxl avatar Mar 04 '23 06:03 tpraxl

Gibt es zu dem genannten Problem: panic: runtime error: invalid memory adress or nil Pointer dereference mittlerweile eine Lösung. Bei mir ist auch seltsam, das dieses Problem nur bei 2 Panels von 7 auftritt. Alle sind in einem Dashboard und 5 funktionieren einwandfrei, obwohl der Code überall identisch ist. from(bucket: "DBANDY1") |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r["_measurement"] == "Laufzeit_Smart_Kettle_Pro") |> filter(fn: (r) => r["_field"] == "value") |> aggregateWindow(every: v.windowPeriod, fn: mean, createEmpty: false) |> yield(name: "mean") Ich nutze Bullseye 64 Bit

Damrak2022 avatar Mar 25 '23 07:03 Damrak2022

Same error with influx OSS 2.7 N.B. The two queries run separately without error

image

francotiveron avatar Jul 16 '23 00:07 francotiveron

Same issue here. I'm getting this error when using aggregateWindow with sum or mean as the function. For instance, count and median work fine.

image

mbalasz avatar Sep 03 '23 18:09 mbalasz

In my case, upgrading to 2.7.1 fixed the problem

francotiveron avatar Sep 03 '23 22:09 francotiveron

Having a similar issue on InfluxDB v2.7.1 (Server: https://github.com/influxdata/influxdb/tree/407fa622e9 Frontend: https://github.com/influxdata/ui/tree/9d25a2f) influx-error

I have a tag called "enabled" which i recently added. When I add exists r["enabled"] to my filter it works without a problem. The tag can have the string value of "true" and "false". Adding the extra filter doesnt solve my problem though because i want to be able to retreive historical data where the field hasnt existed. Also it doesnt seem to have problems with other tags missing.

Influx Log:

ts=2023-09-21T17:04:39.422525Z 
lvl=info 
msg="Execute source panic" 
log_id=0kPb9uzG000 
service=storage-reads 
error="panic: runtime error: invalid memory address or nil pointer dereference" 
stacktrace="
	goroutine 2308069 [running]:
	\nruntime/debug.Stack()
		\n\t/go/src/runtime/debug/stack.go:24 +0x65
	\ngithub.com/influxdata/flux/execute.(*executionState).recover(0xc024650120)
		\n\t/go/pkg/mod/github.com/influxdata/[email protected]/execute/recover.go:32 +0x208
	\npanic({0x7f874a7fdb80, 0x7f874bdbec60})
		\n\t/go/src/runtime/panic.go:884 +0x213
	\ngithub.com/influxdata/influxdb/v2/tsdb/cursors.(*StringArray).Len(...)
		\n\t/root/project/tsdb/cursors/arrayvalues.gen.go:621
	\ngithub.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*stringArrayAscendingCursor).reset(0xc024680140, 0x1784d2bf53d47291, 0xc017894d20?, {0x0?, 0xc026b68900?, 0x4?}, 0xc026d3c6c0)
		\n\t/root/project/tsdb/engine/tsm1/array_cursor.gen.go:897 +0xf0
	\ngithub.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*arrayCursorIterator).buildStringArrayCursor(_, {_, _}, {_, _, _}, {_, _, _}, {0xc01a2008e0, ...}, ...)
		\n\t/root/project/tsdb/engine/tsm1/array_cursor_iterator.gen.go:86 +0x456
	\ngithub.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*arrayCursorIterator).Next(0xc0242cb260, {0x7f874abc6c78, 0xc017894d20}, 0xc015d9e490)
		\n\t/root/project/tsdb/engine/tsm1/array_cursor_iterator.go:71 +0x596
	\ngithub.com/influxdata/influxdb/v2/storage/reads.(*multiShardArrayCursors).createCursor(0xc015d9e480, {{0x0, 0x0, 0x0}, {0x7f86b5f8fb04, 0xa, 0x3ff4fc}, {0xc026b68900, 0x4, 0x4}, ...})
		\n\t/root/project/storage/reads/array_cursor.go:118 +0x182
	\ngithub.com/influxdata/influxdb/v2/storage/reads.(*resultSet).Cursor(0xc02f6a09c0?)
		\n\t/root/project/storage/reads/resultset.go:61 +0x96
	\ngithub.com/influxdata/influxdb/v2/storage/flux.(*filterIterator).handleRead(0xc0242cb1f0, 0xc0010af800, {0x7f874abd3be0, 0xc02f6a09c0})
		\n\t/root/project/storage/flux/reader.go:197 +0x12e
	\ngithub.com/influxdata/influxdb/v2/storage/flux.(*filterIterator).Do(0xc0242cb1f0, 0x7f874aa71760?)
		\n\t/root/project/storage/flux/reader.go:174 +0x130
	\ngithub.com/influxdata/influxdb/v2/query/stdlib/influxdata/influxdb.(*Source).processTables(0xc02463c600, {0x7f874abc6c78?, 0xc017894d20}, {0x7f874abba088, 0xc0242cb1f0}, 0xb3c7b66fe6586234?)
		\n\t/root/project/query/stdlib/influxdata/influxdb/source.go:69 +0xbf
	\ngithub.com/influxdata/influxdb/v2/query/stdlib/influxdata/influxdb.(*readFilterSource).run(0xc02463c600, {0x7f874abc6c78, 0xc017894d20})
		\n\t/root/project/query/stdlib/influxdata/influxdb/source.go:146 +0xf3
	\ngithub.com/influxdata/influxdb/v2/query/stdlib/influxdata/influxdb.(*Source).Run(0xc02463c600, {0x7f874abc6c78, 0xc017894d20})
		\n\t/root/project/query/stdlib/influxdata/influxdb/source.go:50 +0xa9
	\ngithub.com/influxdata/flux/execute.(*executionState).do.func2({0x7f874abc8bf8?, 0xc02463c600})
		\n\t/go/pkg/mod/github.com/influxdata/[email protected]/execute/executor.go:535 +0x3ce
	\ncreated by github.com/influxdata/flux/execute.(*executionState).do
		\n\t/go/pkg/mod/github.com/influxdata/[email protected]/execute/executor.go:515 +0xf7
	\n"

InfluxQL works fine for querying the same data:

SELECT * FROM "measurement" WHERE "nodeId" = 'myNodeId' AND time > '2023-09-14T14:13:01.4732408Z'

MSDev201 avatar Sep 21 '23 14:09 MSDev201

I'm also experiencing this bug. Is there any workaround specifically to calculate the mean across columns of a table? Thanks.

averter avatar Apr 25 '24 12:04 averter

I'm having this issue with following query:

from(bucket: "my_bucket")
  |> range(start: time(v:1707942660000000000), stop: time(v:1716016260001000000))
  |> filter(fn: (r) => r._measurement == "my_measurement")
  |> filter(fn: (r) => r._field == "my_field")
  |> keep(columns: ["_time"])

Error log in docker:

ts=2024-05-18T11:53:57.921651Z lvl=info msg="Execute source panic" log_id=0pEtUDiG000 service=storage-reads error="panic: runtime error: invalid memory address or nil pointer dereference" stacktrace="goroutine 6194 [running]:
    runtime/debug.Stack()
        /go/src/runtime/debug/stack.go:24 +0x65
    github.com/influxdata/flux/execute.(*executionState).recover(0xc0014c86c0)
        /go/pkg/mod/github.com/influxdata/[email protected]/execute/recover.go:32 +0x208
    panic({0x7f48a6ac9b80, 0x7f48a808ac60})
        /go/src/runtime/panic.go:884 +0x213
    github.com/influxdata/influxdb/v2/tsdb/cursors.(*StringArray).Len(...)
        /root/project/tsdb/cursors/arrayvalues.gen.go:621
    github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*stringArrayAscendingCursor).reset(0xc0091b0f50, 0x17b3d4ccda092800, 0xc002df8db0?, {0x0?, 0xc002e004b0?, 0x1?}, 0xc0014c8a20)
        /root/project/tsdb/engine/tsm1/array_cursor.gen.go:897 +0xf0
    github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*arrayCursorIterator).buildStringArrayCursor(_, {_, _}, {_, _, _}, {_, _, _}, {0xc00098aa50, ...}, ...)
        /root/project/tsdb/engine/tsm1/array_cursor_iterator.gen.go:86 +0x456
    github.com/influxdata/influxdb/v2/tsdb/engine/tsm1.(*arrayCursorIterator).Next(0xc007d663f0, {0x7f48a6e92c78, 0xc002df8db0}, 0xc002fedb10)
        /root/project/tsdb/engine/tsm1/array_cursor_iterator.go:71 +0x596
    github.com/influxdata/influxdb/v2/storage/reads.(*multiShardArrayCursors).createCursor(0xc002fedb00, {{0x0, 0x0, 0x0}, {0x7f487639f202, 0xf, 0x3ffdfe}, {0xc002e004b0, 0x1, 0x1}, ...})
        /root/project/storage/reads/array_cursor.go:118 +0x182
    github.com/influxdata/influxdb/v2/storage/reads.(*resultSet).Cursor(0xc008f76a90?)
        /root/project/storage/reads/resultset.go:61 +0x96
    github.com/influxdata/influxdb/v2/storage/flux.(*filterIterator).handleRead(0xc007d66230, 0xc00165dba0, {0x7f48a6e9fbe0, 0xc008f76a90})
        /root/project/storage/flux/reader.go:197 +0x12e
    github.com/influxdata/influxdb/v2/storage/flux.(*filterIterator).Do(0xc007d66230, 0x7f48a6d3d760?)
        /root/project/storage/flux/reader.go:174 +0x130
    github.com/influxdata/influxdb/v2/query/stdlib/influxdata/influxdb.(*Source).processTables(0xc002db4600, {0x7f48a6e92c78?, 0xc002df8db0}, {0x7f48a6e86088, 0xc007d66230}, 0xece6b02ed22a690d?)
        /root/project/query/stdlib/influxdata/influxdb/source.go:69 +0xbf
    github.com/influxdata/influxdb/v2/query/stdlib/influxdata/influxdb.(*readFilterSource).run(0xc002db4600, {0x7f48a6e92c78, 0xc002df8db0})
        /root/project/query/stdlib/influxdata/influxdb/source.go:146 +0xf3
    github.com/influxdata/influxdb/v2/query/stdlib/influxdata/influxdb.(*Source).Run(0xc002db4600, {0x7f48a6e92c78, 0xc002df8db0})
        /root/project/query/stdlib/influxdata/influxdb/source.go:50 +0xa9
    github.com/influxdata/flux/execute.(*executionState).do.func2({0x7f48a6e94bf8?, 0xc002db4600})
        /go/pkg/mod/github.com/influxdata/[email protected]/execute/executor.go:535 +0x3ce
    created by github.com/influxdata/flux/execute.(*executionState).do
        /go/pkg/mod/github.com/influxdata/[email protected]/execute/executor.go:515 +0xf7
    "

Versions are:

InfluxDB v2.7.0 Server: 85f725f Frontend: 0c844dc

This query used to work fine, it stopped working recently - I did not update, nor change anything.

shalak avatar May 18 '24 11:05 shalak

Wanted to add to this thread- This can also happen with an input array if used with some of these functions before casting _time to the native _time type see example below

What was broken for me

example = ${JSON.stringify([
  {_time: "1970-01-01T00:00:00.000Z", _value: 0.0}
])}

array_values = array.from(rows: example)
|> group(columns: ["_time"])

data = from(bucket: "Bucket")
|> range(start: ${start}, stop: ${stop})
|> aggregateWindow(every: 1m, fn: sum, createEmpty: true, timeSrc: "_start")
|> pivot(
  rowKey: ["_time"],
  columnKey: ["_field"],
  valueColumn: "_value"
)
|> group(columns: ["_time"])

joined_data_on_time = join.left(left: data, right: array_values,
  on: (l, r) => l._time == r._time,
  as: (l, r) => ({
    l with array_val: r._value
  })
)

|> yield(name:"result")

What worked

example = ${JSON.stringify([
  {_time: "1970-01-01T00:00:00.000Z", _value: 0.0}
])}
|> array.map(fn: (x) => ({  
  x with _value: float(v: x._value), // the cast to float is just because of the nature of JS numbers
  _time: time(v: x._time) // this fixed the runtime error
}))

array_values = array.from(rows: example)
|> group(columns: ["_time"])

data = from(bucket: "Bucket")
|> range(start: ${start}, stop: ${stop})
|> aggregateWindow(every: 1m, fn: sum, createEmpty: true, timeSrc: "_start")
|> pivot(
  rowKey: ["_time"],
  columnKey: ["_field"],
  valueColumn: "_value"
)
|> group(columns: ["_time"])

joined_data_on_time = join.left(left: data, right: array_values,
  on: (l, r) => l._time == r._time,
  as: (l, r) => ({
    l with array_val: r._value
  })
)

|> yield(name:"result")

WaylayOvercast avatar Jun 21 '24 01:06 WaylayOvercast

The panic in iterators.Close has been fixed. Here are the some of the PRs.

https://github.com/influxdata/influxdb/pull/23319 https://github.com/influxdata/influxdb/pull/21697 https://github.com/influxdata/influxdb/pull/21666

This GitHub issue has many different unrelated problems described, but it cannot serve as the record for every panic in InfluxDB.

The problem in the first comment with a usable stack trace has been fixed, so we are closing this issue.

If you added a comment with a stack trace which does not end in Iterators.Close, please test against the latest OSS version to see if the problem replicates, and open a new issue for it. If you are commenting on an issue with a stack trace, please ensure that your stack trace matches the original, or open a new issue.

davidby-influx avatar Jul 11 '24 17:07 davidby-influx