clickhouse-driver icon indicating copy to clipboard operation
clickhouse-driver copied to clipboard

"KeyError" when insert in Nested

Open MAH69IK opened this issue 7 months ago • 0 comments

Describe the bug I'm trying to use the example from the documentation but I'm encountering an error:

$ python /tmp/a.py
Traceback (most recent call last):
  File "/tmp/a.py", line 17, in <module>
    ch_client.execute('INSERT INTO security_events.test_nested VALUES', [{"col": [{"name": "a", "version": 100}]}])
  File "/home/mah69ik/.vmedio/cbc/lib/python3.12/site-packages/clickhouse_driver/client.py", line 376, in execute
    rv = self.process_insert_query(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mah69ik/.vmedio/cbc/lib/python3.12/site-packages/clickhouse_driver/client.py", line 607, in process_insert_query
    rv = self.send_data(sample_block, data,
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mah69ik/.vmedio/cbc/lib/python3.12/site-packages/clickhouse_driver/client.py", line 658, in send_data
    block = block_cls(sample_block.columns_with_types, chunk,
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mah69ik/.vmedio/cbc/lib/python3.12/site-packages/clickhouse_driver/block.py", line 40, in __init__
    self.data = self.normalize(data or [])
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mah69ik/.vmedio/cbc/lib/python3.12/site-packages/clickhouse_driver/block.py", line 128, in normalize
    self._mutate_dicts_to_rows(data)
  File "/home/mah69ik/.vmedio/cbc/lib/python3.12/site-packages/clickhouse_driver/block.py", line 159, in _mutate_dicts_to_rows
    return self._pure_mutate_dicts_to_rows(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mah69ik/.vmedio/cbc/lib/python3.12/site-packages/clickhouse_driver/block.py", line 186, in _pure_mutate_dicts_to_rows
    new_data.append(row[name])
                    ~~~^^^^^^
KeyError: 'col.name'

To Reproduce Clickhouse:

CREATE TABLE security_events.test_nested (col Nested(name String, version UInt32)) Engine = Memory;
DESCRIBE TABLE security_events.test_nested;
   ┌─name────────┬─type──────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐
1. │ col.name    │ Array(String) │              │                    │         │                  │                │
2. │ col.version │ Array(UInt32) │              │                    │         │                  │                │
   └─────────────┴───────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘

Python:

#!/usr/bin/env python3

from clickhouse_driver import Client

CLICKHOUSE_ADDRESS = '...'
CLICKHOUSE_USER = '...'
CLICKHOUSE_PASSWORD = '...'

ch_client = Client(CLICKHOUSE_ADDRESS, user=CLICKHOUSE_USER, password=CLICKHOUSE_PASSWORD, compression=True, settings={"flatten_nested": 0})

ch_client.execute('INSERT INTO security_events.test_nested VALUES', [{"col": [{"name": "a", "version": 100}]}])

Expected behavior The data is successfully inserted

Versions

  • Version of package with the problem:
$ pip list | grep clickhouse-driver
clickhouse-driver                0.2.8
  • ClickHouse server version. Version can be obtained by running SELECT version() query: 24.4.1.2088.
  • Python version:
$ python --version
Python 3.12.3

MAH69IK avatar Jul 01 '24 13:07 MAH69IK