[BUG] Unknown data type JSON

[aida-mirabadi]

Description This package raises a data type error for JSONFeild, although this type is supported and available in your sample model inside the README file.

Sample code

# models.py
class Event(models.ClickhouseModel):
    content = models.JSONField(default=dict)

# migrations/0001_initial.py
("content", clickhouse_backend.models.JSONField(default=dict)),
           (
                "timestamp",
                clickhouse_backend.models.DateTime64Field(
                    default=django.utils.timezone.now
                ),
           ),

# command
$ python manage.py migrate --database clickhouse

Error

Exception: Unknown data type family: JSON. Stack trace:

0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xaebee1a in /usr/bin/clickhouse
1. DB::DataTypeFactory::findCreatorByName(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const @ 0x14378ba4 in /usr/bin/clickhouse
2. DB::DataTypeFactory::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::IAST> const&) const @ 0x14377bfa in /usr/bin/clickhouse
3. DB::InterpreterCreateQuery::getColumnsDescription(DB::ASTExpressionList const&, std::__1::shared_ptr<DB::Context const>, bool) @ 0x14bb5ae5 in /usr/bin/clickhouse
4. DB::InterpreterCreateQuery::getTablePropertiesAndNormalizeCreateQuery(DB::ASTCreateQuery&) const @ 0x14bb7bf7 in /usr/bin/clickhouse
5. DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x14bbef16 in /usr/bin/clickhouse
6. DB::InterpreterCreateQuery::execute() @ 0x14bc6f6b in /usr/bin/clickhouse
7. ? @ 0x14ee9bd9 in /usr/bin/clickhouse
8. DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x14ee7755 in /usr/bin/clickhouse
9. DB::TCPHandler::runImpl() @ 0x159f059a in /usr/bin/clickhouse
10. DB::TCPHandler::run() @ 0x15a04579 in /usr/bin/clickhouse
11. Poco::Net::TCPServerConnection::start() @ 0x18668b6f in /usr/bin/clickhouse
12. Poco::Net::TCPServerDispatcher::run() @ 0x1866afc1 in /usr/bin/clickhouse
13. Poco::PooledThread::run() @ 0x1881b6a9 in /usr/bin/clickhouse
14. Poco::ThreadImpl::runnableEntry(void*) @ 0x18818da0 in /usr/bin/clickhouse
15. ? @ 0x7fee4afed609 in ?
16. __clone @ 0x7fee4af14293 in ?

Expected behavior As you have supported JSONField inside the models/fields/json.py, I assume it would be no problem to create a column with JSON datatype. Of course, as far as I know, Clickhouse doesn't support JSON by default and it's necessary to save it as a string and then convert it to JSON in result.

Versions

• ClickHouse 22.2.3.1
• Python 3.12
• Clickhouse-driver 0.2.8
• Django 5.0.7
• Django clickhouse backend 1.3.0

[jayvynl]

JSON type is a experiment feature of ClickHouse, enable this by set allow_experimental_object_type:

DATABASES = {
    "default": {
        "ENGINE": "clickhouse_backend.backend",
        "OPTIONS": {
            "settings": {
                "allow_experimental_object_type": 1,
            },
        }
    }
}

[aida-mirabadi]

Thanks for your response @jayvynl Even though I added this option, I have the same error. Would you have any idea?

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "HOST": os.getenv("POSTGRES_HOST", "localhost"),
        "PORT": os.getenv("POSTGRES_PORT", "5432"),
        "USER": os.getenv("POSTGRES_USER", "user"),
        "PASSWORD": os.getenv("POSTGRES_PASSWORD", "password"),
        "NAME": os.getenv("POSTGRES_DB", "db"),
    },
    "clickhouse": {
        "ENGINE": "clickhouse_backend.backend",
        "OPTIONS": {
            "settings": {
                "allow_experimental_object_type": 1,
            },
        },
        "HOST": os.getenv("CLICKHOUSE_HOST", "localhost"),
        "PORT": os.getenv("CLICKHOUSE_PORT", "9000"),
        "USER": os.getenv("CLICKHOUSE_USER", "user"),
        "PASSWORD": os.getenv("CLICKHOUSE_PASSWORD", "password"),
        "NAME": os.getenv("CLICKHOUSE_DB", "db"),
    }
}

Error

DB::Exception: Unknown data type family: JSON

[jayvynl]

Maybe your clickhouse version is too low？

[aida-mirabadi]

I'm using almost the latest version of the Clickhouse docker. Mine is 22.2.3.1. I think, there is something else.

https://hub.docker.com/r/clickhouse/clickhouse-server

Branch tags like 22.2 point to the latest release of the corresponding branch.
Full version tags like 22.2.3.5 point to the corresponding release.

[jayvynl]

Latest version should be 24.x.y.z , clickhouse is versioned by year month.

[jayvynl]

Object('json') type is not production-ready and is now deprecated.

From ClickHouse 24.8 LTS, the data type name "JSON" refers to the new json type.

I have fix problem when use JSONField in ClickHouse 24.8. But JSONField use the deprecated Object('json') type. Support of new json type is not planed.