KafkaSQL storage log compaction bug and ccompat/v6 API

[redsk]

Recently, I've run into this bug.

What I have done:

1. Disabled compaction as show in the article
2. Upgraded to apicurio 2.2.3.Final from 2.0.0.Final
3. Reset the updated sequences how shown in the article:

2022-05-25 13:46:56 INFO <_> [io.apicurio.registry.storage.impl.sql.AbstractSqlRegistryStorage] (KSQL Kafka Consumer Thread) Resetting globalId sequence
2022-05-25 13:46:56 INFO <_> [io.apicurio.registry.storage.impl.sql.AbstractSqlRegistryStorage] (KSQL Kafka Consumer Thread) Successfully reset globalId to 7774
2022-05-25 13:46:56 INFO <_> [io.apicurio.registry.storage.impl.sql.AbstractSqlRegistryStorage] (KSQL Kafka Consumer Thread) Resetting contentId sequence
2022-05-25 13:46:56 INFO <_> [io.apicurio.registry.storage.impl.sql.AbstractSqlRegistryStorage] (KSQL Kafka Consumer Thread) Successfully reset contentId to 7254
2022-05-25 13:46:56 INFO <_> [io.apicurio.common.apps.logging.audit.AuditLogService] (executor-thread-6) apicurio.audit action="importData" result="success" src_ip="<redacted>" 

However, the ccompat/v6 API does not seem to work as expected. Context: I'm using Avro schemas, Kafka, and Debezium.

Here I want to retrieve the schema by Id (in the example it's 5914, it's what I get by reading the bytes from 1 to 5 in the Kafka payload)

If I use v2 api:

curl -X GET apicurio-registry-service/apis/registry/v2/ids/contentIds/5914
<returns the schema>

If I use ccompat/v6 api:

curl -X GET apicurio-registry-service/apis/ccompat/v6/schemas/ids/5914
{"message":"Failed to discover artifact type from content.","error_code":0}

I'm not sure if I'm doing something wrong or if this is a bug with the updated sequences fix.

FYI: @jsenko

[redsk]

Adding more details:

There are two versions for the subject I'm using, one with schema id 521, and one with schema id 5914. I did the following:

curl -X GET apicurio-registry-service/apis/ccompat/v6/schemas/ids/521/versions
[{"subject":"apidb.public.balance_charges-value","version":1}]

curl -X GET apicurio-registry-service/apis/ccompat/v6/schemas/ids/5914/versions
[{"subject":"apidb.public.balance_charges-value","version":2}]

curl -X GET apicurio-registry-service/apis/ccompat/v6/schemas/ids/521
{"message":"Failed to discover artifact type from content.","error_code":0}

curl -X GET apicurio-registry-service/apis/ccompat/v6/schemas/ids/5914
{"message":"Failed to discover artifact type from content.","error_code":0}

[jsenko]

hi @redsk Are you able to show us the schema you are using? The error happens in the artifact type discovery code, which is not used in the non-compat API call, though the underlying cause may be something different.

[jsenko]

Did you use compat API to create the artifact as well, or did you create it using UI (via registry v2 API)? Because the specific type discovery code is used in the former, but not the latter.

EDIT: The discovery for creation was added to compat API after 2.0.0. So you may want to try it again with 2.2.3.