pulsar-resources-operator icon indicating copy to clipboard operation
pulsar-resources-operator copied to clipboard

v0.9.0 deletes topic schema

Open piotr-bzdyl-vertexinc opened this issue 8 months ago • 0 comments

After upgrading to v0.9.0 we have noticed that the operator deletes topic schemas when reconciling PulsarTopic.

Before the upgrade we had the following topic definition:

apiVersion: resource.streamnative.io/v1alpha1
kind: PulsarTopic
metadata:
  name: test-topic
spec:
  connectionRef:
    name: test-cluster
  name: persistent://test-tenant/test-ns/test-topic
  persistent: true

We had producers and consumers connected to this topic with Schema/STRING schema declaration and the topic had the following schemas:

$ pulsar-admin schemas get persistent://test-tenant/test-ns/test-topic
{
  "version": 2,
  "schemaInfo": {
    "name": "test-topic",
    "schema": "",
    "type": "STRING",
    "timestamp": 1744809500469,
    "properties": {}
  }
}

Some time after we upgraded from v0.8.1 to v0.9.0, we have modified the resource to:

apiVersion: resource.streamnative.io/v1alpha1
kind: PulsarTopic
metadata:
  name: test-topic
spec:
  connectionRef:
    name: test-cluster
  name: persistent://test-tenant/test-ns/test-topic
  persistent: true
  retentionSize: "-1"
  retentionTime: 14d

After the operator reconciled it, the schemas got deleted:

pulsar-admin schemas get persistent://test-tenant/test-ns/test-topic
Schema not found

Reason: Schema not found

This caused errors for our consumers trying to connect:

Failed to add schema to an active topic with empty(BYTES) schema: new schema type STRING

(this error message appears in pulsar broker logs and is also returned and thrown as an exception in the consuming application).

We had to workaround it by explicitly adding spec.schemaInfo to PulsarTopic resource:

apiVersion: resource.streamnative.io/v1alpha1
kind: PulsarTopic
metadata:
  name: test-topic
spec:
  connectionRef:
    name: test-cluster
  name: persistent://test-tenant/test-ns/test-topic
  persistent: true
  retentionSize: "-1"
  retentionTime: 14d
  schemaInfo:
    type: STRING
    schema: ""

piotr-bzdyl-vertexinc avatar Apr 17 '25 07:04 piotr-bzdyl-vertexinc