franz-go
franz-go copied to clipboard
twmb
unable to initialize sasl
clickhouse_sinker uses franz-go as the kafka client library by default. However I noticed it fails to auth with Kafka via Kerberos.
The following is the clickhouse_sinker log (including franz-go debug log, auth failed): sinker_franz.log
The following is the clickhouse_sinker log (changed to use sarama as the kafka client library, including sarama debug log, auth OK): sinker_sarama.log
Both franz-go and sarama use github.com/jcmturner/gokrb5/v8 v8.4.2. I guess it's a franz-go issue. Could you help to check it?
FYI. clickhouse_sinker prints its config into log: going to apply the first config
This does look weird. The error is coming within the client and happens when the Username field is not set. I see you're initializing both sarama and franz-go identically, so I would expect both to have the same username (and thus, same error or same no-error).
Can you use github.com/davecgh/go-spew and spew.Dump auth.Client just above this line? https://github.com/housepower/clickhouse_sinker/blob/019fa9e30e903c3051e1568f145d3e1ce48adcb5/input/kafka_franz.go#L153
I'm wondering if the internal struct doesn't have a username present for some reason.
(this might need some fields to be redacted on your side)
I changed the code per your seggestion.
[eoi@eoi03 zhichyu]$ rm -f sinker.log && ./clickhouse_sinker --local-cfg-file sinker_franz.json --log-paths=stdout,sinker_franz2.log
{"level":"info","ts":"2022-07-29T22:06:28.588+0800","msg":"version v2.5.2-1-gc66b25e-dirty, commit c66b25ed1e5cc49e1ef95be4cecd68a374b72502, date 2022-07-29T22:06:04+0800, builtBy eoi@ck16, pid 35523"}
{"level":"info","ts":"2022-07-29T22:06:28.589+0800","msg":"clickhouse_sinker initialization"}
{"level":"info","ts":"2022-07-29T22:06:28.589+0800","msg":"Run http server at http://192.168.101.27:35959/"}
{"level":"info","ts":"2022-07-29T22:06:28.589+0800","msg":"get config from local file sinker_franz.json"}
{"level":"info","ts":"2022-07-29T22:06:28.589+0800","msg":"clickhouse_sinker initialization completed"}
{"level":"info","ts":"2022-07-29T22:06:28.591+0800","msg":"going to apply the first config","config":{"Kafka":{"Brokers":"eoi03.aiops.com:9092","Version":"2.0.0","Security":{"sasl.jaas.config":"com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true debug=true keyTab=\"/home/eoi/app/kafka_2.13-2.8.1/config/kerberos/kafka02.keytab\" principal=\"[email protected]\";","sasl.kerberos.service.name":"kafka","sasl.mechanism":"GSSAPI","security.protocol":"SASL_PLAINTEXT","ssl.enabled.protocols":"","ssl.endpoint.identification.algorithm":"","ssl.key.password":"","ssl.keystore.location":"","ssl.keystore.password":"","ssl.keystore.type":"","ssl.truststore.location":"","ssl.truststore.password":"","ssl.truststore.type":""},"TLS":{"Enable":false,"CaCertFiles":"","ClientCertFile":"","ClientKeyFile":"","TrustStoreLocation":"","TrustStorePassword":"","KeystoreLocation":"","KeystorePassword":"","EndpIdentAlgo":""},"Sasl":{"Enable":true,"Mechanism":"GSSAPI","Username":"","Password":"","GSSAPI":{"AuthType":2,"KeyTabPath":"/home/eoi/app/kafka_2.13-2.8.1/config/kerberos/kafka02.keytab","KerberosConfigPath":"/etc/krb5.conf","ServiceName":"kafka","Username":"eoi","Password":"","Realm":"AIOPS.COM","DisablePAFXFAST":false}}},"Clickhouse":{"Cluster":"physic_cluster","DB":"sensor","Hosts":[["192.168.103.26"]],"Port":9000,"Username":"eoi","Password":"123456","DsnParams":"","Secure":false,"InsecureSkipVerify":false,"RetryTimes":0,"MaxOpenConns":7},"Task":null,"Tasks":[{"Name":"sensor-online-detect","KafkaClient":"franz","Topic":"sensor-online-detect","ConsumerGroup":"sensor_sync_tb_result__consumer_group2","Earliest":false,"Parser":"fastjson","CsvFormat":null,"Delimiter":"","TableName":"tb_source","AutoSchema":true,"ExcludeColumns":null,"dims":null,"DynamicSchema":{"Enable":false,"MaxDims":0,"WhiteList":"","BlackList":""},"PrometheusSchema":false,"PromLabelsBlackList":"","LoadSeriesAtStartup":false,"flushInterval":3,"bufferSize":32768,"timeZone":"Local","timeUnit":1}],"Assignment":{"Version":0,"UpdatedAt":0,"UpdatedBy":"","Map":null},"LogLevel":"debug"}}
{"level":"info","ts":"2022-07-29T22:06:28.591+0800","msg":"clickhouse.Open succeeded","dbVer":1,"replica":"192.168.103.26:9000"}
{"level":"info","ts":"2022-07-29T22:06:28.592+0800","msg":"initialized parsing pool","maxWorkers":8,"queueSize":65536}
{"level":"info","ts":"2022-07-29T22:06:28.592+0800","msg":"initialized writing pool","maxWorkers":7,"queueSize":3}
{"level":"info","ts":"2022-07-29T22:06:28.592+0800","msg":"task initializing","task":"sensor-online-detect"}
{"level":"info","ts":"2022-07-29T22:06:28.602+0800","msg":"Prepare sql=> INSERT INTO `sensor`.`tb_source` (`timestamp`,`sync_id`,`value`,`item_guid`,`metric_name`,`instance_name`,`tier1_name`,`tier2_name`,`tier3_name`,`tier4_name`,`tier5_name`)","task":"sensor-online-detect"}
{"level":"info","ts":"2022-07-29T22:06:28.602+0800","msg":"executing sql=> SELECT name FROM system.tables WHERE engine='Distributed' AND database='sensor' AND match(create_table_query, 'Distributed\\(\\'physic_cluster\\', \\'sensor\\', \\'tb_source\\'.*\\)')","task":"sensor-online-detect"}
(*client.Client)(0xc0002f6b70)({
Credentials: (*credentials.Credentials)(0xc0001620c0)({
username: (string) (len=3) "eoi",
displayName: (string) (len=3) "eoi",
realm: (string) (len=9) "AIOPS.COM",
cname: (types.PrincipalName) {
NameType: (int32) 1,
NameString: ([]string) (len=1 cap=1) {
(string) (len=3) "eoi"
}
},
keytab: (*keytab.Keytab)(0xc0000b8be0)(KVNO Timestamp Principal ET Key
---- ----------------- -------------------------------------------------------- -- ----------------------------------------------------------------
5 11/07/22 12:02:52 kafka/[email protected] 18 6cf1dc96bd9f2e1d1ec818ca2599941948dee6a6fb492024355551d71438411d
5 11/07/22 12:02:52 kafka/[email protected] 17 414e57dff2bf45ed2e91d39f9de99e7f
5 11/07/22 12:02:52 kafka/[email protected] 16 fe296ea1f14f1ae0465bdffd54384f85910ed9da10a2a483
5 11/07/22 12:02:52 kafka/[email protected] 23 f4a6f0e1d0fbc532880006a922c12e48
5 11/07/22 12:02:52 kafka/[email protected] 26 922d8b83d9ed1d04a22fa08139ddb7155a352701a7d62c576443a5e9ab1b14e2
5 11/07/22 12:02:52 kafka/[email protected] 25 b685c6bccd9b67294426438989889178
5 11/07/22 12:02:52 kafka/[email protected] 8 d07c04cda2b07a7c
5 11/07/22 12:02:52 kafka/[email protected] 3 732a64132586a8f8
4 11/07/22 12:03:01 zookeeper/[email protected] 18 9e1290de287262abefc64293843bc62c2aef23d9d63ccfe64d418d86cdb6d37a
4 11/07/22 12:03:01 zookeeper/[email protected] 17 58dd76b1cc3cb70647110351ef0d4fdf
4 11/07/22 12:03:01 zookeeper/[email protected] 16 d9702a86b3ab6bae3dfb495810545757a8689832250bece9
4 11/07/22 12:03:01 zookeeper/[email protected] 23 e85904c40792b0630047bd12da113ffa
4 11/07/22 12:03:01 zookeeper/[email protected] 26 ea516fa0b2c7da0d7293932e05d92abe547bcc149621a501fd406a5e98a9d8ad
4 11/07/22 12:03:01 zookeeper/[email protected] 25 2634c6631c2f7759b884c757c21c531d
4 11/07/22 12:03:01 zookeeper/[email protected] 8 a4d567e6d3b3d65e
4 11/07/22 12:03:01 zookeeper/[email protected] 3 08c77c9291570bdf
7 11/07/22 20:40:48 kafka/[email protected] 18 ab6b7057a69e2ae2a3a120ea9b5a7646a3584923aba997a6e8cb8e59424debc4
7 11/07/22 20:40:48 kafka/[email protected] 17 08341932d6e53f3a2fe22e9d092f2692
7 11/07/22 20:40:48 kafka/[email protected] 16 bf40269d6b8c76fe02fbd0ae045dce522a0e4029ec0483a2
7 11/07/22 20:40:48 kafka/[email protected] 23 9f0523065f87ceb1ff245234f274394b
7 11/07/22 20:40:48 kafka/[email protected] 26 70e46cefd514eceef9066b5818fc650f5cd2f35a73fc158e40b958e350d297a8
7 11/07/22 20:40:48 kafka/[email protected] 25 55a9ea3ded1a0a1540822820697b28e3
7 11/07/22 20:40:48 kafka/[email protected] 8 f1348f8ffdf25257
7 11/07/22 20:40:48 kafka/[email protected] 3 d5d93d79799231a7
8 11/07/22 20:41:09 zookeeper/[email protected] 18 f42aa34010972483a792d456683bbda6d9b241b6fdb97f6da62ae9512572f332
8 11/07/22 20:41:09 zookeeper/[email protected] 17 01a815580801a0d0d8565a13e9d80091
8 11/07/22 20:41:09 zookeeper/[email protected] 16 d5800e4abc2a57e0250425f854314f75ea9216fbec831cb9
8 11/07/22 20:41:09 zookeeper/[email protected] 23 81183219ced8a01a9c0f2051eafb57a1
8 11/07/22 20:41:09 zookeeper/[email protected] 26 dbdee2352fc471414f1d56ddd53fa232d9206df4bfc49c984c49cc83a6fc68b3
8 11/07/22 20:41:09 zookeeper/[email protected] 25 47fc5df943a11d2fe55efcef9da9cd87
8 11/07/22 20:41:09 zookeeper/[email protected] 8 c1c4a486b346589b
8 11/07/22 20:41:09 zookeeper/[email protected] 3 d94c9e1537a29d8c
10 14/07/22 11:36:52 zookeeper/[email protected] 18 97479e2d7e30d982c702caa0edceafe02d9936d253627d78ec74edc71c575291
10 14/07/22 11:36:52 zookeeper/[email protected] 17 14f44604fbfdd484a84f478cdd8bd97f
10 14/07/22 11:36:52 zookeeper/[email protected] 16 404f9ec8fbc867ef643b4cb64c384a54e55b5825f864ad37
10 14/07/22 11:36:52 zookeeper/[email protected] 23 dd0d09de05c0a2a347b05f18347fb969
10 14/07/22 11:36:52 zookeeper/[email protected] 26 10a23bc7c946d90275925b247ea6d36fe6a3d5e44451538f24058ac4423c3b0e
10 14/07/22 11:36:52 zookeeper/[email protected] 25 3354e3a8db5493ac665e90d8c263b9b3
10 14/07/22 11:36:52 zookeeper/[email protected] 8 a120da29b5589497
10 14/07/22 11:36:52 zookeeper/[email protected] 3 dff84340f7c43445
8 14/07/22 11:37:01 kafka/[email protected] 18 07a6eae411da42234788c3d078de3e8dc685fd1f8e18ff07bdf4a6dce7ef6a9a
8 14/07/22 11:37:01 kafka/[email protected] 17 e6b984e4ee106d0afed0b62a9e6e939e
8 14/07/22 11:37:01 kafka/[email protected] 16 d32526d60ea8cd7cb023e6bcfdf737517c1c0bc2ad3ec168
8 14/07/22 11:37:01 kafka/[email protected] 23 a621da6d608aecc31b9b2ff369562de8
8 14/07/22 11:37:01 kafka/[email protected] 26 0dccf1a2dc5ffcb657352a4975e0d36ddd0de121e03bb8b27cd4f7c69ae1ecfc
8 14/07/22 11:37:01 kafka/[email protected] 25 4c674c589fd908a6b2fcc7fafae2a7cf
8 14/07/22 11:37:01 kafka/[email protected] 8 b06464c77367b329
8 14/07/22 11:37:01 kafka/[email protected] 3 31ba3b43b3297a0d
3 14/07/22 11:37:11 [email protected] 18 148e8570d34db908cd11adb7fb4bdcb9c50c3ce95728be8f866a7e357c5c8369
3 14/07/22 11:37:11 [email protected] 17 f10c98bc3e636ca496d9b16497377e7e
3 14/07/22 11:37:11 [email protected] 16 b3d9ecd0fb6737763437453886f1204664bc237f0e5e2604
3 14/07/22 11:37:11 [email protected] 23 489223d95398e212603b33d14ee080ca
3 14/07/22 11:37:11 [email protected] 26 1440c3888abcdaf3749ec3ea42ac2547e57be7c97bdf954e560f0b130ef08fa9
3 14/07/22 11:37:11 [email protected] 25 a73f2824200324eef9e53a1283800480
3 14/07/22 11:37:11 [email protected] 8 67a2ad9852731f62
3 14/07/22 11:37:11 [email protected] 3 adc11fd5168f54d9
),
password: (string) "",
attributes: (map[string]interface {}) {
},
validUntil: (time.Time) 0001-01-01 00:00:00 +0000 UTC,
authenticated: (bool) false,
human: (bool) true,
authTime: (time.Time) 0001-01-01 00:00:00 +0000 UTC,
groupMembership: (map[string]bool) {
},
sessionID: (string) (len=36) "e137314f-0970-82d2-fa58-07459c341cfb"
}),
Config: (*config.Config)(0xc000252e00)({
LibDefaults: (config.LibDefaults) {
AllowWeakCrypto: (bool) false,
Canonicalize: (bool) false,
CCacheType: (int) 4,
Clockskew: (time.Duration) 5m0s,
DefaultClientKeytabName: (string) (len=43) "/usr/local/var/krb5/user/1000/client.keytab",
DefaultKeytabName: (string) (len=16) "/etc/krb5.keytab",
DefaultRealm: (string) (len=9) "AIOPS.COM",
DefaultTGSEnctypes: ([]string) (len=9 cap=9) {
(string) (len=23) "aes256-cts-hmac-sha1-96",
(string) (len=23) "aes128-cts-hmac-sha1-96",
(string) (len=13) "des3-cbc-sha1",
(string) (len=16) "arcfour-hmac-md5",
(string) (len=20) "camellia256-cts-cmac",
(string) (len=20) "camellia128-cts-cmac",
(string) (len=11) "des-cbc-crc",
(string) (len=11) "des-cbc-md5",
(string) (len=11) "des-cbc-md4"
},
DefaultTktEnctypes: ([]string) (len=9 cap=9) {
(string) (len=23) "aes256-cts-hmac-sha1-96",
(string) (len=23) "aes128-cts-hmac-sha1-96",
(string) (len=13) "des3-cbc-sha1",
(string) (len=16) "arcfour-hmac-md5",
(string) (len=20) "camellia256-cts-cmac",
(string) (len=20) "camellia128-cts-cmac",
(string) (len=11) "des-cbc-crc",
(string) (len=11) "des-cbc-md5",
(string) (len=11) "des-cbc-md4"
},
DefaultTGSEnctypeIDs: ([]int32) (len=3 cap=4) {
(int32) 18,
(int32) 17,
(int32) 23
},
DefaultTktEnctypeIDs: ([]int32) (len=3 cap=4) {
(int32) 18,
(int32) 17,
(int32) 23
},
DNSCanonicalizeHostname: (bool) true,
DNSLookupKDC: (bool) false,
DNSLookupRealm: (bool) false,
ExtraAddresses: ([]net.IP) <nil>,
Forwardable: (bool) true,
IgnoreAcceptorHostname: (bool) false,
K5LoginAuthoritative: (bool) false,
K5LoginDirectory: (string) (len=9) "/home/eoi",
KDCDefaultOptions: (asn1.BitString) {
Bytes: ([]uint8) (len=4 cap=8) {
00000000 00 00 00 10 |....|
},
BitLength: (int) 32
},
KDCTimeSync: (int) 1,
NoAddresses: (bool) true,
PermittedEnctypes: ([]string) (len=9 cap=9) {
(string) (len=23) "aes256-cts-hmac-sha1-96",
(string) (len=23) "aes128-cts-hmac-sha1-96",
(string) (len=13) "des3-cbc-sha1",
(string) (len=16) "arcfour-hmac-md5",
(string) (len=20) "camellia256-cts-cmac",
(string) (len=20) "camellia128-cts-cmac",
(string) (len=11) "des-cbc-crc",
(string) (len=11) "des-cbc-md5",
(string) (len=11) "des-cbc-md4"
},
PermittedEnctypeIDs: ([]int32) (len=3 cap=4) {
(int32) 18,
(int32) 17,
(int32) 23
},
PreferredPreauthTypes: ([]int) (len=4 cap=4) {
(int) 17,
(int) 16,
(int) 15,
(int) 14
},
Proxiable: (bool) false,
RDNS: (bool) false,
RealmTryDomains: (int) -1,
RenewLifetime: (time.Duration) 0s,
SafeChecksumType: (int) 8,
TicketLifetime: (time.Duration) 24h0m0s,
UDPPreferenceLimit: (int) 1465,
VerifyAPReqNofail: (bool) false
},
Realms: ([]config.Realm) (len=1 cap=1) {
(config.Realm) {
Realm: (string) (len=9) "AIOPS.COM",
AdminServer: ([]string) (len=1 cap=1) {
(string) (len=15) "eoi03.aiops.com"
},
DefaultDomain: (string) "",
KDC: ([]string) (len=1 cap=1) {
(string) (len=18) "eoi03.aiops.com:88"
},
KPasswdServer: ([]string) (len=1 cap=1) {
(string) (len=19) "eoi03.aiops.com:464"
},
MasterKDC: ([]string) <nil>
}
},
DomainRealm: (config.DomainRealm) (len=2) {
(string) (len=10) ".aiops.com": (string) (len=9) "AIOPS.COM",
(string) (len=9) "aiops.com": (string) (len=9) "AIOPS.COM"
}
}),
settings: (*client.Settings)(0xc0005db110)({
disablePAFXFast: (bool) false,
assumePreAuthentication: (bool) false,
preAuthEType: (int32) 0,
logger: (*log.Logger)(<nil>)
}),
sessions: (*client.sessions)(0xc000414c80)({
Entries: (map[string]*client.session) {
},
mux: (sync.RWMutex) {
w: (sync.Mutex) {
state: (int32) 0,
sema: (uint32) 0
},
writerSem: (uint32) 0,
readerSem: (uint32) 0,
readerCount: (int32) 0,
readerWait: (int32) 0
}
}),
cache: (*client.Cache)(0xc000400020)({
Entries: (map[string]client.CacheEntry) {
},
mux: (sync.RWMutex) {
w: (sync.Mutex) {
state: (int32) 0,
sema: (uint32) 0
},
writerSem: (uint32) 0,
readerSem: (uint32) 0,
readerCount: (int32) 0,
readerWait: (int32) 0
}
})
})
{"level":"debug","ts":"2022-07-29T22:06:28.620+0800","msg":"franz","auth.Client":"(*client.Client)(0xc0002f6b70)({\n Credentials: (*credentials.Credentials)(0xc0001620c0)({\n username: (string) (len=3) \"eoi\",\n displayName: (string) (len=3) \"eoi\",\n realm: (string) (len=9) \"AIOPS.COM\",\n cname: (types.PrincipalName) {\n NameType: (int32) 1,\n NameString: ([]string) (len=1 cap=1) {\n (string) (len=3) \"eoi\"\n }\n },\n keytab: (*keytab.Keytab)(0xc0000b8be0)(KVNO Timestamp Principal ET Key\n---- ----------------- -------------------------------------------------------- -- ----------------------------------------------------------------\n 5 11/07/22 12:02:52 kafka/[email protected] 18 6cf1dc96bd9f2e1d1ec818ca2599941948dee6a6fb492024355551d71438411d\n 5 11/07/22 12:02:52 kafka/[email protected] 17 414e57dff2bf45ed2e91d39f9de99e7f \n 5 11/07/22 12:02:52 kafka/[email protected] 16 fe296ea1f14f1ae0465bdffd54384f85910ed9da10a2a483 \n 5 11/07/22 12:02:52 kafka/[email protected] 23 f4a6f0e1d0fbc532880006a922c12e48 \n 5 11/07/22 12:02:52 kafka/[email protected] 26 922d8b83d9ed1d04a22fa08139ddb7155a352701a7d62c576443a5e9ab1b14e2\n 5 11/07/22 12:02:52 kafka/[email protected] 25 b685c6bccd9b67294426438989889178 \n 5 11/07/22 12:02:52 kafka/[email protected] 8 d07c04cda2b07a7c \n 5 11/07/22 12:02:52 kafka/[email protected] 3 732a64132586a8f8 \n 4 11/07/22 12:03:01 zookeeper/[email protected] 18 9e1290de287262abefc64293843bc62c2aef23d9d63ccfe64d418d86cdb6d37a\n 4 11/07/22 12:03:01 zookeeper/[email protected] 17 58dd76b1cc3cb70647110351ef0d4fdf \n 4 11/07/22 12:03:01 zookeeper/[email protected] 16 d9702a86b3ab6bae3dfb495810545757a8689832250bece9 \n 4 11/07/22 12:03:01 zookeeper/[email protected] 23 e85904c40792b0630047bd12da113ffa \n 4 11/07/22 12:03:01 zookeeper/[email protected] 26 ea516fa0b2c7da0d7293932e05d92abe547bcc149621a501fd406a5e98a9d8ad\n 4 11/07/22 12:03:01 zookeeper/[email protected] 25 2634c6631c2f7759b884c757c21c531d \n 4 11/07/22 12:03:01 zookeeper/[email protected] 8 a4d567e6d3b3d65e \n 4 11/07/22 12:03:01 zookeeper/[email protected] 3 08c77c9291570bdf \n 7 11/07/22 20:40:48 kafka/[email protected] 18 ab6b7057a69e2ae2a3a120ea9b5a7646a3584923aba997a6e8cb8e59424debc4\n 7 11/07/22 20:40:48 kafka/[email protected] 17 08341932d6e53f3a2fe22e9d092f2692 \n 7 11/07/22 20:40:48 kafka/[email protected] 16 bf40269d6b8c76fe02fbd0ae045dce522a0e4029ec0483a2 \n 7 11/07/22 20:40:48 kafka/[email protected] 23 9f0523065f87ceb1ff245234f274394b \n 7 11/07/22 20:40:48 kafka/[email protected] 26 70e46cefd514eceef9066b5818fc650f5cd2f35a73fc158e40b958e350d297a8\n 7 11/07/22 20:40:48 kafka/[email protected] 25 55a9ea3ded1a0a1540822820697b28e3 \n 7 11/07/22 20:40:48 kafka/[email protected] 8 f1348f8ffdf25257 \n 7 11/07/22 20:40:48 kafka/[email protected] 3 d5d93d79799231a7 \n 8 11/07/22 20:41:09 zookeeper/[email protected] 18 f42aa34010972483a792d456683bbda6d9b241b6fdb97f6da62ae9512572f332\n 8 11/07/22 20:41:09 zookeeper/[email protected] 17 01a815580801a0d0d8565a13e9d80091 \n 8 11/07/22 20:41:09 zookeeper/[email protected] 16 d5800e4abc2a57e0250425f854314f75ea9216fbec831cb9 \n 8 11/07/22 20:41:09 zookeeper/[email protected] 23 81183219ced8a01a9c0f2051eafb57a1 \n 8 11/07/22 20:41:09 zookeeper/[email protected] 26 dbdee2352fc471414f1d56ddd53fa232d9206df4bfc49c984c49cc83a6fc68b3\n 8 11/07/22 20:41:09 zookeeper/[email protected] 25 47fc5df943a11d2fe55efcef9da9cd87 \n 8 11/07/22 20:41:09 zookeeper/[email protected] 8 c1c4a486b346589b \n 8 11/07/22 20:41:09 zookeeper/[email protected] 3 d94c9e1537a29d8c \n 10 14/07/22 11:36:52 zookeeper/[email protected] 18 97479e2d7e30d982c702caa0edceafe02d9936d253627d78ec74edc71c575291\n 10 14/07/22 11:36:52 zookeeper/[email protected] 17 14f44604fbfdd484a84f478cdd8bd97f \n 10 14/07/22 11:36:52 zookeeper/[email protected] 16 404f9ec8fbc867ef643b4cb64c384a54e55b5825f864ad37 \n 10 14/07/22 11:36:52 zookeeper/[email protected] 23 dd0d09de05c0a2a347b05f18347fb969 \n 10 14/07/22 11:36:52 zookeeper/[email protected] 26 10a23bc7c946d90275925b247ea6d36fe6a3d5e44451538f24058ac4423c3b0e\n 10 14/07/22 11:36:52 zookeeper/[email protected] 25 3354e3a8db5493ac665e90d8c263b9b3 \n 10 14/07/22 11:36:52 zookeeper/[email protected] 8 a120da29b5589497 \n 10 14/07/22 11:36:52 zookeeper/[email protected] 3 dff84340f7c43445 \n 8 14/07/22 11:37:01 kafka/[email protected] 18 07a6eae411da42234788c3d078de3e8dc685fd1f8e18ff07bdf4a6dce7ef6a9a\n 8 14/07/22 11:37:01 kafka/[email protected] 17 e6b984e4ee106d0afed0b62a9e6e939e \n 8 14/07/22 11:37:01 kafka/[email protected] 16 d32526d60ea8cd7cb023e6bcfdf737517c1c0bc2ad3ec168 \n 8 14/07/22 11:37:01 kafka/[email protected] 23 a621da6d608aecc31b9b2ff369562de8 \n 8 14/07/22 11:37:01 kafka/[email protected] 26 0dccf1a2dc5ffcb657352a4975e0d36ddd0de121e03bb8b27cd4f7c69ae1ecfc\n 8 14/07/22 11:37:01 kafka/[email protected] 25 4c674c589fd908a6b2fcc7fafae2a7cf \n 8 14/07/22 11:37:01 kafka/[email protected] 8 b06464c77367b329 \n 8 14/07/22 11:37:01 kafka/[email protected] 3 31ba3b43b3297a0d \n 3 14/07/22 11:37:11 [email protected] 18 148e8570d34db908cd11adb7fb4bdcb9c50c3ce95728be8f866a7e357c5c8369\n 3 14/07/22 11:37:11 [email protected] 17 f10c98bc3e636ca496d9b16497377e7e \n 3 14/07/22 11:37:11 [email protected] 16 b3d9ecd0fb6737763437453886f1204664bc237f0e5e2604 \n 3 14/07/22 11:37:11 [email protected] 23 489223d95398e212603b33d14ee080ca \n 3 14/07/22 11:37:11 [email protected] 26 1440c3888abcdaf3749ec3ea42ac2547e57be7c97bdf954e560f0b130ef08fa9\n 3 14/07/22 11:37:11 [email protected] 25 a73f2824200324eef9e53a1283800480 \n 3 14/07/22 11:37:11 [email protected] 8 67a2ad9852731f62 \n 3 14/07/22 11:37:11 [email protected] 3 adc11fd5168f54d9 \n),\n password: (string) \"\",\n attributes: (map[string]interface {}) {\n },\n validUntil: (time.Time) 0001-01-01 00:00:00 +0000 UTC,\n authenticated: (bool) false,\n human: (bool) true,\n authTime: (time.Time) 0001-01-01 00:00:00 +0000 UTC,\n groupMembership: (map[string]bool) {\n },\n sessionID: (string) (len=36) \"e137314f-0970-82d2-fa58-07459c341cfb\"\n }),\n Config: (*config.Config)(0xc000252e00)({\n LibDefaults: (config.LibDefaults) {\n AllowWeakCrypto: (bool) false,\n Canonicalize: (bool) false,\n CCacheType: (int) 4,\n Clockskew: (time.Duration) 5m0s,\n DefaultClientKeytabName: (string) (len=43) \"/usr/local/var/krb5/user/1000/client.keytab\",\n DefaultKeytabName: (string) (len=16) \"/etc/krb5.keytab\",\n DefaultRealm: (string) (len=9) \"AIOPS.COM\",\n DefaultTGSEnctypes: ([]string) (len=9 cap=9) {\n (string) (len=23) \"aes256-cts-hmac-sha1-96\",\n (string) (len=23) \"aes128-cts-hmac-sha1-96\",\n (string) (len=13) \"des3-cbc-sha1\",\n (string) (len=16) \"arcfour-hmac-md5\",\n (string) (len=20) \"camellia256-cts-cmac\",\n (string) (len=20) \"camellia128-cts-cmac\",\n (string) (len=11) \"des-cbc-crc\",\n (string) (len=11) \"des-cbc-md5\",\n (string) (len=11) \"des-cbc-md4\"\n },\n DefaultTktEnctypes: ([]string) (len=9 cap=9) {\n (string) (len=23) \"aes256-cts-hmac-sha1-96\",\n (string) (len=23) \"aes128-cts-hmac-sha1-96\",\n (string) (len=13) \"des3-cbc-sha1\",\n (string) (len=16) \"arcfour-hmac-md5\",\n (string) (len=20) \"camellia256-cts-cmac\",\n (string) (len=20) \"camellia128-cts-cmac\",\n (string) (len=11) \"des-cbc-crc\",\n (string) (len=11) \"des-cbc-md5\",\n (string) (len=11) \"des-cbc-md4\"\n },\n DefaultTGSEnctypeIDs: ([]int32) (len=3 cap=4) {\n (int32) 18,\n (int32) 17,\n (int32) 23\n },\n DefaultTktEnctypeIDs: ([]int32) (len=3 cap=4) {\n (int32) 18,\n (int32) 17,\n (int32) 23\n },\n DNSCanonicalizeHostname: (bool) true,\n DNSLookupKDC: (bool) false,\n DNSLookupRealm: (bool) false,\n ExtraAddresses: ([]net.IP) <nil>,\n Forwardable: (bool) true,\n IgnoreAcceptorHostname: (bool) false,\n K5LoginAuthoritative: (bool) false,\n K5LoginDirectory: (string) (len=9) \"/home/eoi\",\n KDCDefaultOptions: (asn1.BitString) {\n Bytes: ([]uint8) (len=4 cap=8) {\n 00000000 00 00 00 10 |....|\n },\n BitLength: (int) 32\n },\n KDCTimeSync: (int) 1,\n NoAddresses: (bool) true,\n PermittedEnctypes: ([]string) (len=9 cap=9) {\n (string) (len=23) \"aes256-cts-hmac-sha1-96\",\n (string) (len=23) \"aes128-cts-hmac-sha1-96\",\n (string) (len=13) \"des3-cbc-sha1\",\n (string) (len=16) \"arcfour-hmac-md5\",\n (string) (len=20) \"camellia256-cts-cmac\",\n (string) (len=20) \"camellia128-cts-cmac\",\n (string) (len=11) \"des-cbc-crc\",\n (string) (len=11) \"des-cbc-md5\",\n (string) (len=11) \"des-cbc-md4\"\n },\n PermittedEnctypeIDs: ([]int32) (len=3 cap=4) {\n (int32) 18,\n (int32) 17,\n (int32) 23\n },\n PreferredPreauthTypes: ([]int) (len=4 cap=4) {\n (int) 17,\n (int) 16,\n (int) 15,\n (int) 14\n },\n Proxiable: (bool) false,\n RDNS: (bool) false,\n RealmTryDomains: (int) -1,\n RenewLifetime: (time.Duration) 0s,\n SafeChecksumType: (int) 8,\n TicketLifetime: (time.Duration) 24h0m0s,\n UDPPreferenceLimit: (int) 1465,\n VerifyAPReqNofail: (bool) false\n },\n Realms: ([]config.Realm) (len=1 cap=1) {\n (config.Realm) {\n Realm: (string) (len=9) \"AIOPS.COM\",\n AdminServer: ([]string) (len=1 cap=1) {\n (string) (len=15) \"eoi03.aiops.com\"\n },\n DefaultDomain: (string) \"\",\n KDC: ([]string) (len=1 cap=1) {\n (string) (len=18) \"eoi03.aiops.com:88\"\n },\n KPasswdServer: ([]string) (len=1 cap=1) {\n (string) (len=19) \"eoi03.aiops.com:464\"\n },\n MasterKDC: ([]string) <nil>\n }\n },\n DomainRealm: (config.DomainRealm) (len=2) {\n (string) (len=10) \".aiops.com\": (string) (len=9) \"AIOPS.COM\",\n (string) (len=9) \"aiops.com\": (string) (len=9) \"AIOPS.COM\"\n }\n }),\n settings: (*client.Settings)(0xc0005db110)({\n disablePAFXFast: (bool) false,\n assumePreAuthentication: (bool) false,\n preAuthEType: (int32) 0,\n logger: (*log.Logger)(<nil>)\n }),\n sessions: (*client.sessions)(0xc000414c80)({\n Entries: (map[string]*client.session) {\n },\n mux: (sync.RWMutex) {\n w: (sync.Mutex) {\n state: (int32) 0,\n sema: (uint32) 0\n },\n writerSem: (uint32) 0,\n readerSem: (uint32) 0,\n readerCount: (int32) 0,\n readerWait: (int32) 0\n }\n }),\n cache: (*client.Cache)(0xc000400020)({\n Entries: (map[string]client.CacheEntry) {\n },\n mux: (sync.RWMutex) {\n w: (sync.Mutex) {\n state: (int32) 0,\n sema: (uint32) 0\n },\n writerSem: (uint32) 0,\n readerSem: (uint32) 0,\n readerCount: (int32) 0,\n readerWait: (int32) 0\n }\n })\n})\n"}
{"level":"info","ts":"2022-07-29T22:06:28.621+0800","msg":"applied the first config"}
{"level":"info","ts":"2022-07-29T22:06:28.621+0800","msg":"immediate metadata update triggered","why":"client initialization"}
{"level":"debug","ts":"2022-07-29T22:06:28.621+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"seed 0"}
{"level":"debug","ts":"2022-07-29T22:06:28.622+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"seed 0"}
{"level":"debug","ts":"2022-07-29T22:06:28.622+0800","msg":"issuing api versions request","broker":"seed 0","version":3}
{"level":"debug","ts":"2022-07-29T22:06:28.623+0800","msg":"wrote ApiVersions v3","broker":"seed 0","bytes_written":30,"write_wait":0.000065869,"time_to_write":0.000038068,"err":null}
{"level":"debug","ts":"2022-07-29T22:06:28.623+0800","msg":"read ApiVersions v3","broker":"seed 0","bytes_read":14,"read_wait":0.000199542,"time_to_read":0.000015797,"err":null}
{"level":"debug","ts":"2022-07-29T22:06:28.623+0800","msg":"kafka does not know our ApiVersions version, downgrading to version 0 and retrying","broker":"seed 0"}
{"level":"debug","ts":"2022-07-29T22:06:28.623+0800","msg":"issuing api versions request","broker":"seed 0","version":0}
{"level":"debug","ts":"2022-07-29T22:06:28.623+0800","msg":"wrote ApiVersions v0","broker":"seed 0","bytes_written":17,"write_wait":0.000029521,"time_to_write":0.000038001,"err":null}
{"level":"debug","ts":"2022-07-29T22:06:28.623+0800","msg":"read ApiVersions v0","broker":"seed 0","bytes_read":278,"read_wait":0.000094273,"time_to_read":0.000065268,"err":null}
{"level":"debug","ts":"2022-07-29T22:06:28.623+0800","msg":"beginning sasl authentication","broker":"seed 0","mechanism":"GSSAPI","authenticate":false}
{"level":"debug","ts":"2022-07-29T22:06:28.634+0800","msg":"issuing raw sasl authenticate","broker":"seed 0","step":0}
{"level":"error","ts":"2022-07-29T22:06:28.635+0800","msg":"unable to initialize sasl","broker":"seed 0","err":"EOF","stacktrace":"github.com/twmb/franz-go/plugin/kzap.(*Logger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/franz-go/plugin/[email protected]/kzap.go:108\ngithub.com/twmb/franz-go/pkg/kgo.(*wrappedLogger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/logger.go:123\ngithub.com/twmb/franz-go/pkg/kgo.(*brokerCxn).init\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:643\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).loadConnection\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:481\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReq\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:269\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReqs\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:255"}
{"level":"debug","ts":"2022-07-29T22:06:28.636+0800","msg":"connection initialization failed","addr":"eoi03.aiops.com:9092","broker":"seed 0","err":"EOF"}
{"level":"debug","ts":"2022-07-29T22:06:28.636+0800","msg":"retrying request","tries":1,"backoff":0.263041561,"request_error":"EOF","response_error":"EOF"}
{"level":"debug","ts":"2022-07-29T22:06:28.900+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"seed 0"}
{"level":"debug","ts":"2022-07-29T22:06:28.900+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"seed 0"}
{"level":"debug","ts":"2022-07-29T22:06:28.900+0800","msg":"beginning sasl authentication","broker":"seed 0","mechanism":"GSSAPI","authenticate":false}
{"level":"debug","ts":"2022-07-29T22:06:28.901+0800","msg":"issuing raw sasl authenticate","broker":"seed 0","step":0}
{"level":"debug","ts":"2022-07-29T22:06:28.904+0800","msg":"issuing raw sasl authenticate","broker":"seed 0","step":1}
{"level":"debug","ts":"2022-07-29T22:06:28.904+0800","msg":"connection initialized successfully","addr":"eoi03.aiops.com:9092","broker":"seed 0"}
{"level":"debug","ts":"2022-07-29T22:06:28.904+0800","msg":"wrote Metadata v7","broker":"seed 0","bytes_written":44,"write_wait":0.004593806,"time_to_write":0.000025786,"err":null}
{"level":"debug","ts":"2022-07-29T22:06:28.905+0800","msg":"read Metadata v7","broker":"seed 0","bytes_read":138,"read_wait":0.000216524,"time_to_read":0.000508622,"err":null}
{"level":"info","ts":"2022-07-29T22:06:28.905+0800","msg":"beginning to manage the group lifecycle","group":"sensor_sync_tb_result__consumer_group2"}
{"level":"info","ts":"2022-07-29T22:06:28.905+0800","msg":"joining group","group":"sensor_sync_tb_result__consumer_group2"}
{"level":"debug","ts":"2022-07-29T22:06:28.906+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"0"}
{"level":"debug","ts":"2022-07-29T22:06:28.906+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"0"}
{"level":"debug","ts":"2022-07-29T22:06:28.906+0800","msg":"issuing api versions request","broker":"0","version":3}
{"level":"debug","ts":"2022-07-29T22:06:28.906+0800","msg":"wrote ApiVersions v3","broker":"0","bytes_written":30,"write_wait":0.000031441,"time_to_write":0.000047681,"err":null}
{"level":"debug","ts":"2022-07-29T22:06:28.906+0800","msg":"read ApiVersions v3","broker":"0","bytes_read":14,"read_wait":0.000094968,"time_to_read":0.000012885,"err":null}
{"level":"debug","ts":"2022-07-29T22:06:28.907+0800","msg":"kafka does not know our ApiVersions version, downgrading to version 0 and retrying","broker":"0"}
{"level":"debug","ts":"2022-07-29T22:06:28.907+0800","msg":"issuing api versions request","broker":"0","version":0}
{"level":"debug","ts":"2022-07-29T22:06:28.907+0800","msg":"wrote ApiVersions v0","broker":"0","bytes_written":17,"write_wait":0.00001792,"time_to_write":0.000034146,"err":null}
{"level":"debug","ts":"2022-07-29T22:06:28.907+0800","msg":"read ApiVersions v0","broker":"0","bytes_read":278,"read_wait":0.000086092,"time_to_read":0.000015669,"err":null}
{"level":"debug","ts":"2022-07-29T22:06:28.907+0800","msg":"beginning sasl authentication","broker":"0","mechanism":"GSSAPI","authenticate":false}
{"level":"debug","ts":"2022-07-29T22:06:28.908+0800","msg":"issuing raw sasl authenticate","broker":"0","step":0}
{"level":"error","ts":"2022-07-29T22:06:28.909+0800","msg":"unable to initialize sasl","broker":"0","err":"EOF","stacktrace":"github.com/twmb/franz-go/plugin/kzap.(*Logger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/franz-go/plugin/[email protected]/kzap.go:108\ngithub.com/twmb/franz-go/pkg/kgo.(*wrappedLogger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/logger.go:123\ngithub.com/twmb/franz-go/pkg/kgo.(*brokerCxn).init\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:643\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).loadConnection\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:481\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReq\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:269\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReqs\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:255"}
{"level":"debug","ts":"2022-07-29T22:06:28.910+0800","msg":"connection initialization failed","addr":"eoi03.aiops.com:9092","broker":"0","err":"EOF"}
{"level":"debug","ts":"2022-07-29T22:06:28.910+0800","msg":"retrying request","tries":1,"backoff":0.242803605,"request_error":"EOF","response_error":"EOF"}
{"level":"debug","ts":"2022-07-29T22:06:29.154+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"0"}
{"level":"debug","ts":"2022-07-29T22:06:29.154+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"0"}
{"level":"debug","ts":"2022-07-29T22:06:29.154+0800","msg":"beginning sasl authentication","broker":"0","mechanism":"GSSAPI","authenticate":false}
{"level":"error","ts":"2022-07-29T22:06:29.154+0800","msg":"unable to initialize sasl","broker":"0","err":"client does not have a username","stacktrace":"github.com/twmb/franz-go/plugin/kzap.(*Logger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/franz-go/plugin/[email protected]/kzap.go:108\ngithub.com/twmb/franz-go/pkg/kgo.(*wrappedLogger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/logger.go:123\ngithub.com/twmb/franz-go/pkg/kgo.(*brokerCxn).init\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:643\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).loadConnection\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:481\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReq\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:269\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReqs\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:255"}
{"level":"debug","ts":"2022-07-29T22:06:29.154+0800","msg":"connection initialization failed","addr":"eoi03.aiops.com:9092","broker":"0","err":"client does not have a username"}
{"level":"debug","ts":"2022-07-29T22:06:29.155+0800","msg":"entering OnPartitionsLost","with":null}
{"level":"info","ts":"2022-07-29T22:06:29.155+0800","msg":"assigning partitions","why":"clearing assignment at end of group management session","how":"unassigning everything","input":""}
{"level":"error","ts":"2022-07-29T22:06:29.155+0800","msg":"join and sync loop errored","group":"sensor_sync_tb_result__consumer_group2","err":"client does not have a username","consecutive_errors":1,"backoff":0.212754357,"stacktrace":"github.com/twmb/franz-go/plugin/kzap.(*Logger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/franz-go/plugin/[email protected]/kzap.go:108\ngithub.com/twmb/franz-go/pkg/kgo.(*wrappedLogger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/logger.go:123\ngithub.com/twmb/franz-go/pkg/kgo.(*groupConsumer).manage\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/consumer_group.go:375"}
{"level":"info","ts":"2022-07-29T22:06:29.368+0800","msg":"joining group","group":"sensor_sync_tb_result__consumer_group2"}
{"level":"debug","ts":"2022-07-29T22:06:29.369+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"0"}
{"level":"debug","ts":"2022-07-29T22:06:29.370+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"0"}
{"level":"debug","ts":"2022-07-29T22:06:29.370+0800","msg":"beginning sasl authentication","broker":"0","mechanism":"GSSAPI","authenticate":false}
{"level":"error","ts":"2022-07-29T22:06:29.370+0800","msg":"unable to initialize sasl","broker":"0","err":"client does not have a username","stacktrace":"github.com/twmb/franz-go/plugin/kzap.(*Logger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/franz-go/plugin/[email protected]/kzap.go:108\ngithub.com/twmb/franz-go/pkg/kgo.(*wrappedLogger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/logger.go:123\ngithub.com/twmb/franz-go/pkg/kgo.(*brokerCxn).init\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:643\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).loadConnection\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:481\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReq\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:269\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReqs\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:255"}
{"level":"debug","ts":"2022-07-29T22:06:29.370+0800","msg":"connection initialization failed","addr":"eoi03.aiops.com:9092","broker":"0","err":"client does not have a username"}
{"level":"debug","ts":"2022-07-29T22:06:29.371+0800","msg":"entering OnPartitionsLost","with":null}
{"level":"info","ts":"2022-07-29T22:06:29.371+0800","msg":"assigning partitions","why":"clearing assignment at end of group management session","how":"unassigning everything","input":""}
{"level":"error","ts":"2022-07-29T22:06:29.371+0800","msg":"join and sync loop errored","group":"sensor_sync_tb_result__consumer_group2","err":"client does not have a username","consecutive_errors":2,"backoff":0.45384072,"stacktrace":"github.com/twmb/franz-go/plugin/kzap.(*Logger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/franz-go/plugin/[email protected]/kzap.go:108\ngithub.com/twmb/franz-go/pkg/kgo.(*wrappedLogger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/logger.go:123\ngithub.com/twmb/franz-go/pkg/kgo.(*groupConsumer).manage\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/consumer_group.go:375"}
{"level":"info","ts":"2022-07-29T22:06:29.825+0800","msg":"joining group","group":"sensor_sync_tb_result__consumer_group2"}
{"level":"debug","ts":"2022-07-29T22:06:29.825+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"0"}
{"level":"debug","ts":"2022-07-29T22:06:29.826+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"0"}
{"level":"debug","ts":"2022-07-29T22:06:29.826+0800","msg":"beginning sasl authentication","broker":"0","mechanism":"GSSAPI","authenticate":false}
{"level":"error","ts":"2022-07-29T22:06:29.826+0800","msg":"unable to initialize sasl","broker":"0","err":"client does not have a username","stacktrace":"github.com/twmb/franz-go/plugin/kzap.(*Logger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/franz-go/plugin/[email protected]/kzap.go:108\ngithub.com/twmb/franz-go/pkg/kgo.(*wrappedLogger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/logger.go:123\ngithub.com/twmb/franz-go/pkg/kgo.(*brokerCxn).init\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:643\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).loadConnection\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:481\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReq\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:269\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReqs\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:255"}
{"level":"debug","ts":"2022-07-29T22:06:29.826+0800","msg":"connection initialization failed","addr":"eoi03.aiops.com:9092","broker":"0","err":"client does not have a username"}
{"level":"debug","ts":"2022-07-29T22:06:29.827+0800","msg":"entering OnPartitionsLost","with":null}
{"level":"info","ts":"2022-07-29T22:06:29.827+0800","msg":"assigning partitions","why":"clearing assignment at end of group management session","how":"unassigning everything","input":""}
{"level":"error","ts":"2022-07-29T22:06:29.827+0800","msg":"join and sync loop errored","group":"sensor_sync_tb_result__consumer_group2","err":"client does not have a username","consecutive_errors":3,"backoff":0.906468089,"stacktrace":"github.com/twmb/franz-go/plugin/kzap.(*Logger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/franz-go/plugin/[email protected]/kzap.go:108\ngithub.com/twmb/franz-go/pkg/kgo.(*wrappedLogger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/logger.go:123\ngithub.com/twmb/franz-go/pkg/kgo.(*groupConsumer).manage\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/consumer_group.go:375"}
{"level":"info","ts":"2022-07-29T22:06:30.733+0800","msg":"joining group","group":"sensor_sync_tb_result__consumer_group2"}
{"level":"debug","ts":"2022-07-29T22:06:30.734+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"0"}
{"level":"debug","ts":"2022-07-29T22:06:30.735+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"0"}
{"level":"debug","ts":"2022-07-29T22:06:30.735+0800","msg":"beginning sasl authentication","broker":"0","mechanism":"GSSAPI","authenticate":false}
{"level":"error","ts":"2022-07-29T22:06:30.735+0800","msg":"unable to initialize sasl","broker":"0","err":"client does not have a username","stacktrace":"github.com/twmb/franz-go/plugin/kzap.(*Logger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/franz-go/plugin/[email protected]/kzap.go:108\ngithub.com/twmb/franz-go/pkg/kgo.(*wrappedLogger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/logger.go:123\ngithub.com/twmb/franz-go/pkg/kgo.(*brokerCxn).init\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:643\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).loadConnection\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:481\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReq\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:269\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReqs\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:255"}
{"level":"debug","ts":"2022-07-29T22:06:30.735+0800","msg":"connection initialization failed","addr":"eoi03.aiops.com:9092","broker":"0","err":"client does not have a username"}
{"level":"debug","ts":"2022-07-29T22:06:30.735+0800","msg":"entering OnPartitionsLost","with":null}
{"level":"info","ts":"2022-07-29T22:06:30.735+0800","msg":"assigning partitions","why":"clearing assignment at end of group management session","how":"unassigning everything","input":""}
{"level":"error","ts":"2022-07-29T22:06:30.735+0800","msg":"join and sync loop errored","group":"sensor_sync_tb_result__consumer_group2","err":"client does not have a username","consecutive_errors":4,"backoff":2.041606523,"stacktrace":"github.com/twmb/franz-go/plugin/kzap.(*Logger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/franz-go/plugin/[email protected]/kzap.go:108\ngithub.com/twmb/franz-go/pkg/kgo.(*wrappedLogger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/logger.go:123\ngithub.com/twmb/franz-go/pkg/kgo.(*groupConsumer).manage\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/consumer_group.go:375"}
^C{"level":"info","ts":"2022-07-29T22:06:30.736+0800","msg":"clickhouse_sinker got the exit signal, start to clean"}
{"level":"debug","ts":"2022-07-29T22:06:30.736+0800","msg":"stopping task service...","task":"sensor-online-detect"}
{"level":"debug","ts":"2022-07-29T22:06:30.736+0800","msg":"stopped internal timers","task":"sensor-online-detect"}
{"level":"info","ts":"2022-07-29T22:06:30.737+0800","msg":"assigning partitions","why":"invalidating all assignments in LeaveGroup","how":"unassigning everything","input":""}
{"level":"info","ts":"2022-07-29T22:06:30.737+0800","msg":"leaving group","group":"sensor_sync_tb_result__consumer_group2","member_id":""}
{"level":"debug","ts":"2022-07-29T22:06:30.737+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"0"}
{"level":"debug","ts":"2022-07-29T22:06:30.738+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"0"}
{"level":"debug","ts":"2022-07-29T22:06:30.738+0800","msg":"beginning sasl authentication","broker":"0","mechanism":"GSSAPI","authenticate":false}
{"level":"error","ts":"2022-07-29T22:06:30.738+0800","msg":"unable to initialize sasl","broker":"0","err":"client does not have a username","stacktrace":"github.com/twmb/franz-go/plugin/kzap.(*Logger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/franz-go/plugin/[email protected]/kzap.go:108\ngithub.com/twmb/franz-go/pkg/kgo.(*wrappedLogger).Log\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/logger.go:123\ngithub.com/twmb/franz-go/pkg/kgo.(*brokerCxn).init\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:643\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).loadConnection\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:481\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReq\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:269\ngithub.com/twmb/franz-go/pkg/kgo.(*broker).handleReqs\n\t/home/eoi/go/pkg/mod/github.com/twmb/[email protected]/pkg/kgo/broker.go:255"}
I think it may be good if I automatically set that to true in AsMechanism, basically what I think is happening is that after the first authentication, the client is Destroyed and then future authentications do not have usernames because Destroy clears the credentials internally.
EOF would be the server closing the connection unexpectedly connection for some reason. I think it is eventually retried, but I’m not sure why the server is closing the cxn—something to potentially look into on your side?
On Mon, Aug 8, 2022 at 08:06 Zhichang Yu @.***> wrote:
Setting the PersistAfterAuth option to true, "unable to initialize sasl","broker":"0","err":"client does not have a username" disappeares, however there're two "unable to initialize sasl","broker":"seed 0","err":"EOF" errors. Is this normal?
@.*** zhichyu]$ ./clickhouse_sinker --local-cfg-file sinker_franz.json --log-paths=stdout,sinker_franz.log {"level":"info","ts":"2022-08-08T21:53:35.909+0800","msg":"version v2.5.2-5-gba0ab37, commit ba0ab3769c4fec4794bc8707074cacde30d4a10f, date 2022-08-08T21:52:37+0800, builtBy @., pid 45350"} {"level":"info","ts":"2022-08-08T21:53:35.910+0800","msg":"clickhouse_sinker initialization"} {"level":"info","ts":"2022-08-08T21:53:35.910+0800","msg":"Run http server at http://192.168.101.27:34177/"} {"level":"info","ts":"2022-08-08T21:53:35.910+0800","msg":"get config from local file sinker_franz.json"} {"level":"info","ts":"2022-08-08T21:53:35.910+0800","msg":"clickhouse_sinker initialization completed"} {"level":"info","ts":"2022-08-08T21:53:35.911+0800","msg":"going to apply the first config","config":{"Kafka":{"Brokers":"eoi03.aiops.com:9092","Version":"2.0.0","Security":{"sasl.jaas.config":"com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true debug=true keyTab="/home/eoi/app/kafka_2.13-2.8.1/config/kerberos/kafka02.keytab" @.";","sasl.kerberos.service.name":"kafka","sasl.mechanism":"GSSAPI","security.protocol":"SASL_PLAINTEXT","ssl.enabled.protocols":"","ssl.endpoint.identification.algorithm":"","ssl.key.password":"","ssl.keystore.location":"","ssl.keystore.password":"","ssl.keystore.type":"","ssl.truststore.location":"","ssl.truststore.password":"","ssl.truststore.type":""},"TLS":{"Enable":false,"CaCertFiles":"","ClientCertFile":"","ClientKeyFile":"","TrustStoreLocation":"","TrustStorePassword":"","KeystoreLocation":"","KeystorePassword":"","EndpIdentAlgo":""},"Sasl":{"Enable":true,"Mechanism":"GSSAPI","Username":"","Password":"","GSSAPI":{"AuthType":2,"KeyTabPath":"/home/eoi/app/kafka_2.13-2.8.1/config/kerberos/kafka02.keytab","KerberosConfigPath":"/etc/krb5.conf","ServiceName":"kafka","Username":"eoi","Password":"","Realm":"AIOPS.COM","DisablePAFXFAST":false}}},"Clickhouse":{"Cluster":"physic_cluster","DB":"sensor","Hosts":[["192.168.103.26"]],"Port":9000,"Username":"eoi","Password":"123456","DsnParams":"","Secure":false,"InsecureSkipVerify":false,"RetryTimes":0,"MaxOpenConns":7},"Task":null,"Tasks":[{"Name":"sensor-online-detect","KafkaClient":"franz","Topic":"sensor-online-detect","ConsumerGroup":"sensor_sync_tb_result__consumer_group2","Earliest":false,"Parser":"fastjson","CsvFormat":null,"Delimiter":"","TableName":"tb_source","AutoSchema":true,"ExcludeColumns":null,"dims":null,"DynamicSchema":{"Enable":false,"MaxDims":0,"WhiteList":"","BlackList":""},"PrometheusSchema":false,"PromLabelsBlackList":"","LoadSeriesAtStartup":false,"flushInterval":3,"bufferSize":32768,"timeZone":"Local","timeUnit":1}],"Assignment":{"Version":0,"UpdatedAt":0,"UpdatedBy":"","Map":null},"LogLevel":"debug"}} {"level":"info","ts":"2022-08-08T21:53:35.911+0800","msg":"clickhouse.Open succeeded","dbVer":1,"replica":"192.168.103.26:9000"} {"level":"info","ts":"2022-08-08T21:53:35.911+0800","msg":"initialized parsing pool","maxWorkers":8,"queueSize":65536} {"level":"info","ts":"2022-08-08T21:53:35.911+0800","msg":"initialized writing pool","maxWorkers":7,"queueSize":3} {"level":"info","ts":"2022-08-08T21:53:35.911+0800","msg":"task initializing","task":"sensor-online-detect"} {"level":"info","ts":"2022-08-08T21:53:35.919+0800","msg":"Prepare sql=> INSERT INTO
sensor.tb_source(timestamp,sync_id,value,item_guid,metric_name,instance_name,tier1_name,tier2_name,tier3_name,tier4_name,tier5_name)","task":"sensor-online-detect"} {"level":"info","ts":"2022-08-08T21:53:35.919+0800","msg":"executing sql=> SELECT name FROM system.tables WHERE engine='Distributed' AND database='sensor' AND match(create_table_query, 'Distributed\(\'physic_cluster\', \'sensor\', \'tb_source\'.*\)')","task":"sensor-online-detect"} (*client.Client)(0xc0003497a0)({ Credentials: (*credentials.Credentials)(0xc0001b00c0)({ username: (string) (len=3) "eoi", displayName: (string) (len=3) "eoi", realm: (string) (len=9) "AIOPS.COM", cname: (types.PrincipalName) { NameType: (int32) 1, NameString: ([]string) (len=1 cap=1) { (string) (len=3) "eoi" } }, keytab: (*keytab.Keytab)(0xc000098a20)(KVNO Timestamp Principal ET Key
5 11/07/22 12:02:52 @.*** 18 6cf1dc96bd9f2e1d1ec818ca2599941948dee6a6fb492024355551d71438411d 5 11/07/22 12:02:52 @.*** 17 414e57dff2bf45ed2e91d39f9de99e7f 5 11/07/22 12:02:52 @.*** 16 fe296ea1f14f1ae0465bdffd54384f85910ed9da10a2a483 5 11/07/22 12:02:52 @.*** 23 f4a6f0e1d0fbc532880006a922c12e48 5 11/07/22 12:02:52 @.*** 26 922d8b83d9ed1d04a22fa08139ddb7155a352701a7d62c576443a5e9ab1b14e2 5 11/07/22 12:02:52 @.*** 25 b685c6bccd9b67294426438989889178 5 11/07/22 12:02:52 @.*** 8 d07c04cda2b07a7c 5 11/07/22 12:02:52 @.*** 3 732a64132586a8f8 4 11/07/22 12:03:01 @.*** 18 9e1290de287262abefc64293843bc62c2aef23d9d63ccfe64d418d86cdb6d37a 4 11/07/22 12:03:01 @.*** 17 58dd76b1cc3cb70647110351ef0d4fdf 4 11/07/22 12:03:01 @.*** 16 d9702a86b3ab6bae3dfb495810545757a8689832250bece9 4 11/07/22 12:03:01 @.*** 23 e85904c40792b0630047bd12da113ffa 4 11/07/22 12:03:01 @.*** 26 ea516fa0b2c7da0d7293932e05d92abe547bcc149621a501fd406a5e98a9d8ad 4 11/07/22 12:03:01 @.*** 25 2634c6631c2f7759b884c757c21c531d 4 11/07/22 12:03:01 @.*** 8 a4d567e6d3b3d65e 4 11/07/22 12:03:01 @.*** 3 08c77c9291570bdf 7 11/07/22 20:40:48 @.*** 18 ab6b7057a69e2ae2a3a120ea9b5a7646a3584923aba997a6e8cb8e59424debc4 7 11/07/22 20:40:48 @.*** 17 08341932d6e53f3a2fe22e9d092f2692 7 11/07/22 20:40:48 @.*** 16 bf40269d6b8c76fe02fbd0ae045dce522a0e4029ec0483a2 7 11/07/22 20:40:48 @.*** 23 9f0523065f87ceb1ff245234f274394b 7 11/07/22 20:40:48 @.*** 26 70e46cefd514eceef9066b5818fc650f5cd2f35a73fc158e40b958e350d297a8 7 11/07/22 20:40:48 @.*** 25 55a9ea3ded1a0a1540822820697b28e3 7 11/07/22 20:40:48 @.*** 8 f1348f8ffdf25257 7 11/07/22 20:40:48 @.*** 3 d5d93d79799231a7 8 11/07/22 20:41:09 @.*** 18 f42aa34010972483a792d456683bbda6d9b241b6fdb97f6da62ae9512572f332 8 11/07/22 20:41:09 @.*** 17 01a815580801a0d0d8565a13e9d80091 8 11/07/22 20:41:09 @.*** 16 d5800e4abc2a57e0250425f854314f75ea9216fbec831cb9 8 11/07/22 20:41:09 @.*** 23 81183219ced8a01a9c0f2051eafb57a1 8 11/07/22 20:41:09 @.*** 26 dbdee2352fc471414f1d56ddd53fa232d9206df4bfc49c984c49cc83a6fc68b3 8 11/07/22 20:41:09 @.*** 25 47fc5df943a11d2fe55efcef9da9cd87 8 11/07/22 20:41:09 @.*** 8 c1c4a486b346589b 8 11/07/22 20:41:09 @.*** 3 d94c9e1537a29d8c 10 14/07/22 11:36:52 @.*** 18 97479e2d7e30d982c702caa0edceafe02d9936d253627d78ec74edc71c575291 10 14/07/22 11:36:52 @.*** 17 14f44604fbfdd484a84f478cdd8bd97f 10 14/07/22 11:36:52 @.*** 16 404f9ec8fbc867ef643b4cb64c384a54e55b5825f864ad37 10 14/07/22 11:36:52 @.*** 23 dd0d09de05c0a2a347b05f18347fb969 10 14/07/22 11:36:52 @.*** 26 10a23bc7c946d90275925b247ea6d36fe6a3d5e44451538f24058ac4423c3b0e 10 14/07/22 11:36:52 @.*** 25 3354e3a8db5493ac665e90d8c263b9b3 10 14/07/22 11:36:52 @.*** 8 a120da29b5589497 10 14/07/22 11:36:52 @.*** 3 dff84340f7c43445 8 14/07/22 11:37:01 @.*** 18 07a6eae411da42234788c3d078de3e8dc685fd1f8e18ff07bdf4a6dce7ef6a9a 8 14/07/22 11:37:01 @.*** 17 e6b984e4ee106d0afed0b62a9e6e939e 8 14/07/22 11:37:01 @.*** 16 d32526d60ea8cd7cb023e6bcfdf737517c1c0bc2ad3ec168 8 14/07/22 11:37:01 @.*** 23 a621da6d608aecc31b9b2ff369562de8 8 14/07/22 11:37:01 @.*** 26 0dccf1a2dc5ffcb657352a4975e0d36ddd0de121e03bb8b27cd4f7c69ae1ecfc 8 14/07/22 11:37:01 @.*** 25 4c674c589fd908a6b2fcc7fafae2a7cf 8 14/07/22 11:37:01 @.*** 8 b06464c77367b329 8 14/07/22 11:37:01 @.*** 3 31ba3b43b3297a0d 3 14/07/22 11:37:11 @.*** 18 148e8570d34db908cd11adb7fb4bdcb9c50c3ce95728be8f866a7e357c5c8369 3 14/07/22 11:37:11 @.*** 17 f10c98bc3e636ca496d9b16497377e7e 3 14/07/22 11:37:11 @.*** 16 b3d9ecd0fb6737763437453886f1204664bc237f0e5e2604 3 14/07/22 11:37:11 @.*** 23 489223d95398e212603b33d14ee080ca 3 14/07/22 11:37:11 @.*** 26 1440c3888abcdaf3749ec3ea42ac2547e57be7c97bdf954e560f0b130ef08fa9 3 14/07/22 11:37:11 @.*** 25 a73f2824200324eef9e53a1283800480 3 14/07/22 11:37:11 @.*** 8 67a2ad9852731f62 3 14/07/22 11:37:11 @.*** 3 adc11fd5168f54d9 ), password: (string) "", attributes: (map[string]interface {}) { }, validUntil: (time.Time) 0001-01-01 00:00:00 +0000 UTC, authenticated: (bool) false, human: (bool) true, authTime: (time.Time) 0001-01-01 00:00:00 +0000 UTC, groupMembership: (map[string]bool) { }, sessionID: (string) (len=36) "8a4c7909-6720-ae8d-a79e-794d9dbdd9e8" }), Config: (config.Config)(0xc000292e00)({ LibDefaults: (config.LibDefaults) { AllowWeakCrypto: (bool) false, Canonicalize: (bool) false, CCacheType: (int) 4, Clockskew: (time.Duration) 5m0s, DefaultClientKeytabName: (string) (len=43) "/usr/local/var/krb5/user/1000/client.keytab", DefaultKeytabName: (string) (len=16) "/etc/krb5.keytab", DefaultRealm: (string) (len=9) "AIOPS.COM", DefaultTGSEnctypes: ([]string) (len=9 cap=9) { (string) (len=23) "aes256-cts-hmac-sha1-96", (string) (len=23) "aes128-cts-hmac-sha1-96", (string) (len=13) "des3-cbc-sha1", (string) (len=16) "arcfour-hmac-md5", (string) (len=20) "camellia256-cts-cmac", (string) (len=20) "camellia128-cts-cmac", (string) (len=11) "des-cbc-crc", (string) (len=11) "des-cbc-md5", (string) (len=11) "des-cbc-md4" }, DefaultTktEnctypes: ([]string) (len=9 cap=9) { (string) (len=23) "aes256-cts-hmac-sha1-96", (string) (len=23) "aes128-cts-hmac-sha1-96", (string) (len=13) "des3-cbc-sha1", (string) (len=16) "arcfour-hmac-md5", (string) (len=20) "camellia256-cts-cmac", (string) (len=20) "camellia128-cts-cmac", (string) (len=11) "des-cbc-crc", (string) (len=11) "des-cbc-md5", (string) (len=11) "des-cbc-md4" }, DefaultTGSEnctypeIDs: ([]int32) (len=3 cap=4) { (int32) 18, (int32) 17, (int32) 23 }, DefaultTktEnctypeIDs: ([]int32) (len=3 cap=4) { (int32) 18, (int32) 17, (int32) 23 }, DNSCanonicalizeHostname: (bool) true, DNSLookupKDC: (bool) false, DNSLookupRealm: (bool) false, ExtraAddresses: ([]net.IP)
, Forwardable: (bool) true, IgnoreAcceptorHostname: (bool) false, K5LoginAuthoritative: (bool) false, K5LoginDirectory: (string) (len=9) "/home/eoi", KDCDefaultOptions: (asn1.BitString) { Bytes: ([]uint8) (len=4 cap=8) { 00000000 00 00 00 10 |....| }, BitLength: (int) 32 }, KDCTimeSync: (int) 1, NoAddresses: (bool) true, PermittedEnctypes: ([]string) (len=9 cap=9) { (string) (len=23) "aes256-cts-hmac-sha1-96", (string) (len=23) "aes128-cts-hmac-sha1-96", (string) (len=13) "des3-cbc-sha1", (string) (len=16) "arcfour-hmac-md5", (string) (len=20) "camellia256-cts-cmac", (string) (len=20) "camellia128-cts-cmac", (string) (len=11) "des-cbc-crc", (string) (len=11) "des-cbc-md5", (string) (len=11) "des-cbc-md4" }, PermittedEnctypeIDs: ([]int32) (len=3 cap=4) { (int32) 18, (int32) 17, (int32) 23 }, PreferredPreauthTypes: ([]int) (len=4 cap=4) { (int) 17, (int) 16, (int) 15, (int) 14 }, Proxiable: (bool) false, RDNS: (bool) false, RealmTryDomains: (int) -1, RenewLifetime: (time.Duration) 0s, SafeChecksumType: (int) 8, TicketLifetime: (time.Duration) 24h0m0s, UDPPreferenceLimit: (int) 1465, VerifyAPReqNofail: (bool) false }, Realms: ([]config.Realm) (len=1 cap=1) { (config.Realm) { Realm: (string) (len=9) "AIOPS.COM", AdminServer: ([]string) (len=1 cap=1) { (string) (len=15) "eoi03.aiops.com" }, DefaultDomain: (string) "", KDC: ([]string) (len=1 cap=1) { (string) (len=18) "eoi03.aiops.com:88" }, KPasswdServer: ([]string) (len=1 cap=1) { (string) (len=19) "eoi03.aiops.com:464" }, MasterKDC: ([]string) 16 fe296ea1f14f1ae0465bdffd54384f85910ed9da10a2a483 \n 5 11/07/22 12:02:52 @.*** 23 f4a6f0e1d0fbc532880006a922c12e48 \n 5 11/07/22 12:02:52 @.*** 26 922d8b83d9ed1d04a22fa08139ddb7155a352701a7d62c576443a5e9ab1b14e2\n 5 11/07/22 12:02:52 @.*** 25 b685c6bccd9b67294426438989889178 \n 5 11/07/22 12:02:52 @.*** 8 d07c04cda2b07a7c \n 5 11/07/22 12:02:52 @.*** 3 732a64132586a8f8 \n 4 11/07/22 12:03:01 @.*** 18 9e1290de287262abefc64293843bc62c2aef23d9d63ccfe64d418d86cdb6d37a\n 4 11/07/22 12:03:01 @.*** 17 58dd76b1cc3cb70647110351ef0d4fdf \n 4 11/07/22 12:03:01 @.*** 16 d9702a86b3ab6bae3dfb495810545757a8689832250bece9 \n 4 11/07/22 12:03:01 @.*** 23 e85904c40792b0630047bd12da113ffa \n 4 11/07/22 12:03:01 @.*** 26 ea516fa0b2c7da0d7293932e05d92abe547bcc149621a501fd406a5e98a9d8ad\n 4 11/07/22 12:03:01 @.*** 25 2634c6631c2f7759b884c757c21c531d \n 4 11/07/22 12:03:01 @.*** 8 a4d567e6d3b3d65e \n 4 11/07/22 12:03:01 @.*** 3 08c77c9291570bdf \n 7 11/07/22 20:40:48 @.*** 18 ab6b7057a69e2ae2a3a120ea9b5a7646a3584923aba997a6e8cb8e59424debc4\n 7 11/07/22 20:40:48 @.*** 17 08341932d6e53f3a2fe22e9d092f2692 \n 7 11/07/22 20:40:48 @.*** 16 bf40269d6b8c76fe02fbd0ae045dce522a0e4029ec0483a2 \n 7 11/07/22 20:40:48 @.*** 23 9f0523065f87ceb1ff245234f274394b \n 7 11/07/22 20:40:48 @.*** 26 70e46cefd514eceef9066b5818fc650f5cd2f35a73fc158e40b958e350d297a8\n 7 11/07/22 20:40:48 @.*** 25 55a9ea3ded1a0a1540822820697b28e3 \n 7 11/07/22 20:40:48 @.*** 8 f1348f8ffdf25257 \n 7 11/07/22 20:40:48 @.*** 3 d5d93d79799231a7 \n 8 11/07/22 20:41:09 @.*** 18 f42aa34010972483a792d456683bbda6d9b241b6fdb97f6da62ae9512572f332\n 8 11/07/22 20:41:09 @.*** 17 01a815580801a0d0d8565a13e9d80091 \n 8 11/07/22 20:41:09 @.*** 16 d5800e4abc2a57e0250425f854314f75ea9216fbec831cb9 \n 8 11/07/22 20:41:09 @.*** 23 81183219ced8a01a9c0f2051eafb57a1 \n 8 11/07/22 20:41:09 @.*** 26 dbdee2352fc471414f1d56ddd53fa232d9206df4bfc49c984c49cc83a6fc68b3\n 8 11/07/22 20:41:09 @.*** 25 47fc5df943a11d2fe55efcef9da9cd87 \n 8 11/07/22 20:41:09 @.*** 8 c1c4a486b346589b \n 8 11/07/22 20:41:09 @.*** 3 d94c9e1537a29d8c \n 10 14/07/22 11:36:52 @.*** 18 97479e2d7e30d982c702caa0edceafe02d9936d253627d78ec74edc71c575291\n 10 14/07/22 11:36:52 @.*** 17 14f44604fbfdd484a84f478cdd8bd97f \n 10 14/07/22 11:36:52 @.*** 16 404f9ec8fbc867ef643b4cb64c384a54e55b5825f864ad37 \n 10 14/07/22 11:36:52 @.*** 23 dd0d09de05c0a2a347b05f18347fb969 \n 10 14/07/22 11:36:52 @.*** 26 10a23bc7c946d90275925b247ea6d36fe6a3d5e44451538f24058ac4423c3b0e\n 10 14/07/22 11:36:52 @.*** 25 3354e3a8db5493ac665e90d8c263b9b3 \n 10 14/07/22 11:36:52 @.*** 8 a120da29b5589497 \n 10 14/07/22 11:36:52 @.*** 3 dff84340f7c43445 \n 8 14/07/22 11:37:01 @.*** 18 07a6eae411da42234788c3d078de3e8dc685fd1f8e18ff07bdf4a6dce7ef6a9a\n 8 14/07/22 11:37:01 @.*** 17 e6b984e4ee106d0afed0b62a9e6e939e \n 8 14/07/22 11:37:01 @.*** 16 d32526d60ea8cd7cb023e6bcfdf737517c1c0bc2ad3ec168 \n 8 14/07/22 11:37:01 @.*** 23 a621da6d608aecc31b9b2ff369562de8 \n 8 14/07/22 11:37:01 @.*** 26 0dccf1a2dc5ffcb657352a4975e0d36ddd0de121e03bb8b27cd4f7c69ae1ecfc\n 8 14/07/22 11:37:01 @.*** 25 4c674c589fd908a6b2fcc7fafae2a7cf \n 8 14/07/22 11:37:01 @.*** 8 b06464c77367b329 \n 8 14/07/22 11:37:01 @.*** 3 31ba3b43b3297a0d \n 3 14/07/22 11:37:11 @.*** 18 148e8570d34db908cd11adb7fb4bdcb9c50c3ce95728be8f866a7e357c5c8369\n 3 14/07/22 11:37:11 @.*** 17 f10c98bc3e636ca496d9b16497377e7e \n 3 14/07/22 11:37:11 @.*** 16 b3d9ecd0fb6737763437453886f1204664bc237f0e5e2604 \n 3 14/07/22 11:37:11 @.*** 23 489223d95398e212603b33d14ee080ca \n 3 14/07/22 11:37:11 @.*** 26 1440c3888abcdaf3749ec3ea42ac2547e57be7c97bdf954e560f0b130ef08fa9\n 3 14/07/22 11:37:11 @.*** 25 a73f2824200324eef9e53a1283800480 \n 3 14/07/22 11:37:11 @.*** 8 67a2ad9852731f62 \n 3 14/07/22 11:37:11 @.*** 3 adc11fd5168f54d9 \n),\n password: (string) "",\n attributes: (map[string]interface {}) {\n },\n validUntil: (time.Time) 0001-01-01 00:00:00 +0000 UTC,\n authenticated: (bool) false,\n human: (bool) true,\n authTime: (time.Time) 0001-01-01 00:00:00 +0000 UTC,\n groupMembership: (map[string]bool) {\n },\n sessionID: (string) (len=36) "8a4c7909-6720-ae8d-a79e-794d9dbdd9e8"\n }),\n Config: (config.Config)(0xc000292e00)({\n LibDefaults: (config.LibDefaults) {\n AllowWeakCrypto: (bool) false,\n Canonicalize: (bool) false,\n CCacheType: (int) 4,\n Clockskew: (time.Duration) 5m0s,\n DefaultClientKeytabName: (string) (len=43) "/usr/local/var/krb5/user/1000/client.keytab",\n DefaultKeytabName: (string) (len=16) "/etc/krb5.keytab",\n DefaultRealm: (string) (len=9) "AIOPS.COM",\n DefaultTGSEnctypes: ([]string) (len=9 cap=9) {\n (string) (len=23) "aes256-cts-hmac-sha1-96",\n (string) (len=23) "aes128-cts-hmac-sha1-96",\n (string) (len=13) "des3-cbc-sha1",\n (string) (len=16) "arcfour-hmac-md5",\n (string) (len=20) "camellia256-cts-cmac",\n (string) (len=20) "camellia128-cts-cmac",\n (string) (len=11) "des-cbc-crc",\n (string) (len=11) "des-cbc-md5",\n (string) (len=11) "des-cbc-md4"\n },\n DefaultTktEnctypes: ([]string) (len=9 cap=9) {\n (string) (len=23) "aes256-cts-hmac-sha1-96",\n (string) (len=23) "aes128-cts-hmac-sha1-96",\n (string) (len=13) "des3-cbc-sha1",\n (string) (len=16) "arcfour-hmac-md5",\n (string) (len=20) "camellia256-cts-cmac",\n (string) (len=20) "camellia128-cts-cmac",\n (string) (len=11) "des-cbc-crc",\n (string) (len=11) "des-cbc-md5",\n (string) (len=11) "des-cbc-md4"\n },\n DefaultTGSEnctypeIDs: ([]int32) (len=3 cap=4) {\n (int32) 18,\n (int32) 17,\n (int32) 23\n },\n DefaultTktEnctypeIDs: ([]int32) (len=3 cap=4) {\n (int32) 18,\n (int32) 17,\n (int32) 23\n },\n DNSCanonicalizeHostname: (bool) true,\n DNSLookupKDC: (bool) false,\n DNSLookupRealm: (bool) false,\n ExtraAddresses: ([]net.IP)} }, DomainRealm: (config.DomainRealm) (len=2) { (string) (len=10) ".aiops.com": (string) (len=9) "AIOPS.COM", (string) (len=9) "aiops.com": (string) (len=9) "AIOPS.COM" } }), settings: (client.Settings)(0xc0004a9f10)({ disablePAFXFast: (bool) false, assumePreAuthentication: (bool) false, preAuthEType: (int32) 0, logger: (log.Logger)( ) }), sessions: (client.sessions)(0xc000099120)({ Entries: (map[string]client.session) { }, mux: (sync.RWMutex) { w: (sync.Mutex) { state: (int32) 0, sema: (uint32) 0 }, writerSem: (uint32) 0, readerSem: (uint32) 0, readerCount: (int32) 0, readerWait: (int32) 0 } }), cache: (client.Cache)(0xc000099100)({ Entries: (map[string]client.CacheEntry) { }, mux: (sync.RWMutex) { w: (sync.Mutex) { state: (int32) 0, sema: (uint32) 0 }, writerSem: (uint32) 0, readerSem: (uint32) 0, readerCount: (int32) 0, readerWait: (int32) 0 } }) }) {"level":"debug","ts":"2022-08-08T21:53:35.932+0800","msg":"franz","auth.Client":"(client.Client)(0xc0003497a0)({\n Credentials: (credentials.Credentials)(0xc0001b00c0)({\n username: (string) (len=3) "eoi",\n displayName: (string) (len=3) "eoi",\n realm: (string) (len=9) "AIOPS.COM",\n cname: (types.PrincipalName) {\n NameType: (int32) 1,\n NameString: ([]string) (len=1 cap=1) {\n (string) (len=3) "eoi"\n }\n },\n keytab: (keytab.Keytab)(0xc000098a20)(KVNO Timestamp Principal ET Key\n---- ----------------- -------------------------------------------------------- -- ----------------------------------------------------------------\n 5 11/07/22 12:02:52 @. 18 6cf1dc96bd9f2e1d1ec818ca2599941948dee6a6fb492024355551d71438411d\n 5 11/07/22 12:02:52 @. 17 414e57dff2bf45ed2e91d39f9de99e7f \n 5 11/07/22 12:02:52 @. ,\n Forwardable: (bool) true,\n IgnoreAcceptorHostname: (bool) false,\n K5LoginAuthoritative: (bool) false,\n K5LoginDirectory: (string) (len=9) "/home/eoi",\n KDCDefaultOptions: (asn1.BitString) {\n Bytes: ([]uint8) (len=4 cap=8) {\n 00000000 00 00 00 10 |....|\n },\n BitLength: (int) 32\n },\n KDCTimeSync: (int) 1,\n NoAddresses: (bool) true,\n PermittedEnctypes: ([]string) (len=9 cap=9) {\n (string) (len=23) "aes256-cts-hmac-sha1-96",\n (string) (len=23) "aes128-cts-hmac-sha1-96",\n (string) (len=13) "des3-cbc-sha1",\n (string) (len=16) "arcfour-hmac-md5",\n (string) (len=20) "camellia256-cts-cmac",\n (string) (len=20) "camellia128-cts-cmac",\n (string) (len=11) "des-cbc-crc",\n (string) (len=11) "des-cbc-md5",\n (string) (len=11) "des-cbc-md4"\n },\n PermittedEnctypeIDs: ([]int32) (len=3 cap=4) {\n (int32) 18,\n (int32) 17,\n (int32) 23\n },\n PreferredPreauthTypes: ([]int) (len=4 cap=4) {\n (int) 17,\n (int) 16,\n (int) 15,\n (int) 14\n },\n Proxiable: (bool) false,\n RDNS: (bool) false,\n RealmTryDomains: (int) -1,\n RenewLifetime: (time.Duration) 0s,\n SafeChecksumType: (int) 8,\n TicketLifetime: (time.Duration) 24h0m0s,\n UDPPreferenceLimit: (int) 1465,\n VerifyAPReqNofail: (bool) false\n },\n Realms: ([]config.Realm) (len=1 cap=1) {\n (config.Realm) {\n Realm: (string) (len=9) "AIOPS.COM",\n AdminServer: ([]string) (len=1 cap=1) {\n (string) (len=15) "eoi03.aiops.com"\n },\n DefaultDomain: (string) "",\n KDC: ([]string) (len=1 cap=1) {\n (string) (len=18) "eoi03.aiops.com:88"\n },\n KPasswdServer: ([]string) (len=1 cap=1) {\n (string) (len=19) "eoi03.aiops.com:464"\n },\n MasterKDC: ([]string) @.@.@.@.@./pkg/kgo/broker.go:255 @.@.@.@.@.@./pkg/kgo/broker.go:255>"} {"level":"debug","ts":"2022-08-08T21:53:35.942+0800","msg":"connection initialization failed","addr":"eoi03.aiops.com:9092","broker":"seed 0","err":"EOF"} {"level":"debug","ts":"2022-08-08T21:53:35.942+0800","msg":"retrying request","tries":1,"backoff":0.205132064,"request_error":"EOF","response_error":"EOF"} {"level":"debug","ts":"2022-08-08T21:53:36.148+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"seed 0"} {"level":"debug","ts":"2022-08-08T21:53:36.149+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"seed 0"} {"level":"debug","ts":"2022-08-08T21:53:36.149+0800","msg":"beginning sasl authentication","broker":"seed 0","mechanism":"GSSAPI","authenticate":false} {"level":"debug","ts":"2022-08-08T21:53:36.149+0800","msg":"issuing raw sasl authenticate","broker":"seed 0","step":0} {"level":"debug","ts":"2022-08-08T21:53:36.155+0800","msg":"issuing raw sasl authenticate","broker":"seed 0","step":1} {"level":"debug","ts":"2022-08-08T21:53:36.155+0800","msg":"connection initialized successfully","addr":"eoi03.aiops.com:9092","broker":"seed 0"} {"level":"debug","ts":"2022-08-08T21:53:36.156+0800","msg":"wrote Metadata v7","broker":"seed 0","bytes_written":44,"write_wait":0.007804852,"time_to_write":0.00002404,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.156+0800","msg":"read Metadata v7","broker":"seed 0","bytes_read":138,"read_wait":0.000199002,"time_to_read":0.000478177,"err":null} {"level":"info","ts":"2022-08-08T21:53:36.157+0800","msg":"beginning to manage the group lifecycle","group":"sensor_sync_tb_result__consumer_group2"} {"level":"info","ts":"2022-08-08T21:53:36.157+0800","msg":"joining group","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:53:36.157+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"0"} {"level":"debug","ts":"2022-08-08T21:53:36.157+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"0"} {"level":"debug","ts":"2022-08-08T21:53:36.157+0800","msg":"issuing api versions request","broker":"0","version":3} {"level":"debug","ts":"2022-08-08T21:53:36.158+0800","msg":"wrote ApiVersions v3","broker":"0","bytes_written":30,"write_wait":0.000054414,"time_to_write":0.00009986,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.158+0800","msg":"read ApiVersions v3","broker":"0","bytes_read":14,"read_wait":0.000149071,"time_to_read":0.00001377,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.158+0800","msg":"kafka does not know our ApiVersions version, downgrading to version 0 and retrying","broker":"0"} {"level":"debug","ts":"2022-08-08T21:53:36.158+0800","msg":"issuing api versions request","broker":"0","version":0} {"level":"debug","ts":"2022-08-08T21:53:36.158+0800","msg":"wrote ApiVersions v0","broker":"0","bytes_written":17,"write_wait":0.000011087,"time_to_write":0.000044637,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.158+0800","msg":"read ApiVersions v0","broker":"0","bytes_read":278,"read_wait":0.000085243,"time_to_read":0.000175691,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.158+0800","msg":"beginning sasl authentication","broker":"0","mechanism":"GSSAPI","authenticate":false} {"level":"debug","ts":"2022-08-08T21:53:36.159+0800","msg":"issuing raw sasl authenticate","broker":"0","step":0} {"level":"error","ts":"2022-08-08T21:53:36.160+0800","msg":"unable to initialize @.@.@.@.@.@./pkg/kgo/broker.go:255 @.@.@.@.@.@./pkg/kgo/broker.go:255>"} {"level":"debug","ts":"2022-08-08T21:53:36.160+0800","msg":"connection initialization failed","addr":"eoi03.aiops.com:9092","broker":"0","err":"EOF"} {"level":"debug","ts":"2022-08-08T21:53:36.161+0800","msg":"retrying request","tries":1,"backoff":0.249883136,"request_error":"EOF","response_error":"EOF"} {"level":"debug","ts":"2022-08-08T21:53:36.411+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"0"} {"level":"debug","ts":"2022-08-08T21:53:36.412+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"0"} {"level":"debug","ts":"2022-08-08T21:53:36.412+0800","msg":"beginning sasl authentication","broker":"0","mechanism":"GSSAPI","authenticate":false} {"level":"debug","ts":"2022-08-08T21:53:36.413+0800","msg":"issuing raw sasl authenticate","broker":"0","step":0} {"level":"debug","ts":"2022-08-08T21:53:36.419+0800","msg":"issuing raw sasl authenticate","broker":"0","step":1} {"level":"debug","ts":"2022-08-08T21:53:36.419+0800","msg":"connection initialized successfully","addr":"eoi03.aiops.com:9092","broker":"0"} {"level":"debug","ts":"2022-08-08T21:53:36.420+0800","msg":"wrote FindCoordinator v2","broker":"0","bytes_written":58,"write_wait":0.008339123,"time_to_write":0.000039035,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.421+0800","msg":"read FindCoordinator v2","broker":"0","bytes_read":41,"read_wait":0.000287621,"time_to_read":0.000612147,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.421+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"0"} {"level":"debug","ts":"2022-08-08T21:53:36.421+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"0"} {"level":"debug","ts":"2022-08-08T21:53:36.421+0800","msg":"beginning sasl authentication","broker":"0","mechanism":"GSSAPI","authenticate":false} {"level":"debug","ts":"2022-08-08T21:53:36.422+0800","msg":"issuing raw sasl authenticate","broker":"0","step":0} {"level":"debug","ts":"2022-08-08T21:53:36.424+0800","msg":"issuing raw sasl authenticate","broker":"0","step":1} {"level":"debug","ts":"2022-08-08T21:53:36.424+0800","msg":"connection initialized successfully","addr":"eoi03.aiops.com:9092","broker":"0"} {"level":"debug","ts":"2022-08-08T21:53:36.424+0800","msg":"wrote JoinGroup v4","broker":"0","bytes_written":149,"write_wait":0.003110953,"time_to_write":0.000037075,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.425+0800","msg":"read JoinGroup v4","broker":"0","bytes_read":68,"read_wait":0.000208609,"time_to_read":0.000617387,"err":null} {"level":"info","ts":"2022-08-08T21:53:36.425+0800","msg":"join returned MemberIDRequired, rejoining with response's MemberID","group":"sensor_sync_tb_result__consumer_group2","member_id":"kgo-afee8a95-6d92-4bc9-8692-a1ac3a8163b0"} {"level":"debug","ts":"2022-08-08T21:53:36.425+0800","msg":"wrote JoinGroup v4","broker":"0","bytes_written":189,"write_wait":0.000052592,"time_to_write":0.000089062,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.428+0800","msg":"read JoinGroup v4","broker":"0","bytes_read":216,"read_wait":0.000160901,"time_to_read":0.002034644,"err":null} {"level":"info","ts":"2022-08-08T21:53:36.429+0800","msg":"joined, balancing group","group":"sensor_sync_tb_result__consumer_group2","member_id":"kgo-afee8a95-6d92-4bc9-8692-a1ac3a8163b0","instance_id":null,"generation":1,"balance_protocol":"cooperative-sticky","leader":true} {"level":"info","ts":"2022-08-08T21:53:36.429+0800","msg":"balancing group as leader"} {"level":"info","ts":"2022-08-08T21:53:36.429+0800","msg":"balance group member","id":"kgo-afee8a95-6d92-4bc9-8692-a1ac3a8163b0","interests":"interested topics: [sensor-online-detect], previously owned: "} {"level":"info","ts":"2022-08-08T21:53:36.429+0800","msg":"balanced","plan":"kgo-afee8a95-6d92-4bc9-8692-a1ac3a8163b0{sensor-online-detect[0]}"} {"level":"info","ts":"2022-08-08T21:53:36.430+0800","msg":"syncing","group":"sensor_sync_tb_result__consumer_group2","protocol_type":"consumer","protocol":"cooperative-sticky"} {"level":"debug","ts":"2022-08-08T21:53:36.430+0800","msg":"wrote SyncGroup v2","broker":"0","bytes_written":193,"write_wait":0.000082896,"time_to_write":0.000106663,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.432+0800","msg":"read SyncGroup v2","broker":"0","bytes_read":58,"read_wait":0.000201882,"time_to_read":0.000987525,"err":null} {"level":"info","ts":"2022-08-08T21:53:36.432+0800","msg":"synced","group":"sensor_sync_tb_result__consumer_group2","assigned":"sensor-online-detect[0]"} {"level":"info","ts":"2022-08-08T21:53:36.432+0800","msg":"new group session begun","group":"sensor_sync_tb_result__consumer_group2","added":"sensor-online-detect[0]","lost":""} {"level":"info","ts":"2022-08-08T21:53:36.432+0800","msg":"beginning heartbeat loop","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:53:36.432+0800","msg":"entering OnPartitionsAssigned","with":{"sensor-online-detect":[0]}} {"level":"debug","ts":"2022-08-08T21:53:36.432+0800","msg":"sharded request","destinations":["0"]} {"level":"debug","ts":"2022-08-08T21:53:36.433+0800","msg":"wrote OffsetFetch v5","broker":"0","bytes_written":91,"write_wait":0.000110089,"time_to_write":0.000054629,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.433+0800","msg":"read OffsetFetch v5","broker":"0","bytes_read":64,"read_wait":0.000209909,"time_to_read":0.000546262,"err":null} {"level":"info","ts":"2022-08-08T21:53:36.434+0800","msg":"assigning partitions","why":"newly fetched offsets for group sensor_sync_tb_result__consumer_group2","how":"assigning everything new, keeping current assignment","input":"sensor-online-detect[0{-1 e-1 ce0}]"} {"level":"debug","ts":"2022-08-08T21:53:36.434+0800","msg":"assign requires loading offsets"} {"level":"debug","ts":"2022-08-08T21:53:36.434+0800","msg":"offsets to load broker","broker":0,"load":{"List":{"sensor-online-detect":{"0":{"At":-1,"Epoch":-1,"CurrentEpoch":-1}}},"Epoch":null}} {"level":"debug","ts":"2022-08-08T21:53:36.434+0800","msg":"wrote ListOffsets v5","broker":"0","bytes_written":68,"write_wait":0.000063425,"time_to_write":0.000055628,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.435+0800","msg":"read ListOffsets v5","broker":"0","bytes_read":68,"read_wait":0.000183102,"time_to_read":0.00034523,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.435+0800","msg":"handled list results","broker":"0","using":{"sensor-online-detect":{"0":{"Offset":43081,"LeaderEpoch":0}}},"reloading":{}} {"level":"debug","ts":"2022-08-08T21:53:36.436+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"0"} {"level":"debug","ts":"2022-08-08T21:53:36.436+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"0"} {"level":"debug","ts":"2022-08-08T21:53:36.436+0800","msg":"beginning sasl authentication","broker":"0","mechanism":"GSSAPI","authenticate":false} {"level":"debug","ts":"2022-08-08T21:53:36.437+0800","msg":"issuing raw sasl authenticate","broker":"0","step":0} {"level":"debug","ts":"2022-08-08T21:53:36.439+0800","msg":"issuing raw sasl authenticate","broker":"0","step":1} {"level":"debug","ts":"2022-08-08T21:53:36.440+0800","msg":"connection initialized successfully","addr":"eoi03.aiops.com:9092","broker":"0"} {"level":"debug","ts":"2022-08-08T21:53:36.440+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":104,"write_wait":0.004231101,"time_to_write":0.000026991,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.933+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:53:36.934+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000118263,"time_to_write":0.000072352,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.935+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000216693,"time_to_read":0.000771378,"err":null} {"level":"debug","ts":"2022-08-08T21:53:36.935+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:53:39.433+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:53:39.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000047264,"time_to_write":0.000056006,"err":null} {"level":"debug","ts":"2022-08-08T21:53:39.433+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000159455,"time_to_read":0.000368021,"err":null} {"level":"debug","ts":"2022-08-08T21:53:39.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:53:41.441+0800","msg":"read Fetch v10","broker":"0","bytes_read":86,"read_wait":0.000237577,"time_to_read":5.001357899,"err":null} {"level":"debug","ts":"2022-08-08T21:53:41.442+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":50,"write_wait":0.000095733,"time_to_write":0.000047468,"err":null} {"level":"debug","ts":"2022-08-08T21:53:42.433+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:53:42.434+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000094636,"time_to_write":0.000104473,"err":null} {"level":"debug","ts":"2022-08-08T21:53:42.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000206289,"time_to_read":0.000553904,"err":null} {"level":"debug","ts":"2022-08-08T21:53:42.435+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:53:45.433+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:53:45.434+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000093736,"time_to_write":0.000108296,"err":null} {"level":"debug","ts":"2022-08-08T21:53:45.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000219613,"time_to_read":0.000584214,"err":null} {"level":"debug","ts":"2022-08-08T21:53:45.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:53:46.445+0800","msg":"read Fetch v10","broker":"0","bytes_read":22,"read_wait":0.000153704,"time_to_read":5.002250854,"err":null} {"level":"debug","ts":"2022-08-08T21:53:46.445+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":50,"write_wait":0.000100046,"time_to_write":0.000065938,"err":null} {"level":"debug","ts":"2022-08-08T21:53:48.432+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:53:48.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000053773,"time_to_write":0.000072742,"err":null} {"level":"debug","ts":"2022-08-08T21:53:48.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000101954,"time_to_read":0.00059827,"err":null} {"level":"debug","ts":"2022-08-08T21:53:48.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:53:51.432+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:53:51.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000089546,"time_to_write":0.000080599,"err":null} {"level":"debug","ts":"2022-08-08T21:53:51.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000221096,"time_to_read":0.000600487,"err":null} {"level":"debug","ts":"2022-08-08T21:53:51.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:53:51.446+0800","msg":"read Fetch v10","broker":"0","bytes_read":22,"read_wait":0.000277931,"time_to_read":5.00046055,"err":null} {"level":"debug","ts":"2022-08-08T21:53:51.446+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":50,"write_wait":0.000093926,"time_to_write":0.000054808,"err":null} {"level":"debug","ts":"2022-08-08T21:53:54.432+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:53:54.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000078025,"time_to_write":0.000074391,"err":null} {"level":"debug","ts":"2022-08-08T21:53:54.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000144759,"time_to_read":0.000887487,"err":null} {"level":"debug","ts":"2022-08-08T21:53:54.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:53:56.447+0800","msg":"read Fetch v10","broker":"0","bytes_read":22,"read_wait":0.000166741,"time_to_read":5.000721184,"err":null} {"level":"debug","ts":"2022-08-08T21:53:56.448+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":50,"write_wait":0.000070338,"time_to_write":0.000024782,"err":null} {"level":"debug","ts":"2022-08-08T21:53:57.433+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:53:57.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000073018,"time_to_write":0.000084473,"err":null} {"level":"debug","ts":"2022-08-08T21:53:57.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000259344,"time_to_read":0.000318078,"err":null} {"level":"debug","ts":"2022-08-08T21:53:57.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:00.433+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:00.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000048723,"time_to_write":0.000029887,"err":null} {"level":"debug","ts":"2022-08-08T21:54:00.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000087079,"time_to_read":0.000951728,"err":null} {"level":"debug","ts":"2022-08-08T21:54:00.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:01.449+0800","msg":"read Fetch v10","broker":"0","bytes_read":22,"read_wait":0.000108507,"time_to_read":5.001327256,"err":null} {"level":"debug","ts":"2022-08-08T21:54:01.450+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":50,"write_wait":0.000058995,"time_to_write":0.000060435,"err":null} {"level":"debug","ts":"2022-08-08T21:54:03.432+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:03.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000096803,"time_to_write":0.000050394,"err":null} {"level":"debug","ts":"2022-08-08T21:54:03.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000203129,"time_to_read":0.000688723,"err":null} {"level":"debug","ts":"2022-08-08T21:54:03.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:06.433+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:06.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000086098,"time_to_write":0.000095907,"err":null} {"level":"debug","ts":"2022-08-08T21:54:06.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000184016,"time_to_read":0.000718634,"err":null} {"level":"debug","ts":"2022-08-08T21:54:06.435+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:06.451+0800","msg":"read Fetch v10","broker":"0","bytes_read":22,"read_wait":0.000178248,"time_to_read":5.001196254,"err":null} {"level":"debug","ts":"2022-08-08T21:54:06.452+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":50,"write_wait":0.000044121,"time_to_write":0.000051951,"err":null} {"level":"debug","ts":"2022-08-08T21:54:09.432+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:09.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000077509,"time_to_write":0.000069131,"err":null} {"level":"debug","ts":"2022-08-08T21:54:09.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000222867,"time_to_read":0.000541548,"err":null} {"level":"debug","ts":"2022-08-08T21:54:09.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:11.452+0800","msg":"read Fetch v10","broker":"0","bytes_read":22,"read_wait":0.000158871,"time_to_read":5.000496087,"err":null} {"level":"debug","ts":"2022-08-08T21:54:11.453+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":50,"write_wait":0.000080139,"time_to_write":0.000057368,"err":null} {"level":"debug","ts":"2022-08-08T21:54:12.433+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:12.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000109172,"time_to_write":0.000069693,"err":null} {"level":"debug","ts":"2022-08-08T21:54:12.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000161467,"time_to_read":0.000889355,"err":null} {"level":"debug","ts":"2022-08-08T21:54:12.435+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:15.432+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:15.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000092588,"time_to_write":0.000040829,"err":null} {"level":"debug","ts":"2022-08-08T21:54:15.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000088989,"time_to_read":0.001612425,"err":null} {"level":"debug","ts":"2022-08-08T21:54:15.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:15.934+0800","msg":"reaped connections","time_since_last_reap":20.000405659,"reap_dur":0.000208529,"num_reaped":2} {"level":"debug","ts":"2022-08-08T21:54:16.454+0800","msg":"read Fetch v10","broker":"0","bytes_read":22,"read_wait":0.000183965,"time_to_read":5.000806328,"err":null} {"level":"debug","ts":"2022-08-08T21:54:16.454+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":50,"write_wait":0.000044301,"time_to_write":0.000027943,"err":null} {"level":"debug","ts":"2022-08-08T21:54:18.433+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:18.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.00010529,"time_to_write":0.000085179,"err":null} {"level":"debug","ts":"2022-08-08T21:54:18.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000297022,"time_to_read":0.000363953,"err":null} {"level":"debug","ts":"2022-08-08T21:54:18.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:21.433+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:21.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000079948,"time_to_write":0.000054622,"err":null} {"level":"debug","ts":"2022-08-08T21:54:21.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000206472,"time_to_read":0.000858798,"err":null} {"level":"debug","ts":"2022-08-08T21:54:21.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:21.456+0800","msg":"read Fetch v10","broker":"0","bytes_read":22,"read_wait":0.000084819,"time_to_read":5.001337876,"err":null} {"level":"debug","ts":"2022-08-08T21:54:21.456+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":50,"write_wait":0.000100242,"time_to_write":0.000047522,"err":null} {"level":"debug","ts":"2022-08-08T21:54:24.433+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:24.434+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000079299,"time_to_write":0.000062262,"err":null} {"level":"debug","ts":"2022-08-08T21:54:24.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000194875,"time_to_read":0.000536899,"err":null} {"level":"debug","ts":"2022-08-08T21:54:24.435+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:26.458+0800","msg":"read Fetch v10","broker":"0","bytes_read":22,"read_wait":0.000185518,"time_to_read":5.001335341,"err":null} {"level":"debug","ts":"2022-08-08T21:54:26.459+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":50,"write_wait":0.000117981,"time_to_write":0.000084095,"err":null} {"level":"debug","ts":"2022-08-08T21:54:27.432+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:27.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000106787,"time_to_write":0.000086552,"err":null} {"level":"debug","ts":"2022-08-08T21:54:27.434+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000313099,"time_to_read":0.000261261,"err":null} {"level":"debug","ts":"2022-08-08T21:54:27.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:30.432+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:30.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.00007735,"time_to_write":0.000049134,"err":null} {"level":"debug","ts":"2022-08-08T21:54:30.433+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000162439,"time_to_read":0.000487474,"err":null} {"level":"debug","ts":"2022-08-08T21:54:30.434+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:31.459+0800","msg":"read Fetch v10","broker":"0","bytes_read":22,"read_wait":0.000238659,"time_to_read":5.000460674,"err":null} {"level":"debug","ts":"2022-08-08T21:54:31.460+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":50,"write_wait":0.000099828,"time_to_write":0.000052516,"err":null} {"level":"debug","ts":"2022-08-08T21:54:33.432+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:33.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000071778,"time_to_write":0.000049358,"err":null} {"level":"debug","ts":"2022-08-08T21:54:33.433+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000175785,"time_to_read":0.000112283,"err":null} {"level":"debug","ts":"2022-08-08T21:54:33.433+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:36.433+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:36.434+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000091912,"time_to_write":0.000044628,"err":null} {"level":"debug","ts":"2022-08-08T21:54:36.435+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.00033708,"time_to_read":0.000620173,"err":null} {"level":"debug","ts":"2022-08-08T21:54:36.435+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:36.462+0800","msg":"read Fetch v10","broker":"0","bytes_read":22,"read_wait":0.000202251,"time_to_read":5.001322693,"err":null} {"level":"debug","ts":"2022-08-08T21:54:36.462+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":50,"write_wait":0.000079352,"time_to_write":0.000067095,"err":null} {"level":"debug","ts":"2022-08-08T21:54:39.432+0800","msg":"heartbeating","group":"sensor_sync_tb_result__consumer_group2"} {"level":"debug","ts":"2022-08-08T21:54:39.433+0800","msg":"wrote Heartbeat v2","broker":"0","bytes_written":103,"write_wait":0.000068472,"time_to_write":0.000076345,"err":null} {"level":"debug","ts":"2022-08-08T21:54:39.433+0800","msg":"read Heartbeat v2","broker":"0","bytes_read":14,"read_wait":0.000248384,"time_to_read":0.000313312,"err":null} {"level":"debug","ts":"2022-08-08T21:54:39.433+0800","msg":"heartbeat complete","group":"sensor_sync_tb_result__consumer_group2","err":null} {"level":"debug","ts":"2022-08-08T21:54:41.464+0800","msg":"read Fetch v10","broker":"0","bytes_read":22,"read_wait":0.000314116,"time_to_read":5.001097684,"err":null} {"level":"debug","ts":"2022-08-08T21:54:41.464+0800","msg":"wrote Fetch v10","broker":"0","bytes_written":50,"write_wait":0.000103903,"time_to_write":0.000049605,"err":null}\n }\n },\n DomainRealm: (config.DomainRealm) (len=2) {\n (string) (len=10) ".aiops.com": (string) (len=9) "AIOPS.COM",\n (string) (len=9) "aiops.com": (string) (len=9) "AIOPS.COM"\n }\n }),\n settings: (client.Settings)(0xc0004a9f10)({\n disablePAFXFast: (bool) false,\n assumePreAuthentication: (bool) false,\n preAuthEType: (int32) 0,\n logger: (log.Logger)( )\n }),\n sessions: (client.sessions)(0xc000099120)({\n Entries: (map[string]client.session) {\n },\n mux: (sync.RWMutex) {\n w: (sync.Mutex) {\n state: (int32) 0,\n sema: (uint32) 0\n },\n writerSem: (uint32) 0,\n readerSem: (uint32) 0,\n readerCount: (int32) 0,\n readerWait: (int32) 0\n }\n }),\n cache: (client.Cache)(0xc000099100)({\n Entries: (map[string]client.CacheEntry) {\n },\n mux: (sync.RWMutex) {\n w: (sync.Mutex) {\n state: (int32) 0,\n sema: (uint32) 0\n },\n writerSem: (uint32) 0,\n readerSem: (uint32) 0,\n readerCount: (int32) 0,\n readerWait: (int32) 0\n }\n })\n})\n"} {"level":"info","ts":"2022-08-08T21:53:35.933+0800","msg":"applied the first config"} {"level":"info","ts":"2022-08-08T21:53:35.934+0800","msg":"immediate metadata update triggered","why":"client initialization"} {"level":"debug","ts":"2022-08-08T21:53:35.934+0800","msg":"opening connection to broker","addr":"eoi03.aiops.com:9092","broker":"seed 0"} {"level":"debug","ts":"2022-08-08T21:53:35.935+0800","msg":"connection opened to broker","addr":"eoi03.aiops.com:9092","broker":"seed 0"} {"level":"debug","ts":"2022-08-08T21:53:35.935+0800","msg":"issuing api versions request","broker":"seed 0","version":3} {"level":"debug","ts":"2022-08-08T21:53:35.935+0800","msg":"wrote ApiVersions v3","broker":"seed 0","bytes_written":30,"write_wait":0.000047282,"time_to_write":0.000082942,"err":null} {"level":"debug","ts":"2022-08-08T21:53:35.935+0800","msg":"read ApiVersions v3","broker":"seed 0","bytes_read":14,"read_wait":0.0002342,"time_to_read":0.00001666,"err":null} {"level":"debug","ts":"2022-08-08T21:53:35.935+0800","msg":"kafka does not know our ApiVersions version, downgrading to version 0 and retrying","broker":"seed 0"} {"level":"debug","ts":"2022-08-08T21:53:35.935+0800","msg":"issuing api versions request","broker":"seed 0","version":0} {"level":"debug","ts":"2022-08-08T21:53:35.936+0800","msg":"wrote ApiVersions v0","broker":"seed 0","bytes_written":17,"write_wait":0.00001175,"time_to_write":0.000046688,"err":null} {"level":"debug","ts":"2022-08-08T21:53:35.936+0800","msg":"read ApiVersions v0","broker":"seed 0","bytes_read":278,"read_wait":0.000067585,"time_to_read":0.000012964,"err":null} {"level":"debug","ts":"2022-08-08T21:53:35.936+0800","msg":"beginning sasl authentication","broker":"seed 0","mechanism":"GSSAPI","authenticate":false} {"level":"debug","ts":"2022-08-08T21:53:35.941+0800","msg":"issuing raw sasl authenticate","broker":"seed 0","step":0} {"level":"error","ts":"2022-08-08T21:53:35.942+0800","msg":"unable to initialize sasl","broker":"seed @. — Reply to this email directly, view it on GitHub https://github.com/twmb/franz-go/issues/182#issuecomment-1208176715, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJZIJKLJNY6CAGA4C2WZ6DVYEH6BANCNFSM54Z5HXJA . You are receiving this because you commented.Message ID: @.***>
At least clickhouse_sinker is able to consume messages as you sugguested code change. Thanks for your help! It's OK to close this issue.
Reading the code piece, I'm afraid that each run of GetTaskStateAndLags(cfg *config.Config) leaks a goroutine. Am I right?
// PersistAfterAuth specifies whether the client should persist after
// logging in or if it should be destroyed (the default).
//
// If persisting, we never call client.Destroy ourselves, and it is
// expected that you will return the same client in every authFn. The
// client itself spins up a goroutine to automatically renew sessions,
// thus if you return the same client, nothing leaks, but if you return
// a new client on every call and set PersistAfterAuth, goroutines will
// leak.
PersistAfterAuth bool
It wont leak goroutines. The key lines are here: https://github.com/twmb/franz-go/blob/514f3638dbf5e74cfdcfdca4e93f7e04b9fcf1ac/pkg/sasl/kerberos/kerberos.go#L65-L72
Basically, this code will always return your same original input client. By persisting after authorization, the sasl flow will avoid ever calling Destroy. This means franz-go trusts that you (the user) will manage if or when the kerberos client is ever destroyed. In clickhouse_sinker, you never destroy, so the client is technically "leaked", but it's leaked once on initialization and used forever after.
In short, the client will be initialized once and used until clickhouse_sinker shuts down.
I am planning to change AsMechanism to set PersistAfterAuth to true, fwiw.
Scratch that reply, sorry I misread what you linked to in my email. Yes, each call there will leak a kerberos client, which will leak a goroutine. I can think about adding something in franz-go to manage that ... but the best thing to do is to close the kerberos client yourself if non-nil.
This is basically the fundamental problem with the underlying kerberos client... the same problem exists in sarama, with NewKerberosClientFunc. (edit: this might not be true, I think they create a new client on every login. I'll see what makes the most sense within franz-go)
I think what I'll do:
- add a new field to Opt,
ClientFn, if that is non-nil, use it. - add an optional interface that destroys a sasl, check this within franz-go and call it on shutdown for all sasls that implement it (i.e., always destroy on shutdown)
The user calls kgo.Client.Close() to release all resources associated with it. I think the kerberos client shall be closed inside there no matter what PersistAfterAuth is.
I think I can implement some of this internally with a 1.6.1 patch -- specifically the second point -- and a new minor version of the kerberos dep.
(deleted two comments above now that the original issue has been diagnosed just in case they have some inadvertent auth in the logs)
The features have now been released, please check out the commits just above, this is in pkg/sasl/kerberos v1.1.0:
go get github.com/twmb/franz-go/pkg/sasl/[email protected]
You will want to change to using AsMechanismWithClose to opt into closing the Kerberos client when you close the franz-go client. As well, AsMechanism and AsMechanismWithClose both by default set PersistAfterAuth to true. Lastly, if you want to use this in a more sarama like manner, ClientFn exists (see docs).