Israel Fruchter comments

Results 974 comments of


                                            Israel Fruchter

[K8S][Nemesis] SCT considers an old DB log line as a new one and creates false ERROR events based on it later running other nemesis

> Ok, I understand the issue. The problem is that we decommission node (k8s) and new one gets the same name. This causes to create `sdcm.db_log_reader.DbLogReader` which opens already existing...

[K8S][Nemesis] SCT considers an old DB log line as a new one and creates false ERROR events based on it later running other nemesis

> @soyacz , @fruch this bug is one more reason to finally implement the > > * [Use daemonset to stream k8s logs to SCT in realtime qa-tasks#1148](https://github.com/scylladb/qa-tasks/issues/1148) > >...

[K8S][Nemesis] SCT considers an old DB log line as a new one and creates false ERROR events based on it later running other nemesis

@vponomaryov you are forgetting our log reading is splits into two, one part that write into a file, and one that reads. the files path stay the same, even if...

[K8S][Nemesis] SCT considers an old DB log line as a new one and creates false ERROR events based on it later running other nemesis

> > @vponomaryov you are forgetting our log reading is splits into two, one part that write into a file, and one that reads. the files path stay the same,...

[K8S][Nemesis] SCT considers an old DB log line as a new one and creates false ERROR events based on it later running other nemesis

@vponomaryov can you estimate how long it would take ? looking for all of the places we might kill a node in k8s, and apply this ?

[K8S][Nemesis] SCT considers an old DB log line as a new one and creates false ERROR events based on it later running other nemesis

> > @vponomaryov can you estimate how long it would take ? looking for all of the places we might kill a node in k8s, and apply this ? >...

[K8S][Nemesis] SCT considers an old DB log line as a new one and creates false ERROR events based on it later running other nemesis

> > The diff between real event happens and it's handling by SCT is about `5h20m`. So, looks like the `disrupt_multiple_hard_reboot_node` nemesis must be fixed the same way as it...

[K8S][Nemesis] SCT considers an old DB log line as a new one and creates false ERROR events based on it later running other nemesis

We need to understand, the actual source of the issue, before we can decide on a proper fix for it.

`node_exporter` may hang on a DB node with the `error encoding and sending metric family: write tcp %IP%:9100` error

@vponomaryov if it's the node_exporter on the DB node, I think it's something that needs to be reported on scylla core...

`node_exporter` may hang on a DB node with the `error encoding and sending metric family: write tcp %IP%:9100` error

> > [@vponomaryov](https://github.com/vponomaryov) > > if it's the node_exporter on the DB node, I think it's something that needs to be reported on scylla core... > > We have a...