Botond Dénes
Botond Dénes
This bucket is write-only for anonymous users, can be read only by scylla employees.
Not yet, I've been busy with another coredump. I will look at this one now.
I cannot get thread debugging to work. You mentioned above that the coredump is 63GB large. After downloading and extracting I have a 59GB file: ``` [root@fedora 11223]# sha256sum ./core.67112...
Please do if you can. Note that the coredump is as big as the memory the process have mapped so far, so making it early won't make it smaller as...
BTW what are your workloads like? Do you delete a lot? Do you do range deletions?
> I have been deleting a lot of stale data lately with token range queries. In one table, time_series_2, about two thirds of the data is stale. I have been...
Thanks. Note that I will be off for the rest of the week, so I'll probably only get a good look at the core next week.
Thread debugging still not working but this time the coredump has a reader in its stacktrace, so I can look at it.
``` (gdb) p $7._semaphore._name $10 = "_streaming_concurrency_sem" (gdb) p $7._semaphore._stats $11 = {permit_based_evictions = 22, time_based_evictions = 1, inactive_reads = 0, total_successful_reads = 4, total_failed_reads = 0, total_reads_shed_due_to_overload = 0,...
The culprit is the new admission algorithm introduced by 3d816b7c1. In particular, when the repair falls back to the multishard reader, shard readers are racing with other repair-meta for permits....