Multi-core Usage, but single-core performance?

[Hossy]

Sorry to re-ask an old question ( #42 ), but I'm trying to troubleshoot a performance issue with my production instance and graphing CPU over time, it seems I'm running into a ceiling on CPU. The process shows 100-103%, but the OD shows 25-26% (4-core box doing basically nothing else). We do not have GOMAXPROCS set in the environment. Any suggestions on what else I could look at if, indeed, tile38 should be using more CPU ticks?

[tidwall]

I'm assuming that in your case the tile38-server process isn't being throttled by the host, such as by using the Docker --cpus flag, and you mention not using the GOMAXPROCS environment variable. Thus, Tile38 will use as many cores as it can.

Perhaps what you are running into is heavy lock contention. Tile38 uses locking for multiple readers and a single writer. Write commands like SET and DEL will lock the database until the that command completes. Read commands like INTERSECTS and SCAN will run concurrently.

This means that if all you are doing is a ton of SETs then you may not see much better that 100%.

But, Tile38 does a lot of things behind the scenes using other cores. Like sending geofence notifications, syncing with replicas, and routine maintenance. Also if you use multiple client connections, then each one of those will benefit a little from the extra cores.

[Hossy]

Thanks, @tidwall. This is good information and we are discussing internally on our use.

Are there any metrics we can get out of tile38 to see what it's spending its time on? commands? operations? fences?

[iwpnd]

https://tile38.com/topics/configuration#prometheus-metrics

That might come in handy

[Hossy]

So, more of a theory/in-practice follow-up question here...

The way we are using Tile38 is to determine if a moving object enters/exits/passes through a geo-fence. The way we are doing this is by issuing SETs each time the location of the object changes. To @tidwall 's point, this would result (and is what we are seeing) in performance being very close to 100%. Is there a better way of achieving the same goal (answering the enter/exit/passthrough)?