teslamotors
High ram use till crash . Raised by 2mb per minute
We are running on an aws box , for now just 2 cars but intention is to add a few hundred.
Db is rds which is always under 2% load. Server is aws t3.large
When service first starts, all is well but the ram usage keeps increasing. After about 3 days the ram is so high that the server crashes. Ram is been used by Fleet-telemetry
Anyone else experienced/ solved this?
That's strange, we serve 1000+ vehicles on 200-250MB of ram. Is it possible that all the messages remain in memory somehow? Maybe logging?
What comes to mind is if you are using something like Kafka but not acknowledging your messages so they are remaining on the Fleet Telemetry server.
Can you share details on your config, like what producer you're using and how it's configured.
I will get the details and config on Monday but we are writing directly to MySQL and not via Kafka.
I will get the details and config on Monday but we are writing directly to MySQL and not via Kafka.
So you have made modifications to the code? That's likely your issue. Are you acknowledging the messages processed in your custom MySQL producer?