fleet-telemetry icon indicating copy to clipboard operation
fleet-telemetry copied to clipboard

Sending a new Fleet Config is ignored sometimes

Open Urkman opened this issue 1 year ago • 15 comments

When I send a new Fleet Config to the vehicle using the fleet_telemetry_config POST. It is sometimes ignored and the Vehicle keeps sending the old data for minutes...

The réponse of the post is fine: {"response":{"updated_vehicles":1}}

What can I do about this?

Urkman avatar Jun 22 '24 19:06 Urkman

And to add: When I get the current config using leet_telemetry_config GET it gives me the config I just send with synced: true

So, why is he still sending old config messages?

Urkman avatar Jun 23 '24 07:06 Urkman

I'm sending the following config. And after sending it, I did a 10 minute drive nd he is still sending the old messages :(

{
  "vins": [
    "MY_VIN"
  ],
  "config": {
    "hostname": "api.my-app.app",
    "port": 443,
    "ca": "MY_CA",
    "fields": {
      "Gear": {
        "interval_seconds": 12
      },
      "ChargeState": {
        "interval_seconds": 12
      }
    }
  }
}

Urkman avatar Jun 23 '24 09:06 Urkman

And now. Another 10 Minutes later it starts sending these messages :( So, after total of about 25 minutes after sending the config it starts sending the messages.

So, please don't tell me, that this is "by design" :(

Urkman avatar Jun 23 '24 09:06 Urkman

Some more tests...

First:

State: Tesla is sending messages. I send a fleet_telemetry_config DELETE Tesla stopps sending messages. fleet_telemetry_config GET return an empty config.

FINE... WORKS

Next:

State: Tesla is not sending messages Now I send the config again with fleet_telemetry_config POST Response: {"response":{"updated_vehicles":1}} Tesla starts sending messages.

FINE

BUT: fleet_telemetry_config GET return the config, but says: synced: false

🤷‍♂️

Urkman avatar Jun 23 '24 10:06 Urkman

Hi @Urkman I know you're frustrated. Let's try and work through these different issues.

  • The vehicle changing to the new configuration is not instantaneous. It depends on async backend delays and vehicle connectivity. It should be within 10-15 seconds when the vehicle is already online.
  • It is expected for there to be a short period in which you're receiving messages with the new config but the GET endpoint says it is not synced.
  • I am surprised you're getting the updated config and synced == true, but are still receiving outdated messages. Does it stay in this state for an extended period of time? I'm looking at logs and it looks like you're sending a lot of POST fleet_telemetry_config requests. I'm wondering if this is contributing to the problem. What is your use case for sending up to 10 configs per minute?

patrickdemers6 avatar Jun 26 '24 22:06 patrickdemers6

I tried maybe the same a coupe of months ago and gave up. Sometimes the config change was working within seconds, sometimes a couple of hours

Depending on vehicle "state" i need different fields and don't want to receive unnecessary fields. I have this states in my app:

  • driving: location, speed ....
  • DC charging: kW, SOC ...
  • AC charging: same as DC charging, but frequency is much lower
  • online: SentryMode, doors, windows ...

bassmaster187 avatar Jun 27 '24 08:06 bassmaster187

The vehicle changing to the new configuration is not instantaneous. It depends on async backend delays and vehicle connectivity. It should be within 10-15 seconds when the vehicle is already online.

Yes, that would be the way I would expect it to work and this should be fine.

It is expected for there to be a short period in which you're receiving messages with the new config but the GET endpoint says it is not synced.

Also fine :)

I am surprised you're getting the updated config and synced == true, but are still receiving outdated messages. Does it stay in this state for an extended period of time?

Yes I had this just an hour ago with one of my testers... I send this config fields: (at: 2024-06-27 09:56:07)

"fields": {
      "Location": {
        "interval_seconds": 60
      },
      "VehicleSpeed": {
        "interval_seconds": 60
      },
      "EnergyRemaining": {
        "interval_seconds": 60
      },
      "Soc": {
        "interval_seconds": 60
      },
      "RatedRange": {
        "interval_seconds": 60
      },
      "Odometer": {
        "interval_seconds": 60
      }
    }

And got the response: {"response":{"updated_vehicles":1}}

But the car keeps sending the old data for nearly 90 minutes... At 2024-06-27 11:20:35 it send the requested data for the first time :(

I'm looking at logs and it looks like you're sending a lot of POST fleet_telemetry_config requests. I'm wondering if this is contributing to the problem. What is your use case for sending up to 10 configs per minute?

This is the as @bassmaster187 said: I try to get around limitations... We need totally different sets of data for:

  • Charging AC and DC
  • Driving
  • Just online data
  • Waiting for a charge or drive to start.

And with 10 fields per minute this is not usable. I'm currently interested in about 25 field (but I think there could be more). When I would ask for all this fields I would get the values about every 3 minutes. But this is way to long to track a drive (eg waypoints) and in a worst case the drive could already be over :(

So, that's why I heavily use the POST fleet_telemetry_config...

Urkman avatar Jun 27 '24 12:06 Urkman

Got it, thanks for explaining. I'll try and figure out a better solution for you 👍

patrickdemers6 avatar Jul 01 '24 20:07 patrickdemers6

@patrickdemers6 This is still happening :(

See the attached screenshot with some logs data: In the bottom row you see, that it detects a drive, so he send the config for driving, what responses with success.

But he keeps sending the Idle data, so he wants to send the new config again, but as you see on the same log-entry the Tesla already has this config, so he doesn't sends it again. And this keeps going on for quite some time :(

Do you have any news on this?

Bildschirmfoto 2024-07-08 um 11 38 02

Urkman avatar Jul 08 '24 09:07 Urkman

I am also seeing similar issues. I am testing with my car and the car is online with sentry mode on and the /api/1/vehicles/<vin>/fleet_telemetry_config endpoint is returning "synced": true a few seconds after sending the config.

Currently waiting ~10 minutes and still no update.

rileymd88 avatar Jul 22 '24 14:07 rileymd88

This is again a huge problem today :( A lot of the Teslas using my App are Hong this problem. And it is not related to the new 2024.26.x software version. It's also happening on 20.24.20.9.

@patrickdemers6 can you please take a Quick Look at this. This is really destroying my App :(

Urkman avatar Aug 09 '24 08:08 Urkman

@patrickdemers6 If you need a vin to check, let my know.

Urkman avatar Aug 09 '24 08:08 Urkman

This is still a huge problem :( @patrickdemers6 can please give us an update on what the problem here is?

Urkman avatar Aug 14 '24 12:08 Urkman

These configurations are meant to be pretty static and syncing configs isn't guaranteed to be immediate as many factors are at play. I'm working on improving the experience so you don't need to change the configuration.

I will check too see if I can figure out why synced would be turning true but updated data not being sent. If anyone can email support a VIN that would be great.

patrickdemers6 avatar Aug 24 '24 01:08 patrickdemers6

Hi @patrickdemers6,

I think the only valid solution for this would be to remove this strict limits on the telemetry. Max 10 fields every 60 sec. is way too low.

Perhaps changing this to: 25 fields every 10 sec.

This would solve a lot of problems, as we don't need to change the fleet config this often. Perhaps we don't need to change it ever :)

Urkman avatar Sep 12 '24 10:09 Urkman

Hi @patrickdemers6 ,

this is still a huge problem :( Today again some drives are not recorded, because the ignored the new Config. How can I solve this and how can I support you to find the problem?

In Tessie the drive was recorded, so the data for them seems to be fine...

Please do something about this.

Thanks, Urkman

PS: What is the reason that we need to set the config with interval_seconds? Is it just to reduce data traffic of the Tesla? But, when I watch 5 Minutes Netflix I wasted more traffic then a year of sending this data...

Urkman avatar Oct 22 '24 08:10 Urkman

This doesn't seem to be a widespread issue anymore. If it still is, please raise a support ticket!

patrickdemers6 avatar Jan 22 '25 03:01 patrickdemers6