Magnetic Deviation

[tkurki]

For example HDG NMEA 0183 has heading, I believe we are missing it.

A bit tricky, as it is associated with the sensor and current heading, so where should it go?

[timmathews]

Deviation should be stored wherever we define static properties of a sensor. Its naturally a table of heading ranges and deviation angles. It should probably be added to the headingMagnetic and headingTrue values before they're sent out on the wire.

What is the NMEA convention here? In the HDG sentence, if deviation is present should it be added to the heading value or is purely informational and assumed to be already accounted for by the instrument?

A deviation table schema might look like:

{
  "deviation": [
    {
      "start": 0,
      "stop": 15.99,
      "deviation": 3
    },
    {
      "start": 16,
      "stop": 30.99,
      "deviation": -1
    },
    {
      "start": 31,
      "stop": 45.99,
      "deviation": 7
    },
    { ... }
  ]
}

[sumps]

@timmathews further to our discussions last night, I think we should have a dynamic deviation value in Signal K that we take from the NMEA 0183 HDG sentence and NMEA2000 PGN 127250 if present that is the current deviation value that is being used by the heading system. I am not sure how many systems output this data but even if they do not, if we have deviation defined in Signal K then sensors can use it or if we also support a Deviation table, a Signal K server or client could interpolate the current deviation and populate the current deviation key.

The deviation table should be a table with deviation values every 15degs, so 24 values which then allows for a very accurate table with all values entered or a less accurate table with values for every 30degs or even 45degs.

[sumps]

A server could also add value by working out magnetic variation using this code from NOAA...

https://www.ngdc.noaa.gov/geomag/WMM/soft.shtml

[rob42]

This has more to do with correcting data than the real value of the heading or any other key. If I understand correctly the deviation we are talking about here is the compass error on the N2K compass for the given heading? There is no point in storing both the inaccurate result and the correction, we should correct the data, then store the accurate value in heading.

This same adjustment is made by various sensors internally (calibration) and needs to be applied to some data by the server, eg wind vane offset.

There was discussion on this recently (slack?) and I proposed a 'correctIncomingSensorData' functionality, but I think its an implementation choice. Not sure we can have a std format for the data we might need to store. Definitely a bad idea to start sending data plus corrections (that the receiver must apply anyway).

Easiest is to add "local adjustments to correct sensor data MUST(SHOULD?) be applied before data is updated in signalk. How this is done is implementation specific" for v1

Then we can do more after v1?

[tkurki]

Changed to after v1.

My point in this is that if we want this as part of the spec it needs to be in the data model, and I agree with Tim's point about this being part of the sensor.

Server correcting the incoming heading with this is beyond SK spec and an implementation detail.

[sumps]

Agree and I am dubious as to how many heading sensors actually output this data. They will send deviation corrected values out.

[rob42]

Probably still a good idea to be clear about it even for V1 "Data in signalk is assumed to already be corrected for any known sensor adjustment"

[sumps]

During some work on iKommunicate, it became clear that we never did support magnetic deviation in the SIgnal K V1 spec. Think at the very least we should have the following key so that we have somewhere to store NMEA0183 and NMEA2000 Deviation values...

/vessels/<RegExp>/navigation/magneticDeviation Units: rad (Radian)

Description: The magnetic deviation at the current heading that must be added to the magnetic heading to correct it for local magnetic deviation. Easterly deviations are positive and Westerly deviations are negative (in Radians).

[rob42]

"Data in signalk is assumed to already be corrected for any known sensor adjustment" .

The gateway or sensor should apply the correction before sending the data.

The actual deviation varies with heading, its traditional to have a 'card' of values around 360deg so you can adjust any given heading for the correct amount. Its much simpler to fix this at the sensor or gateway than propagating the error around the network and having every subscriber compensate.

[sumps]

I agree @rob42 and if we were only dealing with new SIgnal K sensors then this would be fine, but we have to work with legacy, current and future 3rd party NMEA0183 and NMEA2000 based products and these output Deviation and at the moment there is nowhere to put this data in Signal K, so that is why I am proposing having a key for deviation and then developers can decide how they want to use it - No Key, No Option ;-)

[tkurki]

Even if data is adjusted so that deviation is not needed a user may want to display it, so having it in the schema is useful.

@sumps I think Rob's comment was more about your description of the value: _ that must be added to the magnetic heading to correct it_. This is not in line with "all values being corrected beforehand" thinking.

Nor is local appropriate here, or talk about world magnetic model, as they relate to magnetic variation/declination - the diff between true & magnetic north in a certain location.

As I see it there are

1. "current" deviation - related to the boat's current heading and the magnetic sensor that produces it
2. Deviation table - again related to the magnetic sensor

(1) could be navigation/magneticDeviation/current and then the source is the key to associate that with the navigation/headingMagnetic that it is "current" for.

(2) could be a structure such as Tim's above, only where should that go? navigation/magneticDeviation/deviceId/ (where deviceId would be $sourceId?)

[sumps]

I don't think it is correct for developers and users to make assumptions about the data and whether it has been adjusted for variation and/or deviation.

Basically NMEA devices that transmit heading send the raw measured magnetic heading and if known the variation and deviation values for the "local" situation i.e. the local variation taken from the electronic chart or calculated from the world variation model and the local deviation table for that vessel.

Often a compass sensor will have been calibrated to remove the local deviation effects in which case its heading will include deviation and it will either output 0 deviation or "Null" data not available.

The heading, variation and deviation always have to be linked to the source of data, just like the Position, Datum, Satellite Status, etc. from a GPS but this is handled by the source metadata.

Variation can also change, albeit more slowly than deviation, so I think having some abstract concept of a current key is over complicating things.

Basically the majority of heading sensors with be communicating by NMEA and I just want a Signal K key in the schema to put this data in.

[inspirity]

I never saw a magnetic compass with the ability to add the vessels deviation to get a deviation corrected HDM as output. Since deviation is another vessel owned value over 360 degress like the polar performance.* keys. We should handle it the same way. I agree with Turki for the path navigation/magneticDeviation/current and have the deviation table for the vessel separate as the polar table would be. It is a fundamental decision having those correction/performance tables into the SK-tree or as external data sources.

[sumps]

I don't understand this concept of "current" by its very nature the deviation value will be the one applicable to the heading the compass is outputting.

You need another "static" deviation table stored in another part of the schema applicable to a particular compass sensor which an app can load at start up and apply to any values it receives from the applicable compass sensor.

As a general principle, I think we should always have a Signal K key for all NMEA data so that there is clear and accurate conversion of the data, with no further ambiguity or introduction of errors. Then if we think there is benefit in improving or expanding on that data, we extend the schema, create new derived values, implement new ways of doing things, etc.

[rob42]

Just for clarity here - I was talking about compass deviation as compass inaccuracy eg most vessels (especially steel) have local magnetic fields that make the compass read wrongly in various directions.

Then we have the external magnetic deviation due to the earths magnetic poles not being aligned with geographic poles. We have navigation.magneticVariation for that, usually derived from world magnetic model based on location.

Which magnetic deviation are we talking about here?

@inspirity true, most current compasses dont, but in signalk we should fix sensor errors at the edge of the signalk network, not propagate the error info too. Incidentally thats different from navigation.magneticVariation, which is a valid data, since its not an error.

Early on we went to SI units throughout signalk, which dropped the need to send units and greatly simplifies apps and processing since we dont do conversions (except in the gui). To me errors are the same, we should fix the data rather than propagate an error.

[inspirity]

Magentic Variation or Declination is not the same than Magnetic Deviation. Variation or Declination is due to the difference between Magnetic and geographical Northpole Deviation is the ships own error.

I need the deviation AND the variation to navigate without GPS. So if my electronic compass fails and my gps fails i could still have an app calculating my compass course to navigate to the next waypoint. Without the deviation data i simply can't. It is the same if i want know my performance without polars. Its just a guessing. I absolutely agree and support calibrated sensors and not errors. Deviation is not more error than Variation since is not sensor related but boat related. Like the polars are not errors but boat related Information.

[sumps]

This theoretically perfect world where we have Signal K sensors that output perfect data is great but we still need to work with imperfect NMEA based sensors and displays and if we wish to receive and transmit imperfect data then we need to have keys to store them in.

[rob42]

Any data can be wrong. Thats why eyeballs and brains are still the primary navigation instrument :-) But when we know errors we should correct then as near to source as possible.

That said I understand better what @inspirity is after now. If you have a compass that simply sends its heading and has no capacity for holding/correcting for deviation, then storing it and applying it elsewhere makes sense. I was thinking of @sumps example of an N2K packet that sent both heading and deviation, that should be corrected in the gateway.

There was some discussion elsewhere about applying corrections to signalk data but I cant find it. The general idea was that incoming data would be parsed for keys that were known to need correction and the corrections applied. It was a server implementation issue.

I think we need a generic way to add a correction factor into any signalk key. the meta structure seems to be the place, so meta.correction.* ?

The process would need to differentiate by source, and possibly a method to ensure it didnt correct the same value twice (eg on echo, or mirrored servers). Not simple.

[inspirity]

@rob42 i don't see deviation as an error but similar to the polars as a data table connected to the boat. Calibrations in meta.correction for a deviation table would have to be a function since the deviation depends from the headingMagnetic. We already have the headingTrue Key in the definitions wich should contain the Variation AND the deviation. The Deviation is usually a table. Question is do we store this data in SK or do we keep this external. Just to clarify what we are talking about:

[tkurki]

If i want to build a display that shows the current deviation next to my analog compass I want that in the sk data model.

If I want show a deviation graph as in the previous post I want that in the sk data model.

Neither of these have anything to do with whether or not heading* values are corrected. I think that is a separete issue.

pe 14.12.2018 klo 23.25 Olivier Hendrikx [email protected] kirjoitti:

@rob42 https://github.com/rob42 i don't see deviation as an error but similar to the polars as a data table connected to the boat. Calibrations in meta.correction for a deviation table would have to be a function since the deviation depends from the headingMagnetic. We already have the headingTrue Key in the definitions wich should contain the Variation AND the deviation. The Deviation is usually a table. Question is do we store this data in SK or do we keep this external. Just to clarify what we are talking about: [image: deviationtable] https://user-images.githubusercontent.com/6239660/50028164-d2199480-ff92-11e8-9a4f-9ab7455aa8c6.jpg

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/SignalK/specification/issues/242#issuecomment-447483740, or mute the thread https://github.com/notifications/unsubscribe-auth/ABAETnWch3-x2zecg5AnkLTTinRmmMr3ks5u5Bc3gaJpZM4JsY5A .

[inspirity]

I support tkurki's point of view.

Maybe we should just clarify in the key's Description:

• headingMagnetic: without Variation and Deviation
• headingTrue: at least with Variation correction. (headingMagnetic + magneticVariation + Deviation if available)

[sumps]

I also sense from this thread that no one knows whether any systems populate the deviation fields in NMEA 0183 and NMEA 2000 so having a key that iKommunicate, CANBoat, etc can populate will give us, for the first time, the visibility to start understanding what data is available and from which systems.

[inspirity]

@sumps Some Manufacturer let you drive slowly a circle in flat water with no current to populate their internal deviation table by storing the difference between COG and HDM + MagneticVariation.

[sumps]

The more we describe these marine values and terminologies, the more value we add to the schema and work that everyone is doing.

HeadingTrue used to be only generated by expensive gyro systems on commercial vessels and now with GPS Compass and MEMs sensors it will become a much more common data type.

In these circumstances having a Signal K Server that can efficiently and effectively manage the steering compasses deviation table is a desirable goal.

[sumps]

@inspirity yes Digital Yacht’s HSC100 is a good example and it is purely a magnetic calibration no use of COG is made.

[rob42]

Taking the data above from @inspirity we can fulfil both @tkurki 's use cases:

1. show compass heading, and current deviation (match heading against table in meta.corrections)
2. I can draw a graph from the table.

That bits easy, but there are complex issues if we dont apply corrections as early as possible: a) When the autohelm subscribes for headingMagnetic, does it expect the correction has been applied, or does it check for a correction and apply it to every value it receives? b) if the correction is optionally applied, how do you find out if you have a corrected value or not? c) if you pass the value onto another signalk device should you correct or not? d) how do you stop a correction being applied twice when it passes through a chain of servers?

That whole mess is avoided by applying any corrections as early as possible, eg at the first 'smart' signalk device, be it gateway or server. Then we have the general rule that signalk data is as accurate as possible (but no guarantee its actually accurate)

By all means we should store the correction data, but when you subscribe to a value it should already be corrected. So if you want the uncorrected data (the less likely use case), you get corrected value, and the correction data and remove it to find the raw value. Or we store the raw value in meta too?

[inspirity]

@rob42 If you have a mechanical compass on the boat you know the headingMagnetic. You can compare it with the values you get from your electronic devices and you know if and which correction is applied. I can tell on my boat since i have some with the variation correction, some without and none of them with the deviation.

[inspirity]

@rob42 I dont think it is too complicated since the path with the description is clearly defined

To your questions:

a) When the autohelm subscribes for headingMagnetic, does it expect the correction has been applied, or does it check for a correction and apply it to every value it receives?

headingMagnetic has no correction headingTrue has it. So if the autohelm wants the corrected value then it should subscribe to headingTrue. If your Autohelm shows the course on the Autohelm, you can compare it with your gps (when subscription to headingTrue) if there is a small difference depending on which course you are sailing then you dont have the deviation. If your autohelm subscribes to headingMagnetic then autohelm and compas should show the same value.

b) if the correction is optionally applied, how do you find out if you have a corrected value or not?

See my previous post

c) if you pass the value onto another signalk device should you correct or not?

headingMagnetic has no correction headingTrue should have Variation and Deviation If your source is connected to headingTrue then no further correction

d) how do you stop a correction being applied twice when it passes through a chain of servers?

There should be no correction from server to server or if there is correction then the path will change. Same path no correction.

The important step is to figure out what the source delivers.

• headingMagnetic (no correction at all? ) then path headingMagnetic
• headingMagnetic + Variation then add + deviation => to headingTrue
• headingMagnetic + Deviation then add MagneticVariation => to headingTrue
• headingMagnetic + Variation + Deviation => to headingTrue

This is only an installation challenge.

This is an important discussion while most boats i know have a mess with instruments all showing different headings while not many people really care. SK could be very helpful with clear definitions.

[rob42]

I have GPS, which shows heading true, but it becomes increasingly erratic as speed slows, since the GPS inaccuracy of +-10M min affects the heading of a boat at <2knts a lot. At anchor the GPS heading is useless.

I have an IMU, which is accurate to +-2deg, but very susceptible to any stray electric currents and ferrous/magnetic objects. So it constantly auto-calibrates, sometimes stupidly if you place the wrong object near it. Of course my dearest throws my tools onto the helm station, which is where the IMU also lives.....

I have an old school magnetic compass on the helm which was always great until I put a Samsung tablet on the helm. Tablets have powerful magnets to hold covers and keyboards, and if its close enough to read it affects the compass to some degree...The autohelm also has an internal compass, its near the helm too.

So I can actually steer my boat by waving my tablet around, or better yet arranging spanners across the nav table in holistic ways :-)

[rob42]

My view is that the navigation.headingTrue.value is the heading of the vessel relative to N-S on the chart. navigation.headingMagnetic.value is the heading of the vessel relative to magnetic north. So navigation.headingMagnetic.value + declination = navigation.headingTrue.value (or maybe -declination, I forget.)

Both should be the best values we have onboard, errors should be corrected if we know them. headingMagnetic.value is not the value on the compass (which compass?), its the magnetic heading of the vessel.

• headingMagnetic.values.myCompass.value is the value from the compass.
• headingMagnetic.values.myIMU.value is the value from the IMU.
• headingMagnetic.values.myAutohelm.value is the value from the autopilot.

Even those values should be corrected for known errors, eg deviation. This becomes even more important if you want a high reliability autonomous vessel. You read all the compasses and compare. Make sure a majority of sources agree within a certain error, and go with that. Flag the wayward sources as faulty.