obsidian-map-view icon indicating copy to clipboard operation
obsidian-map-view copied to clipboard

Ways to shift coordinates reference system only for a particular tile source? (due to GCJ-02 in China) / 瓦片图源对于国测局(火星)坐标系偏移问题进行矫正的可能性?

Open hdlnii opened this issue 9 months ago • 5 comments

Hi. First, I want to express my excitement and appreciation of this plugin! Thank you so much for making this!! I have always needed a geolocation management tool for all the places I want to go, have been, or simply know around the world. Previously, I used Apple Maps and various other services, but it was extremely scattered, and they all have proprietary format and rarely allow export of my lists. Now, as i just discovered it, it's finally possible to do it within my own local-first PKM system. This has sooo much potential!

I just thoroughly read and studied the doc and have setup my config for my vault;

But there is one major problem remained for me, and potentially for anyone in/visiting China:

Context of Problem

In China, where I live now, all web map services use the GCJ-02 coordinates reference system (CRS), as opposed to the international standard of WGS-84, which is used by OSM and virtually every and any other mainstream international GPS / mapping data providers.

In the process of learning and researching about the working mechanism of this plugin and tiled maps in general, I gathered an extensive list of interesting, function-rich, and beautiful tiled map URLs from various providers around the world, hoping to be able to use them in my own setup in Obsidian via MapView.

Among the URLs on this long list, except the very few China-based providers, all other map URLs use the WGS-84 coordinates.

The Problem

There is a pattern-based shift (spatial difference) for about a couple hundreds metres between a WGS-84 and a GCJ-02 coordinate for any location inside of mainland China.

Example to see the problem in practice

I think the reasons why Google adjusted only the non-satellite maps layers is because of the continuous nature of satellite imagery, that it is difficult to cut and offset while still maintain global / international integrity of the satellite map, especially around national borders.

  • For anyone who might be concerned, to test and see the problem first-hand in MapView in Obsidian:
    • GCJ-02 only in China, WGS elsewhere
      • Google Maps Default: https://mt.google.com/vt/lyrs=m&x={x}&y={y}&z={z}
      • Google Maps Sat. Hybrid (Mixed CRS): https://mt.google.com/vt/lyrs=y&x={x}&y={y}&z={z}
    • GCJ-02 in China, elsewhere no data
      • Autonavi/高德地图 standard: https://webrd01.is.autonavi.com/appmaptile?x={x}&y={y}&z={z}&lang=zh_cn&size=1&scale=1&style=8
      • Autonavi/高德地图 satellite: https://wprd01.is.autonavi.com/appmaptile?x={x}&y={y}&z={z}&lang=zh_cn&size=1&scl=2&style=6
    • WGS everywhere on earth
      • OSM Standard (WGS-84) (which many tiled maps services are based on): https://tile.openstreetmap.org/{z}/{x}/{y}.png
      • Google Maps Satellite Base: https://mt.google.com/vt/lyrs=s&x={x}&y={y}&z={z}

Practical Implications of the Problem

In theory, I can just use a WGS map (like the MapView's default OSM-based CartoDB Voyager) entirely inside of Obsidian, and I will be 100% unbothered by the GCJ thing.

In practice, however,

  • inputting locations to Obsidian via MapView:

    • the way to record a location in China and to put it in my system will have to involve using other tools, since the OSM geocoding has very limited data in China compared to for example Autonavi (biggest and most up-to-date provider). Also, while on a hike, marking non-POI locations in the wild, will also have to go through a GCJ-based GPS service first.
    • In short: I necessarily get coordinates in GCJ to put into Obsidian.
  • outputting locations from Obsidian via MapView:

    • To be able to use the navigation, routing, and other functions properly in a dedicated mapping app, reliance on the "Open In" feature is crucial. Since all mapping apps use GCJ-02 as per industry regulation (e.g. Apple Maps (China), Autonavi), for everything to work in the dedicated mapping app, I need to save locations in China in GCJ coordinates to then "open in" the other app.

In practice, in summary, users in China, or simply visiting China for a trip, might have to record all China locations in GCJ in Obsidian MapView, and simultaneously maintaining elsewhere locations in WGS. It will be a massively inconvenient and laborious to have to convert GCJ to WGS each time for each location.

Current Workaround and Still-Remained Problem

Thankfully MapView allows switching between multiple tile sources easily.

Currently a suboptimal way to alleviate this problem is to use Autonavi's tile URL (GCJ-02) while looking at locations in China, and switch to any other tile providers' URL (all WGS-84) to look at places around the world.

Using the system this way means I maintain 2 set of locations

  1. China locations, all saved in GCJ
  2. elsewhere around the global, all saved in WGS

This means that none of the very useful and interesting tile maps URL I collected are actually usable for locations in China, except Autonavi and Google Maps' default

Questions

Is there a way to shift a certain WGS tile source in MapView to make it accurately reflect GCJ-02 coordinates? If this is doable, then I can create two tiles sources using the same URL, practically and entirely eliminating this problem in use (yay!).

Since Google could change it in their default map, can this be changed in Leaflet.js in MapView? or on the plugin level? or by other technical means?

I am unskilled in coding and script writing, but I am willing to learn and try some stuff out to solve a problem... with some directions and using the help of ChatGPT.


Big thank you again to the developer, any input from anyone is appreciated!

yubo

hdlnii avatar Mar 29 '25 00:03 hdlnii

I'm aware of the issue, and I worked in the past on getting Chinese projections supported, but was unable to get it to work easily :( Since then I became aware of projects that make it easier, e.g. https://github.com/htoooth/Leaflet.ChineseTmsProviders It should certainly be integrated, and sooner rather than later, but I really wish it could be done by someone who's actually familiar with the Chinese mapping constructs, rather than me who has very little clue if what I'll do makes sense, and unable to read the documentation of some of these sources...

esm7 avatar Apr 09 '25 05:04 esm7

upon a little further digging, i came to think that this could be particularly technically difficult.

i realised that the GCJ seems to not be a simple uniform shifting from the WGS. different locations are shifted by different distances. there seems to be a mathematical model behind it, possibly encrypted.

i added the Chinese title to this thread so should a Chinese-speaking user who is technically familiar with this matter find it in the future, they may then further this Issue toward a possible resolution.

for now for me and other users facing this problem, the best approach i think is just to work with switching tile sources when viewing inside/outside of mainland China.

hdlnii avatar Apr 11 '25 14:04 hdlnii

Unfortunately, the official way to do WGS to GCJ conversion seems to be unidirectional, via api from Chinese map providers. There are some reverse engineering online, but I don't think they are reliable. I guess that's the whole point why you want a dedicated coordinate system for security, or it's useless. The best solution I can come up with is having a field called "isGCJ" or something like that in the frontmatter and let the mapview switch among sources based on the field (at the boarder, we may still need switch it manually. Even googel map can't handle it properly if you take at look at the border at Macau). btw, I am using iphone to record my trips in China, the raw data points are actually in WGS, and can be used on OSM. So at least it's not system level, but depending on the app you are using.

XU-Nuo avatar Apr 27 '25 02:04 XU-Nuo

let the mapview switch among sources based on the field

could you elaborate? i don't get how your workaround works.

i don't have much locations with their own notes, i primarily use in-line locations, for now. does it apply?

btw, I am using iphone to record my trips in China, the raw data points are actually in WGS, and can be used on OSM. So at least it's not system level, but depending on the app you are using.

yes it depends on the GPS software. but when you're in China, what WGS-based software can be used normally with all of its functionality? most if not all functional apps are GCJ-based and China-made, no?

There are some reverse engineering online, but I don't think they are reliable. I guess that's the whole point why you want a dedicated coordinate system for security, or it's useless.

there is one bi-directional CRS conversion tool i found a long time ago, which i tested, and it seems to be very accurate: https://www.lddgo.net/en/convert/coordinate

honestly, as a Chinese citizen, i think it is outrageously useless: it does nothing meaningful security-wise, but only introduces massive inconvenience to citizens of the country and of the world who wish to travel.

if we're dealing with foreign military or spying operations, or domestic terrorism, do you think such minor GPS shift of a couple hundred meters, which, can be easily reversed by some random tech guy online, will be an uncrossable obstacle to foreign governments and terrorists, who have all the resources and technologies? and will make them be like, well, we might as well give up, it's simply too well protected, it's almost like the entire China is taken off of the world map!! — lol, no, i don't think so

it's a very restricted and repressed system. and this particular aspect of it, is stupid.

hdlnii avatar Apr 28 '25 13:04 hdlnii

could you elaborate? i don't get how your workaround works. i don't have much locations with their own notes, i primarily use in-line locations, for now. does it apply?

  1. count the number of notes with GCJ notes in the current viewpoint vs WGS ones
  2. if we have more GCJ notes, use the base map from GCJ source.

Another quicker but dirty way without adding a flag to notes (and can be potentially applied to inline locations) is just setting a rough geofence covering all GCJ areas, and use it to filter and index GCJ notes, just like how pokemon go does it.

Yes, this only applies to well-formatted notes, which is my use case. Inline locations can be tricky, but if the collection and use of coordinates links are both in GCJ(as you mentioned, for software reasons), there is no need to convert the coordinates, except for displaying them on base map, which as I said, switching the base map is more convenient. Also, most Chinese apps do not have proper geo URI scheme support, every app tends to have its own URI scheme or just uses regular URLs.

there is one bi-directional CRS conversion tool i found a long time ago, which i tested, and it seems to be very accurate: https://www.lddgo.net/en/convert/coordinate

That's the reverse engineering I meant, along with the accuracy issue and possible changes of the black box GCJ algorithm, that's NOT the way we should expect in an open-source JS plugin, neither converting thousands of coordinates on the fly nor sending data to a random API owned by an individual.

And calm down, there are no politics here, asymmetric encryption is just the best practice for anyone to build a modern security system, no matter if you should build one in the first place, but if you do for whatever reason, you better use it.

XU-Nuo avatar Apr 28 '25 15:04 XU-Nuo