EVMap RFC: Basic NOBIL implementation

This adds a NOBIL data source. Data is available in Denmark, Finland, Iceland, Norway and Sweden and the data is collected both from operators via OCPI and contributed by the public.

This is a draft implementation where some functionality is missing. I'm interested in hearing if this is of interest, and if so, what should I fix/work on going forward?

Known problems / missing features:

Not all charger data is exposed in the app (location (near restaurant/shopping/...), payment methods, ...)
No filters are implemented
Chargeprice connection missing
Does Android Auto work? Not yet tested
When a charging point is removed in nobil it doesn't disappear in the UI (a cache clear is needed). Maybe a generic evmap bug?
The code could use some cleanup from someone with Android development experience

Nov 06 '24 21:11 robho

Very nice, thanks a lot! I'll try to find some time in the weekend to try it out and provide some help with code cleanup.

Not all charger data is exposed in the app (photo, location (near restaurant/shopping/...), payment methods, ...)

Yeah, if there are properties that don't fit into EVMap's data structure we can think about adding new fields or expanding existing ones where it is useful. For payment methods, we have the chargecards structure (used by GoingElectric, but a bit more complex than necessary for what NOBIL provides) or we could just put it into the cost.description for now). For location, the most similar field in GoingElectric would be "category", but that is not exposed through their API, so I haven't added it so far. amenities might also work, though that is usually a slightly longer text description in GoingElectric.

No filters are implemented

As previously mentioned, I guess all the filters would have to be implemented locally, as the Nobil API doesn't really have server-side filtering abilities, right? That should be possible (just like the min_connectors filter for GE and OCM), but in the longer term the better solution is probably to first download all chargers into the local DB (which is a capability I'm implementing for #290) and perform filtering there.

Chargeprice connection missing

Chargeprice currently only support GoingElectric and OpenChargeMap unfortunately. And by now they have also created their own separate database of charging stations aggregated from different sources, so from what I heard they are not so interested in fixing issues with their adapters for GE and OCM, let alone adding new ones 🫤. Also, there is #320 - so I am not sure how long the native Chargeprice integration in EVMap will stay or whether I will have to try to build an alternative at some point.

Does Android Auto work? Not yet tested

Likely yes, I can test that as well.

When a charging point is removed in nobil it doesn't disappear in the UI (a cache clear is needed). Maybe a generic evmap bug?

True, that is probably a general bug - for GoingElectric it's not so relevant as we are only allowed to cache the data for 24h anyway. But yeah, it probably also applies to OpenChargeMap. Let's track that in a separate issue.

Nov 07 '24 22:11 johan12345

Regarding filtering..

in the longer term the better solution is probably to first download all chargers into the local DB (which is a capability I'm implementing for https://github.com/ev-map/EVMap/pull/290) and perform filtering there.

Is your idea to replace the on-demand-loading with a full data fetch or are you thinking of having some combination of the two?

The nobil data is ~130 Mbytes today (up ~6 Mbytes in ~2 months). This is uncompressed data, I haven't checked if the server can compresss data and what the compressed size is. I think I would prefer to use on-demand loading most of the time, but it would be valuable to have the possibility to cache all data when needed (no mobile connection or high mobile roaming prices, ...).

Is the problem that you see with local filters that they need to fetch data for chargers that are uninteresting? Ie unnecessary network traffic and data processing? Are those "unnecessary" downloads cached even if they are not used?

Nov 09 '24 21:11 robho

The nobil data is ~130 Mbytes today (up ~6 Mbytes in ~2 months). This is uncompressed data, I haven't checked if the server can compresss data and what the compressed size is.

Gzip usually helps quite a lot with JSON data, and the Nobil server does seem to support it. According to

curl --compressed -so /dev/null "https://www.nobil.no/api/server/datadump.php?apikey=<key>&fromdate=2005-01-01&format=json" -w '%{size_download}'

the download size is pretty small at just ~5 MB. Of course it might occupy a bit more space in the local database (and from my experience with the implementation for OSM, parsing the JSON and inserting chargers into the DB also takes much more time than the download itself).

Is your idea to replace the on-demand-loading with a full data fetch or are you thinking of having some combination of the two?

In general, EVMap will continue to support both. So we could offer both options for Nobil. On the other hand, with such a small download size, there's not much of a downside to storing it all locally - might also reduce the load on Nobil's servers in the long term.

Is the problem that you see with local filters that they need to fetch data for chargers that are uninteresting? Ie unnecessary network traffic and data processing?

Yeah. I think the data processing (JSON parsing into Kotlin objects, then filtering) is usually the main bottleneck when loading a large map region with thousands or even ten-thousands of chargers - server-side filtering and even server-side clustering makes it a lot faster for GoingElectric where that is supported.

Also, depending on what filters we implement, there might be some filters where we can only determine the available options by iterating through the whole dataset once (e.g., the network filter mentioned in the comments) and then storing that as ReferenceData.

Are those "unnecessary" downloads cached even if they are not used?

not at the moment, because local filtering happens inside the API implementation

Nov 12 '24 23:11 johan12345

I rebased to latest master.

Some comments on this implementation and nobil..

In Sweden the quality of nobil has increased quite much over the past 6 months. Lots of duplicates and invalid chargers have been removed and new OCPI chargers are added daily. Today there are 5485 "OCPI chargers" and 1437 manually added chargers.
Realtime data from enbw seems to work well with nobil and coverage is good. Getting realtime from nobil would add some additional realtime data though.
The speed of the nobil implementation isn't that great (I've only run debug builds with LeakCanary active though).
- The web version is faster. For every server request that the web version sends it tells the server which chargers it already knows about (url parameter excludeIds) -> less network traffic and less data to parse.
- EVMap caches query results, but only if the result is complete. Nobil can at most return 2000 results in a query response. If we're at a zoom level where we hit that limit (there are ~2000 chargers in Stockholm area) results aren't cached and there will be lots of queries as the user browses the map.
Would it make sense to look into adding support for excludeIds url parameter? Or, would it be better to change the implementation to download a full nobil dump and work with that rather than doing on-demand server queries? (I started looking into adding support for excludeIds url parameter, but it got a bit complicated with the merging of database and network chargelocations in EVMap so I never finished it, but I may return to look at it if it would be of value..)

Mar 31 '25 20:03 robho

Hi, sorry for the late reply!

I guess the full download option is a good idea and maybe easier to implement than the excludeIds parameter. As mentioned, I am building the same for the OpenStreetMap data source in #290 - in principle it works, the main thing I'm still working on (when I have time) is to speed up marker clustering at low zoom levels by using Spatialite queries for that instead of loading everything into memory first.
Regarding the Nobil realtime data, I started building a backend based on your prototype webserver, haven't yet deployed it to a server though. In the longterm I might want to extend it to aggregate data from different sources (both static information and realtime data) - especially now that more and more CPOs are starting to provide open data as EU regulations require them to do so.

May 17 '25 18:05 johan12345

I made an attempt to implement "full download" for nobil. It seems to work well. Great work on the OSM implementation, which I've stolen lots of code and inspiration from! :-)

Jun 10 '25 21:06 robho

I just added a missing new column (coordinatesProjected) in DB migration 26, and simplified migration 27 to simply use ALTER TABLE instead of recreating the table. Also, I fixed the CI build (dummy entry for the Nobil API key was missing in _ci/apikeys_ci.xml).

There still seems to be some issue with the DB migration - if I have a pre-existing installation from the master branch with already cached chargers (e.g. from OpenStreetMap) and then switch to the nobil branch and the Nobil data source, spatial index queries are not working correctly so that only few or no chargers appear on the map. Deleting the app data to start with a fresh database fixes it. I guess we'll have to delete and recreate the spatial index in migration 26 where the table gets recreated?

Sep 14 '25 20:09 johan12345

I just added a missing new column (coordinatesProjected) in DB migration 26, and simplified migration 27 to simply use ALTER TABLE instead of recreating the table. Also, I fixed the CI build (dummy entry for the Nobil API key was missing in _ci/apikeys_ci.xml).

Thanks! :+1:

There still seems to be some issue with the DB migration ...

Oh! I hadn't seen this since I haven't been running the db migration code for a long time, but I just tried the steps you provided and I see the problem. I'll see if I can figure something out, but don't hold your breath :-)

Edit: Maybe spatialite functions such as CloneTable(), DropTable(), RenameTable() may help?

Sep 15 '25 19:09 robho

Ah, found it: The issue was probably a combination of what you mentioned (using DropGeoTable() to drop the old SpatialIndex correctly) and also having to explicitly specify AND f_geometry_column = 'coordinates' in any SpatialIndex queries to make sure we are querying the correct column (seems like after the migration it was querying the coordinatesProjected column).

Sep 20 '25 16:09 johan12345

...and I also added German translations and implemented a solution to remove deleted chargers from the DB, at least for when fullDownload is used.

It seems to be working very well now, at least as a first MVP 🙂. @robho Do you think anything else is missing before this can be merged?

Sep 20 '25 17:09 johan12345

Great Johan! Thanks for taking a look at and fixing the migration problem! :+1:

Yes, I think this is in a good enough shape to be merged. There are a few things that I wanted to take a closer look at or get your opinion on, but that could be handled later, if you're fine with that. It's my intention to continue improving the nobil implementation, although I don't have much time to spare.

The TODOs on my list are:

The Tesla availability code for nobil isn't great, it has lots of duplication. I wanted to see if it could be merged in some way with the generic code (now there's a if (nobil) { branch). I also noticed that Tesla chargers have different connector configuration in nobil for different countries and wanted to look into this and work with the nobil people to get rid of the differences, if that's ok with them. The inconsistent connector handling for Tesla chargers in nobil also means that not all Tesla stations get real-time data (I guess, I haven't checked).
I added payment methods to the Cost description (your suggestion a long time ago), but it's all in English right now. I started looking into translating the strings but couldn't translate strings in NobilModel (a design decision I think), instead it looks like translations are in the Cost object itself. I guess I could push the payment methods to the Cost object and have it build the description, but I got the feeling that it would get a bit messy (but I never finished it).
The verified trait for nobil chargers doesn't match the documentation (FAQ) since I set verified if the charger is received through OCPI or has been manually updated recently (6 months). My idea was that this would in some way signal that the charger data is reliable.

Any comments on the bullets above would be appreciated.

Sep 20 '25 20:09 robho

Yes, we can handle those things later 🙂

The Tesla availability code for nobil isn't great, it has lots of duplication. I wanted to see if it could be merged in some way with the generic code (now there's a if (nobil) { branch). I also noticed that Tesla chargers have different connector configuration in nobil for different countries and wanted to look into this and work with the nobil people to get rid of the differences, if that's ok with them. The inconsistent connector handling for Tesla chargers in nobil also means that not all Tesla stations get real-time data (I guess, I haven't checked).

Yeah, would be great if this could be made consistent on the Nobil side.

GoingElectric does duplicate connectors on the V2 Superchargers, but OCM is also not always consistent there I think. So maybe we should even adjust the existing logic to be able to handle both cases. The most complex case is handling sites that have chargers with different powers (i.e., V2 and V3), because Tesla don't seem to provide the power separately for each stall in their API... In the end it's also more of a temporary problem as more and more sites are being upgraded to V3/V4.

I added payment methods to the Cost description (your suggestion a long time ago), but it's all in English right now. I started looking into translating the strings but couldn't translate strings in NobilModel (a design decision I think), instead it looks like translations are in the Cost object itself. I guess I could push the payment methods to the Cost object and have it build the description, but I got the feeling that it would get a bit messy (but I never finished it).

Yeah, translation of the charger information at the moment is only partially supported - none of the data sources provides multilingual strings, so it was not really a priority. It is a design decision to have the API & model implementations not directly access Android APIs (such as Context.getString) so that they can be run independently of the Android system (e.g., for unit tests). I have added a StringProvider interface (which on Android simply wraps getString), but this is only used in getFilters for now.

For chargers we would have to design a different solution, as they get stored in the database and we don't want to have to rebuild the database if the language changes. For example, we could use some kind of templating syntax where we can refer to translations from strings.xml (which would even allow for adding new translations without rebuilding the database), or we store strings in all supported languages into the database (which would be needed if there were a data source providing multilingual strings).

The verified trait for nobil chargers doesn't match the documentation (FAQ) since I set verified if the charger is received through OCPI or has been manually updated recently (6 months). My idea was that this would in some way signal that the charger data is reliable.

Okay, that makes sense. I can adjust this in the FAQ.

Sep 21 '25 12:09 johan12345