Support for larger tiles

[rinigus]

I am developing a simple server that can be used as a drop-in replacement for online map services: https://github.com/rinigus/osmscout-server . The integration with modRana was very simple (see modRana under thirdparty subfolder). However, the generation of tiles would be faster if I could generate them not as 256x256, but larger (512x512). Maybe modRana supports it, but I couldn't find the specific flag for it (in poor maps it was scale, for example). Is it possible to use larger sized tiles in modRana?

[M4rtinK]

I am developing a simple server that can be used as a drop-in replacement for online map services: https://github.com/rinigus/osmscout-server . The integration with modRana was very simple (see modRana under thirdparty subfolder).

Nice! :)

However, the generation of tiles would be faster if I could generate them not as 256x256, but larger (512x512). Maybe modRana supports it, but I couldn't find the specific flag for it (in poor maps it was scale, for example). Is it possible to use larger sized tiles in modRana?

ModRana already has some support for this - it was originally meant for simple tile upscaling to make features on the tiles bigger (but blurry), so it's not totally hardwired to tiles being always 256x256 pixels.

But I'll have to check in what state this is as there have been some reports about the scaling being broken in recent modRana versions.

Also there is the issues of using tiles of different sizes together - modRana supports tile overlay, so it will have to handle some tiles being 256x256 and some 512x512, but that should be doable with some scaling. Also IIRC some map tile providers have started to provide "retina" tiles that are also 512x512 big, so it should less of an issue in the future & it's a further incentive to support bigger tile sizes. :)

[rinigus]

Excellent! I did have problems with using this zoom feature - it did not work for me either (each tile is surrounded by gray area).

I am working on getting search and routing exposed as well. As I could see, these functions are not as accessible as tiles in modRana. So, I'll test them on poor-maps first, where corresponding modules seem to be exposed. But I am sure if it would work in one application, it would be easy to extend the other one as well. The main part for me is to get server providing data - you guys have worked a lot on polishing user interface :)

[M4rtinK]

Excellent! I did have problems with using this zoom feature - it did not work for me either (each tile is surrounded by gray area).

Yeah - that's IIRC how the bug manifests. I'm currently kinda rewriting the tile display code in the modRana Qt5 GUI (the master-better_tile_display branch) so I'll make sure tile scaling works with the new code when I'm at it. :)

I am working on getting search and routing exposed as well.

That's a really good news! There have been many standalone navigation systems with good search and routing support over the years, but very few have ever provided integration points for other applications to make use of the functionality. :)

As I could see, these functions are not as accessible as tiles in modRana.

The issue is IMHO that while the web mercator projected tile system kinda became the standard for how mapping apps handle tile storage and display, there is really no such standard for routing and search queries, just a multitude of various mostly online services, each with a different API.

So modRana handles different search and routing providers by having a search and routing provider base class which is then sub-classed for the given provider to "talk" to its API while still providing a unified search/routing provider interface specified by the base class.

For example this is how it looks like for the various online POI search providers: https://github.com/M4rtinK/modrana/blob/master/modules/mod_onlineServices/online_providers.py Or for offline routing providers: https://github.com/M4rtinK/modrana/blob/master/core/routing_providers.py

So, I'll test them on poor-maps first, where corresponding modules seem to be exposed. But I am sure if it would work in one application, it would be easy to extend the other one as well.

Do you have some (even rough/preliminary) description/specifications of the search & routing interface ? I could use that to add the libosmscout search/routing providers to modRana. Some way to test if I have implemented the interface correctly would also be nice. :)

The main part for me is to get server providing data - you guys have worked a lot on polishing user interface :)

Thanks - that's indeed also my idea how it should be done - it's nice to have a standalone navigation application, but even better if it's functionality can be used by other software. :)

BTW, modRana also kinda exports some of it's functionality via a CLI interface: http://modrana.org/trac/wiki/SystemFunctionalityEN#CommandLineOptions It's rather unfinished at the moment but I would like to eventually make it cover all important areas of the functionality modRana provides. :)

[rinigus]

Yeah - that's IIRC how the bug manifests. I'm currently kinda rewriting the tile display code in the modRana Qt5 GUI (the master-better_tile_display branch) so I'll make sure tile scaling works with the new code when I'm at it. :)

Good to hear. I hope you do it using QML - that would allow you to share the code between platforms, right?

I am working on getting search and routing exposed as well.

That's a really good news! There have been many standalone navigation systems with good search and routing support over the years, but very few have ever provided integration points for other applications to make use of the functionality. :)

This is an idea and would allow us to pull the pieces together.

As I could see, these functions are not as accessible as tiles in modRana.

The issue is IMHO that while the web mercator projected tile system kinda became the standard for how mapping apps handle tile storage and display, there is really no such standard for routing and search queries, just a multitude of various mostly online services, each with a different API.

So modRana handles different search and routing providers by having a search and routing provider base class which is then sub-classed for the given provider to "talk" to its API while still providing a unified search/routing provider interface specified by the base class.

For example this is how it looks like for the various online POI search providers: https://github.com/M4rtinK/modrana/blob/master/modules/ mod_onlineServices/online_providers.py Or for offline routing providers: https://github.com/M4rtinK/modrana/blob/master/core/routing_providers.py

I see. I can look closer into it later when the first full version is out. But, I hope that you would beat me to that and add the interface earlier :)

So, I'll test them on poor-maps first, where corresponding modules seem to be exposed. But I am sure if it would work in one application, it would be easy to extend the other one as well.

Do you have some (even rough/preliminary) description/specifications of the search & routing interface ? I could use that to add the libosmscout search/routing providers to modRana. Some way to test if I have implemented the interface correctly would also be nice. :)

Sure! My plan is as follows:

• polish a code a bit further
• have location search (lookup for an object using address parts), done but requires a bit polishing
• have free text search (like the name of cafe, for example); partially done, have to add address to results
• have POI within specified region. should not be too complicated since there are demos covering it, but this hasn't yet started

When these are done, I was planing to make the first release which is mainly targeted for map app developers.

If you wish, you could clone my repo ( https://github.com/rinigus/osmscout-server) already and compile it under Linux, for example. Use osmscout-server_console.pro as a project in Linux PC and you should be all set. Repo README also would help you started with URL API.

The main sticking point is the dependencies. You would have to install MARISA (its available as package on gentoo, don't know about others) and libosmscout. For MARISA, I wrote Sailfish packaging scripts at https://github.com/rinigus/pkg-marisa . For libosmscout, you should be able to use the upstream version. Sailfish packaging is in upstream already.

Next, you would have to get the map and convert it to libosmscout format. The procedure is given at http://libosmscout.sourceforge.net/tutorials/Importing/ and basically requires that you have libosmscout on your PC.

The main part for me is to get server providing data - you guys have worked a lot on polishing user interface :)

Thanks - that's indeed also my idea how it should be done - it's nice to have a standalone navigation application, but even better if it's functionality can be used by other software. :)

Yep, I remember you voiced it on TJO and I've got inspiration from that and my other project :)

BTW, modRana also kinda exports some of it's functionality via a CLI interface: http://modrana.org/trac/wiki/SystemFunctionalityEN#CommandLineOptions It's rather unfinished at the moment but I would like to eventually make it cover all important areas of the functionality modRana provides. :)

CLI is nice! I have been using something like it in my SystemDataScope to communicate with rrdtool. In that project, I use an interface of rrdtool that is available when you start it with '-' option. Namely, it keeps running and waits for command line options in stdin. When the command is received, it runs it, prints out results (if any) and waits for a new command. That way you could start this tool and close only when you don't need it anymore. This would prevent many start/stop cycles.

cheers,

rinigus

[rinigus]

One small additional comment: As you could see, we could change the URL APIs in future by adding other versions. That should allow us to expose more of libosmscout functionality for client programs, when needed. In respect to search, I probably should implement better the queries with options...

[M4rtinK]

Good to hear. I hope you do it using QML - that would allow you to share the code between platforms, right?

Looking at the README file you mention further below it's basically a REST API with JSON used for output, right ? In such case I will most probably implemented it in the Python part of the code and expose it to the GUIs as needed (modRana actually also has a GTK2 UI but the Qt5 GUI gets the most development these days) - it is in principle quite similar to how the existing online search providers are done, just running over localhost, not the Internet. :)

Sure! My plan is as follows:

• polish a code a bit further
• have location search (lookup for an object using address parts), done but requires a bit polishing
• have free text search (like the name of cafe, for example); partially done, have to add address to results
• have POI within specified region. should not be too complicated since there are demos covering it, but this hasn't yet started

When these are done, I was planing to make the first release which is mainly targeted for map app developers.

Sounds like a good & sensible plan. :)

If you wish, you could clone my repo ( https://github.com/rinigus/osmscout-server) already and compile it under Linux, for example. Use osmscout-server_console.pro as a project in Linux PC and you should be all set. Repo README also would help you started with URL API.

The main sticking point is the dependencies. You would have to install MARISA (its available as package on gentoo, don't know about others) and libosmscout. For MARISA, I wrote Sailfish packaging scripts at https://github.com/rinigus/pkg-marisa . For libosmscout, you should be able to use the upstream version. Sailfish packaging is in upstream already.

I'm running Fedora and it looks like Marisa is already packaged: :) https://admin.fedoraproject.org/pkgdb/package/rpms/marisa/

Next, you would have to get the map and convert it to libosmscout format. The procedure is given at http://libosmscout.sourceforge.net/tutorials/Importing/ and basically requires that you have libosmscout on your PC.

Yep, I remember you voiced it on TJO and I've got inspiration from that and my other project :)

Wow, it's quite an honor to be an inspiration for such a nice project! :)

CLI is nice! I have been using something like it in my SystemDataScope to communicate with rrdtool. In that project, I use an interface of rrdtool that is available when you start it with '-' option. Namely, it keeps running and waits for command line options in stdin. When the command is received, it runs it, prints out results (if any) and waits for a new command. That way you could start this tool and close only when you don't need it anymore. This would prevent many start/stop cycles.

Interesting! :) I think I haven't really used rrdtool directly, but I have been using Smokeping, which IIRC internally uses rrdtool.

[M4rtinK]

One small additional comment: As you could see, we could change the URL APIs in future by adding other versions.

Yeah, that should not really be a problem for modRana - it's certainly a good idea to have versioning built-in in the URL and it should be fine if the old version is still available (at least for some time) when a new version is introduced so that modRana has some time to migrate to the new API version & to account for situations where people update osmscout before updating modRana. :)

And of course I also don't mind the API breaking at the moment due to development. :)

That should allow us to expose more of libosmscout functionality for client programs, when needed. In respect to search, I probably should implement better the queries with options...

Maybe it could work with JSON also for query input ?

something like:

{
"name" : "The Green Cat",
"bbox" : [[12.34, 56.78], [12.34, 56.78], [12.34, 56.78], [[12.34, 56.78]],
"type" : "restaurant",
"type_options" : {
   "cousine" : ["czech", "italian"],
    "open" : true
}
}

Or for routing:

{
"waypoints" : [[49.2,16.616667], [48.946667, 16.316389], [50.083333,14.416667],
"highways" : true
"toll_roads" : false
}

With JSON you could quite easily add more options (by the server learning to understanding more keys) or even whole new subtrees without the need to bump the API version. You could even remove less critical fields if really needed - they would just be ignored during the search.

[rinigus]

Looking at the README file you mention further below it's basically a REST API with JSON used for output, right ? In such case I will most probably implemented it in the Python part of the code and expose it to the GUIs as needed (modRana actually also has a GTK2 UI but the Qt5 GUI gets the most development these days) - it is in principle quite similar to how the existing online search providers are done, just running over localhost, not the Internet. :)

That was an intention. BTW, I use modRana on desktop to test the server. There is a limitation of the server that I run into - I needed to reduce number of connections by modRana in your code. The server has each connection in separate thread and there is a limit imposed in configuration (~4 connections). But I'll file it as a separate issue.

Yep, I remember you voiced it on TJO and I've got inspiration from that and

my other project :)

Wow, it's quite an honor to be an inspiration for such a nice project! :)

:) [and if you are collecting acknowledgments, here is one more to your universal components: https://github.com/rinigus/systemdatascope / same in the app About page :) ] Thank you!

rinigus

[rinigus]

Having JSON as a query object is a bit trickier. So far, I limited API to URL parsing and was not reading in any data sent from the client. We can surely extend URL parsing and I plan to do it for POI search using QUrl and QUrlQuery - that would be simple. Let's see if we need JSON when I cover the basic functionality in v1 :) . As I could see, the main advantage in JSON would be in providing hierarchy...

rinigus

On Wed, Oct 5, 2016 at 3:13 AM, Martin Kolman [email protected] wrote:

One small additional comment: As you could see, we could change the URL APIs in future by adding other versions.

Yeah, that should not really be a problem for modRana - it's certainly a good idea to have versioning built-in in the URL and it should be fine as either the old version is still available (at least for some time) when a new version is introduced so that modRana has some time to migrate to the new API version & to account for situations where people update osmscout before updating modRana. :)

And of course I also don't mind the API breaking at the moment due to development. :)

That should allow us to expose more of libosmscout functionality for client programs, when needed. In respect to search, I probably should implement better the queries with options...

Maybe it could work with JSON also for query input ?

something like:

{ "name" : "The Green Cat", "bbox" : [[12.34, 56.78], [12.34, 56.78], [12.34, 56.78], [[12.34, 56.78]], "type" : "restaurant", "type_options" : { "cousine" : ["czech", "italian"], "open" : true } }

Or for routing:

{ "waypoints" : [[49.2,16.616667], [48.946667, 16.316389], [50.083333,14.416667], "highways" : true "toll_roads" : false }

With JSON you could quite easily add more options (by the server learning to understanding more keys) or even whole new subtrees without the need to bump the API version. You could even remove less critical fields if really needed - they would just be ignored during the search.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/M4rtinK/modrana/issues/144#issuecomment-251549413, or mute the thread https://github.com/notifications/unsubscribe-auth/AO-p7TngZ2HIpm-_KHyav2FSnlKuJFeAks5qwuuUgaJpZM4KGO0A .

[M4rtinK]

:) [and if you are collecting acknowledgments, here is one more to your universal components: https://github.com/rinigus/systemdatascope / same in the app About page :) ] Thank you!

Cool! Can I add you to the list of users of Universal Components ? :)

Also don't hesitate to let me know if you hit some issues with UC/have some suggestions how it could be improved. :) And patches are of course also very welcome. ;-)

Having JSON as a query object is a bit trickier. So far, I limited API to URL parsing and was not reading in any data sent from the client. We can surely extend URL parsing and I plan to do it for POI search using QUrl and QUrlQuery - that would be simple.

Yeah, I URL encoded parameters should be certainly good enough for now. :)

[rinigus]

Its not exactly UC that I ended up using. I took the parts of it and slightly modified here and there to divide things between the platforms (see https://github.com/rinigus/systemdatascope/tree/master/qml). I guess we could say that its based on UC approach. I just don't have import UC anywhere :) . But I would be happy to be listed with systemdatamonitor over there ...

The URL scheme uses parameters now (see new README) with the updated configuration for modRana

cheers,

rinigus

On Tue, Oct 11, 2016 at 2:24 AM, Martin Kolman [email protected] wrote:

:) [and if you are collecting acknowledgments, here is one more to your universal components: https://github.com/rinigus/systemdatascope / same in the app About page :) ] Thank you!

Cool! Can I add you to the list of users https://github.com/M4rtinK/universal-components#applications-using-universal-components of Universal Components ? :)

Also don't hesitate to let me know if you hit some issues with UC/have some suggestions how it could be improved. :) And patches are of course also very welcome. ;-)

Having JSON as a query object is a bit trickier. So far, I limited API to URL parsing and was not reading in any data sent from the client. We can surely extend URL parsing and I plan to do it for POI search using QUrl and QUrlQuery - that would be simple.

Yeah, I URL encoded parameters should be certainly good enough for now. :)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/M4rtinK/modrana/issues/144#issuecomment-252772136, or mute the thread https://github.com/notifications/unsubscribe-auth/AO-p7W91DNRrulobO1Kvob8dEZ1Cj4cIks5qyskfgaJpZM4KGO0A .