photon icon indicating copy to clipboard operation
photon copied to clipboard

Support for custom language sets

Open deicidemilan opened this issue 4 years ago • 12 comments

Hello,

Is there a way to get more details from API as namedetails? For example i need Belgrade on Serbian language (cyrilic).

http://photon.komoot.de/api/?q=beograd&limit=5&osm_tag=place:village&osm_tag=place:town&osm_tag=place:city

This returns only one name.

Nominatim have it, but it's not Autocomplete, Partial search or Wildcard searches.

https://search.meetserbia.rs/search.php?city=beograd&format=jsonv2&namedetails=1&type=administrative

I Install both on my server so I can change code or conf if it posible to get this.

deicidemilan avatar Sep 16 '20 09:09 deicidemilan

I install both nominatim and photon. Import nominatim database into photon, and everiting works perfect. But I need all name spacec as in nominatim database. I have only one country imported.

"namedetails": { "name": "Стара Пазова", "name:de": "Alt-Pasua", "name:en": "Stara Pazova", "name:hu": "Ópazova", "name:ru": "Стара-Пазова", "name:sr": "Стара Пазова", "name:sr-Latn": "Stara Pazova" }

If It's a way to get just name:sr and name:en will be perfect. Is this posible or i need to chenge source code? If I need to change where is file in source code to add this from nominatim database? I can edit anything because all code is on my server, but no idea what :(

deicidemilan avatar Sep 16 '20 11:09 deicidemilan

es/mappings.json lists the 5 languages imported by default. I think you just need to add 'sr' (and 'sr-Latn') there and reimport.

mtmail avatar Sep 16 '20 14:09 mtmail

I will try and give you feedback.

deicidemilan avatar Sep 17 '20 07:09 deicidemilan

I was explore importing and how photon works. I import all supported language and only one to see how it works. So this is definitly what I need. Now I need to edit something from source to see if it's work. Problem is when I change es/mappings.json how to build photon-0.3.4.jar. Because es/mappings.json is in source and i don't know how to build photon-0.3.4.jar. I need additional language only for "name" so if this is a good way for this or I need to add everywhere 'rs'?

{ "name": { "properties": { "alt": { "type": "text", "index": false, "fields": { "raw": { "type": "text", "analyzer": "index_raw" } }, "copy_to": [ "collector.default" ] }, "de": { "type": "text", "index": false, "fields": { "ngrams": { "type": "text", "analyzer": "index_ngram", "search_analyzer": "search_ngram" }, "raw": { "type": "text", "analyzer": "index_raw" } }, "copy_to": [ "collector.de" ] }, "default": { "type": "text", "index": false, "copy_to": [ "collector.default", "name.sr", "name.en", "name.de", "name.fr", "name.it" ] }, "en": { "type": "text", "index": false, "fields": { "ngrams": { "type": "text", "analyzer": "index_ngram", "search_analyzer": "search_ngram" }, "raw": { "type": "text", "analyzer": "index_raw" } }, "copy_to": [ "collector.en" ] }, "sr": { "type": "text", "index": false, "fields": { "ngrams": { "type": "text", "analyzer": "index_ngram", "search_analyzer": "search_ngram" }, "raw": { "type": "text", "analyzer": "index_raw" } }, "copy_to": [ "collector.sr" ] } } } }

deicidemilan avatar Sep 17 '20 08:09 deicidemilan

I found this too in source code @Parameter(names = "-languages", description = "languages nominatim importer should import and use at run-time, comma separated (default is 'en,fr,de,it')") private String languages = "en,fr,de,it"; In file CommandLineArgs.java I think here needs to add 'sr' language too. Can you help me where to add 'sr' and build photon-0.3.4.jar again.

"Updated"

I was add 'sr' everywhere in es/mappings.json https://pasteid.fledix.com/1h1g

"Updated"

Actuly I was successfully update new elasticsearch from nominatim with comand: java -jar photon-0.3.4.jar -nominatim-import -host localhost -port 5432 -database nominatim -user nominatim -password mypass -languages sr But whan i request API &lang=sr I got { "message": "language sr is not supported, supported languages are: default, en, fr, de, it" } Need to add 'sr' in supported languages array.

deicidemilan avatar Sep 17 '20 09:09 deicidemilan

HI @lonvia @deicidemilan were you able to make this work with languages other than the defaults? I'm thinking of trying to make this work with Japanese. I am assuming that, in some manner, it would be possible to get this library to work with all of the options listed here:

https://wiki.openstreetmap.org/wiki/Nominatim/Country_Codes

I understand that Japanese might be particularly tricky given the difference in composition etc, so I appreciate any ideas you may have on how this could be accomplished.

numair avatar Nov 11 '21 11:11 numair

I need further languages. I notice that when searching food i town in china I get results, but when searching for (google translated) I get non related results elsewhere in china, like motorways.

Is it as "simply" as indexing chinese osm data or are there more configuration like in Nomantim needed?

Lelelo1 avatar Mar 26 '22 15:03 Lelelo1

@Lelelo1 Please note that Photon does not searching by category, so a search for 'food' will usually not yield the expected results.

As for integrating Chinese: it can be simply added as a language in the list. The problem is that Chinese is not well supported by standard analysers, so there is probably some effort necessary, to change the analysers that ES uses. See #563 for an attempt to port Photon to Japanese.

lonvia avatar Apr 06 '22 10:04 lonvia

Hello, @lonvia! I try to use some custom languages tags, such as:

  1. "alt_name";
  2. "alt_name:ru".

How can we configurate es/mappings.json file to be able to search for information that may be contained in these tags? Thank you in advance.

Yuri-Chud avatar Oct 23 '23 20:10 Yuri-Chud

alt_name is already considered for search. The list of possible tags is hardcoded here.

lonvia avatar Oct 25 '23 14:10 lonvia

I downloaded realize 0.4.3 , and tried to import using this command by passing the -language with alt_name argument (also tried without it): java -jar photon-*.jar -user_name -port -local host -port 5432 -database name -username -password mysecretpassword -languages alt_name

But my search query can't find the names that exist in these special tag (but I can get result from default languages (e.g. from en)). Do I need to write additional commands to make the search by these tags ("alt_name") too?

Yuri-Chud avatar Oct 26 '23 18:10 Yuri-Chud

Please use the Discussion section to ask questions about your installation. Hiding these comments as unrelated.

lonvia avatar Oct 27 '23 07:10 lonvia