wttr.in
wttr.in copied to clipboard
chubin
Add support for more data sources
Possible data sources
Possible data sources to be supported:
- OpenWeatherMap
- weather.gov (https://www.weather.gov/documentation/services-web-api)
pogoda.by
api links: https://pogoda.by/api/v2/synoptic/date/2024-10-25 https://pogoda.by/api/v2/weather-fact/all-cities
One of the better ones imho: https://pirateweather.net
That appears to require an API key? I assume wttr.in only wants to use sources that don't?
How about as a next step, we would build little proof of concept (PoC) scripts that all contain the same request details and show how useful outputs are achieved? Could someone document the requirements for such a PoC script? For me, it's for instance not clear yet how we use historical weather data, or if we always use prediction results, or if we do our own predictions in some areas as well.
Why? We have nothing against using the keys
So the end user would have to enter their own key?
@GroovyJeff Of course not. This would be needed only if he runs his own local copy of wttr.in, but that is not the primary scenario. For the public wttr.in service, no keys are needed. Using keys on the server side to access data services is a different matter, but that has always been the case. So nothing changes in that regard.
@fallbackerik Fir the moment, the next step would be to enable users to select their preferred data source when they want specific results they trust, e.g. wttr.in/Berlin?from=accuweather.
@chubin That part might be rather easy to implement for you. But for someone who hasn't worked much with the sources yet, it might be a significant change. I'm not sure. But then, there is still a blackbox of how the specific url for accuweather looks like, if a key is needed, and which json (or other format's) data elements should be collected to create an expected UI.
What I'm thinking off as proof-of-concept, is a simple smaller tool, maybe a simple CLI that just calls to accuweather, just returns a small but meaningful dataset, and thereby gets answers to all the small questions that are currently a black box.
Does that make sense?
@GroovyJeff Using keys on the server side to access data services is a different matter
OK, but isn't that an issue? If all wttr.in users are using the server's key (which is hidden on the server's side, so not directly available to the user), won't the key's usage be quickly exhausted by the sheer volume of queries? Or am I missing something?
@chubin If we follow your approach, I guess a new keyword would need to get added here: https://github.com/chubin/wttr.in/blob/master/lib/parse_query.py#L79-L101
@fallbackerik Erik, I agree with you, and yes, it is a big project, but it is quite intresting (from a technical perspective) and also quite promising. It would significantly enhance the data quality and bring wttr.in to the next level.
@GroovyJeff You are almost right, but there are a lot of small nuances, which actually help significantly reduce traffic to the data sources, such as caching, data interpolation, etc.
there are a lot of small nuances, which actually help significantly reduce traffic to the data sources, such as caching, data interpolation, etc.
That is a surprising answer to me. I thought we don't have caching or similar yet, because of: https://github.com/chubin/wttr.in/issues/1086#issuecomment-2962370653
I'd be curious to learn more about the setup. Especially, how data interpolation helps. I've already read that weather data often needs to interpolate because usually the point of interest is somewhere between 3 or more weather stations where data is collected.
I thought we might have a usecase here, where it would help to have users donate worker nodes with user-specific request keys, and a broker at the center which says which kind of requests need to get made in form of asynchronous tasks.
Erik, I agree with you, and yes, it is a big project, but it is quite intresting (from a technical perspective) and also quite promising. It would significantly enhance the data quality and bring wttr.in to the next level.
Yes, so for me a question is how to make a small next step in the right direction. But even if we add the new keyword to the request parser, it might already mean some refactoring of the code base, because it wasn't meant for this kind of parameter in its current form, right?
And @chubin : How open would you be to rewriting the server on top of an existing framework like flask, Twisted, FastAPI? There's a chance to get part of the solution already delivered via such a standard solution.