[weather@mockturtl] Applet Bug: GeoJS.IO dependancy causes "Service Error" from some IPs #5355
Comments
|
I see the same 500 int server error no matter which 'Data Service' I try to use. Interesting. Maybe its related to location services if that fires before the weather service? |
|
ah HA! Check this out: https://get.geojs.io/ So geojs.io gave up the ghost. Why not use gelclue2? |
|
also, i noticed in melange when I switched on 'manual location': File not found: API no longer accessible via this URLUsing the URL /search/ and /reverse/ (with slashes) is no longer supported. Please use URLs as given in the documentation. Examples how to change the URL: You use: https://nominatim.openstreetmap.org/search/?q=Berlin You use: https://nominatim.openstreetmap.org/search/US/Texas/Huston See github issue #3134 for more details. error t=2024-01-06T05:54:24Z [weather@mockturtl#15]: Retrying in the next 225 seconds... This sounds like evnen the nominatim api at OSM has changed. I have had some success with "https://ipinfo.io/$public_ip/json" before. But I also would like to suggest something I've pondered about. Do we really need to look up location more than one time ever for a given network? i.e. once we know the gps coordinate of my house and my gateway is 192.168.0.1 with mac address aa.bb.dd.ee.dd.ff, we probably don't ever need to look that up again. That gets trickier with the use of mobile hotspots I suppose, but most networks aren't on roaming hotspots, so it seems silly to query every time just in case my house moved. There is also a standard way to detect that a network is a mobile hotspot. I've seen Android 'know' when the wifi its using is another phone's hotspot vs a stationary wifi network. To lessen the load on geoip services, I would suggest we identify the discrete network we're on, and count the number of times a geoop service has given back the same answer for that network, and after 5 or 6 we save the 'persistant coordintes' of that network, and don't query for it again. We could probably also identify the ASN we're on, and classify that ASN as a mobile ASN or a stationary ASN. Another hiccup i've seen... My IPv6 service comes from Hurriane Electric's TunnelBroker service in my city's downtown. It should be local enough for the same weather. but it ip geolocates 250 miles away! I'm not trying to vpn to another place. but IPv6 tunnelbrokers 'emerge' to the public web in a different location sometimes. So geoclue might be a smarter way to determine location. And consider the possibility that a client's v4 and v6 addresses could geolocate to different locations. |
|
Hi, You should update the applet first because it's out of date. I'll look into geoclue, but I don't know why https://get.geojs.io/ doesn't work for you |
|
I saw the problem on 3.3.1 as well, but i think its mostly on geojs.io's shoulders, other than it would be nice to have a little explanation of how "saved locations" works. i.e. if there is any saved location, does the applet just use that and no longer tries to geolocate? If so I think the Manual Location slider should cause the saved locations list to grey out when Manual location is turned off. The way I got it working was:
re: geojs.io. it consistently 500's from my laptop at home over ipv6, and succeeds over ipv4. my v6 works everywhere else, its tunnelbroker from hurricane electric. From my phone on carrier data, geojs.io succeeds on ipv6. I'll track down the v6 error with geojs.io. Maybe they blocked Hurricane Electric. others are reporting similar problems: https://app.gitter.im/#/room/#jloh_geojs:gitter.im GeoJS.io no longer maintained: re: GeoClue. Another benefit is that it multiple available methods of geolocation, not just ip geoloation. It will use gps receivers if available, and it will use cell tower data via 3/4g and gprs if available. network gps servers (NMEA from trucking and maritime environments). And if no direct devices provide gps coordinates, it can use the names and mac addresses of nearby wifi access points to figure out the location. GeoClue is maintained by the gnome folks though it works fine in kde, cinnamon, whatever. |
BillyCroan
changed the title
|
the location entry box is for either searching for a location, or the contents match the "Search Entry" field on any of the Saved location it's going to use that. When the applet is not breaking you can use the arrows to switch between them (or to your saved location), maybe I need to make sure they are accessible even when the applet is in an error state. I looked into geoclue (meaning I implemented it). It only provides lat/long - which is enough - but the online solutions give me the timezone, city and country as well. So I will decide on which order I'm going to try it (or geoclue and online at the same time and only fail if I don't get anything) |
|
Applet version/Build date I have encountered the same issue. got Service Error when I use VPN. if turn off the VPN, it works well! |
@Gr3q Can we get text stating that on the config window. even a tool-tip note would help. I don't think I'm an idiot but it was not obvious to me that the "Location" and "Search entry" were connected to each-other. It might also help if the whole "Saved Locations" section was hidden unless you turn on 'Manual Location' @endlessbeloved I suppose it comes down to a question of intent. Does the user want to see the weather of where their computer physically is at the moment, or where "their IP is". I know for me, "where the computer physically is" is my intent. GeoGlue2 accomplishes that. Though it doesn't provide timezone and city. I'm sure there's other resources to connect a GPS coordinate to timezone and city though. Probably even ones that don't require network traffic. Wait a sec. computers know their own timezone typically. cat /etc/timezone in Linux. City is trickier and I'm not sure why the City name would be needed. But in many countries, there is a lot of areas that are not IN or part of a city. I live in city X, and a metropolitan region that goes by the name of its largest city, city Y. I tell locals I live in X, and anyone else I live in Y. I have friends who live outside any city, 50-100 miles away from the nearest metro area, far enough to have different weather for sure. They don't have a city. "Nearest City" would be a better way to solicit city, but know it could be far enough to have different weather. I would imagine that fetching the forecast is not based on a city name, but on GPS coordinates right? i.e. what dot is closest to my dot, read that forecast. That avoids the messiness of multiple city names Istanbul/Constantinople and dialect/accent/locale differences in how a city's name might be represented in text.. |
I should definitely do that, it is indeed unclear. The locations section is also a stock component so I did not have many options explaining but a text description should help.
You can technically switch to your saved locations by using the arrows on the applet, this is why it's not dynamically hidden. But I guess I think about it because those controls are not in the config.
On automatic location the user's real coordinates are always preferred, just like when you use your phone. I simply didn't check if GJS has the capability to expose that before. When I have time to figure out to use it reliable and get all the error messages from I will make a release that uses it then falls back to geoIP services.
I know your set timezone. It's not that same as knowing a set of coordinates' timezone, especially because some users can use more than 1 instance of the applet (using the applet not for their own location, but for some other location)
I'm not debating the usefulness/accuracy of the city/country info at all. Simply it was in the applet from the start so I have to keep it supported. There is a fallback where it displays the lat/long instead of that info so it will not break, just saying.
You are right the applet requests weather info purely on GPS coordinates. On auto, I get it from geoip services (currently), on manual, addresses get converted to coordinates via nominatim |
|
I thought worth sharing some gnome code I just came across. There's GeoClue and GeoCode apparently. GeoClue is what I was blabbing about before and it can get your location by gps coordinate and an accuracy in meters. GeoCode looks like it can lookup (given a GPS coordinate) what the closest city is to that gps coordinate without calling out to a web service. https://discourse.gnome.org/t/commandline-tools-for-geocode/15051/2?u=jrj I guess I am assuming that when possible and reliable, you'd like to reduce dependencies on web services. I know I would. Now if only gnome provided a libweather that predicted the weather without using any web services, that would be a trick! |
|
I'm using geoclue as the primary source in the next version, but if you can recommend a better geoip service I can implement that as well. These are already implemented:
My problem is that I have no way to test (or I just don't know how) which one has the best coverage/reliability. Technically I could call all of them at the same time or let the user choose between them...but out of the 2 I would prefer the former. |
|
maxmind comes to mind. But all of these webservices are middlemen, right? When I want to know where an ip is, i run a whois lookup.whois 1.2.3.4 % Information related to '1.2.3.0 - 1.2.3.255' % Abuse contact for '1.2.3.0 - 1.2.3.255' is '[email protected]' inetnum: 1.2.3.0 - 1.2.3.255 irt: IRT-APNICRANDNET-AU organisation: ORG-RQA1-AP I'm not saying whois is perfect. it doesn't give gps coords and that's a big problem I think. But when provider A is unreliable, and B isn't great either, looking for C or D might not be as good as circumventing them all. How do they get their database? |
|
Here's one that I think geoclue itself uses as a fallback. I found it in geoclue's sourcecode: As for which one has the best reliability...... I suggest keeping counters in the applet. Also store the date/time of the first attempt to use each service. If geoclue isn't working or available, then every time your applet uses a geolocation service, the applet should increment a counter of how many times it thinks it succeeded, and how many times it failed. And if it failed, it should try another service until it has tried them all, then wait X seconds where X is an exponential backoff. Give the user a button to send their counters and that datetime to you if they're having trouble. I'd be reluctant to send more data than that for privacy reasons. Perhaps the button launches a mailto: link. and you could include in the message body, those counters, and a prompt to add any other information or their location or isp if they feel relevant. Or the button could just display a text box and say "copy this into an email to [email protected]" or link to a github ticket and ask people to paste their textbox contents there. I know that won't help with assessing the accuracy of the coordinates, but most of the errors I have experienced were http errors resulting in no coords given. Ultimately, I'd like for you to not have to worry about which is most reliable. I want to get geoclue/geocode to be responsible and reliable, and weather@mockturtl (as well as the dozens of other location-related apps in linux) can all then lean on one project (geoclue) that reliably and quickly gets the location of the computer. But in the short term (year or two) I'd say, Any time the applet is showing 'Unavailable' or 'Error', and someone clicks on it, it should go straight to the settings tab where that "report problems" button is waiting for them. They should see the nature of the problem (can't get ip, ca't get latlon, or can't get weather from OWM). That can help you identify what services work better than others. And if you share that information with the geoclue folks, then cay improve that part. |
|
I implemented both GeoClue and GeoCode.
Maxmind is good, but downloading their full IP table is not feasible, (storing on disk, checking for updates, re-downloading just for this applet is overkill). Also, you need an account which I try to avoid for auto location.
That looks good as well. Honestly, I would prefer this over all other geoip services and yeet all others. If the geoip service that Fedora runs doesn't work for someone then I don't think I could do anything more. I don't think they block IPs, countries or they filter by user-agents either. |
Applet version/Build date
Version 3.2.13 (2023-04-28 02:24:56)
Cinnamon version
5.6.8
Distribution
Debian 12
Graphics hardware and driver used
Intel Corporation Alder Lake-UP3 GT2 [Iris Xe Graphics]
Applet name and maintainer
weather@mockturtl @Gr3q
What happened?
Panel shows "Service Error" instead of weather. It wasintermittant at for about a day now completely constant.
I thought maybe my API key was saturated, but my usage should be around 10% of whats allowed, and I logged into OWM and looked around and there was no indicaton that my key was at fault. Same error occurs if I reset settings to defaults (don't use my key).
In melange, isee a error 500 when calling URL. I added the DEBUG file to try to get more out of it. I'd sure like to know the full URL it's calling, so that I could verify the 500 in curl and maybe report it to OWM not here if I knew it wasn't an applet problem
Other information
error t=2024-01-06T05:33:46Z [weather@mockturtl#15]: Retrying in the next 105 seconds...
<title>500 Internal Server Error</title>info t=2024-01-06T05:33:48Z [weather@mockturtl#15]: Refreshing in progress, refresh skipped.
info t=2024-01-06T05:33:49Z [weather@mockturtl#15]: Error calling URL: Internal Server Error,
500 Internal Server Error
openresty
error t=2024-01-06T05:33:49Z [weather@mockturtl#15]: Retrying in the next 105 seconds...
<title>500 Internal Server Error</title>info t=2024-01-06T05:33:52Z [weather@mockturtl#15]: Error calling URL: Internal Server Error,
500 Internal Server Error
openresty
error t=2024-01-06T05:33:53Z [weather@mockturtl#15]: Retrying in the next 120 seconds...
<title>500 Internal Server Error</title>info t=2024-01-06T05:33:53Z [weather@mockturtl#15]: App is currently refreshing, refresh skipped in main loop
info t=2024-01-06T05:35:53Z [weather@mockturtl#15]: Error calling URL: Internal Server Error,
500 Internal Server Error
openresty
error t=2024-01-06T05:35:53Z [weather@mockturtl#15]: Retrying in the next 135 seconds...
info t=2024-01-06T05:35:53Z [weather@mockturtl#15]: App is currently refreshing, refresh skipped in main loop
The text was updated successfully, but these errors were encountered: