[BUG] 'connection reset by peer' when refreshing

[yrro]

Describe the bug
I just noticed that the extension hasn't downloaded any new wallpaper since September 2022. 'Refresh now' in the UI has no effect.

I turned on debug logging and can see the following message when I 'refresh now':

Mar 08 09:55:36 gnome-shell[3858]: BingWallpaper extension: next check in 86400 seconds @ local time 2023-03-09 09:55 +0000
Mar 08 09:55:36 gnome-shell[3858]: BingWallpaper extension: Network error occured: Gio.IOErrorEnum: Error receiving data: Connection reset by peer
Mar 08 09:55:36 gnome-shell[3858]: BingWallpaper extension: next check in 3600 seconds @ local time 2023-03-08 10:55 +0000

Unfortunately it doesn't output the URL it was trying to fetch, so it's a bit difficult to debug this further. But I can use Wireshark where I see a TLS session being established with 13.107.22.200:443 which sends a server certificate for www.bing.com so that's probably the problematic connection.

After the Server Hello message I can see Client Key Exchange, Change Cipher Spec, then two segments of Application Data before I receive an RST, closing the connection. So this could very well be a middleman deciding that downloading wallpaper from Bing is forbidden by our network policies... but it's hard to say because I don't know what URL the extension is trying to fetch.

www.bing.com works fine in my browser, with curl, etc.

Desktop (please complete the following information):

• OS: Debian testing/unstable
• Extension Version: 44
• GNOME Version: 43

[yrro]

I'm able to fetch the metadata refresh URL just fine myself:

$ http -h https://www.bing.com/HPImageArchive.aspx mkt==auto idx==0 n==8 mbl==1 format==js Accept:application/json
HTTP/1.1 200 OK
Accept-CH: Sec-CH-UA-Arch, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version
Cache-Control: private
Content-Encoding: gzip
Content-Length: 1680
Content-Type: application/json; charset=utf-8
Date: Wed, 08 Mar 2023 10:21:46 GMT
ETag: 0a318398
P3P: CP="NON UNI COM NAV STA LOC CURa DEVa PSAa PSDa OUR IND"
Set-Cookie: SUID=M; domain=.bing.com; expires=Wed, 08-Mar-2023 22:21:46 GMT; path=/; HttpOnly
Set-Cookie: MUID=31A397F1F3A86F913BB4853FF2D16EE2; domain=.bing.com; expires=Mon, 01-Apr-2024 10:21:46 GMT; path=/; secure; SameSite=None
Set-Cookie: MUIDB=31A397F1F3A86F913BB4853FF2D16EE2; expires=Mon, 01-Apr-2024 10:21:46 GMT; path=/; HttpOnly
Set-Cookie: _EDGE_S=F=1&SID=25C7A128C6A36F563DF8B3E6C7DA6EC3; domain=.bing.com; path=/; HttpOnly
Set-Cookie: _EDGE_V=1; domain=.bing.com; expires=Mon, 01-Apr-2024 10:21:46 GMT; path=/; HttpOnly
Set-Cookie: SRCHD=AF=NOFORM; domain=.bing.com; expires=Mon, 01-Apr-2024 10:21:46 GMT; path=/
Set-Cookie: SRCHUID=V=2&GUID=61A1AC42DCB749338E7905E565FBDAD7&dmnchg=1; domain=.bing.com; expires=Mon, 01-Apr-2024 10:21:46 GMT; path=/
Set-Cookie: SRCHUSR=DOB=20230308; domain=.bing.com; expires=Mon, 01-Apr-2024 10:21:46 GMT; path=/
Set-Cookie: SRCHHPGUSR=SRCHLANG=en; domain=.bing.com; expires=Mon, 01-Apr-2024 10:21:46 GMT; path=/
Set-Cookie: _SS=SID=25C7A128C6A36F563DF8B3E6C7DA6EC3; domain=.bing.com; path=/
UserAgentReductionOptOut: A7kgTC5xdZ2WIVGZEfb1hUoNuvjzOZX3VIV/BA6C18kQOOF50Q0D3oWoAm49k3BQImkujKILc7JmPysWk3CSjwUAAACMeyJvcmlnaW4iOiJodHRwczovL3d3dy5iaW5nLmNvbTo0NDMiLCJmZWF0dXJlIjoiU2VuZEZ1bGxVc2VyQWdlbnRBZnRlclJlZHVjdGlvbiIsImV4cGlyeSI6MTY4NDg4NjM5OSwiaXNTdWJkb21haW4iOnRydWUsImlzVGhpcmRQYXJ0eSI6dHJ1ZX0=
Vary: Accept-Encoding
X-Cache: CONFIG_NOCACHE
X-MSEdge-Ref: Ref A: 997D78D706C74F0A9AD05D0899B7720F Ref B: LON212050705025 Ref C: 2023-03-08T10:21:46Z

Then I'm also able to fetch the latest image from the .images[] list:

$ http 'https://www.bing.com/th?id=OHR.IntlWomensDayChange_EN-GB0996253952_1920x1080.jpg&rf=LaDigue_1920x1080.jpg&pid=hp'
HTTP/1.1 200 OK
Accept-CH: Sec-CH-UA-Arch, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version
Access-Control-Allow-Headers: *
Access-Control-Allow-Origin: *
Cache-Control: public, max-age=691200
Content-Length: 315980
Content-Type: image/jpeg
Date: Wed, 08 Mar 2023 10:24:38 GMT
NEL: {"report_to":"network-errors","max_age":604800,"success_fraction":0.001,"failure_fraction":1.0}
Report-To: {"group":"network-errors","max_age":604800,"endpoints":[{"url":"https://aefd.nelreports.net/api/report?cat=bingth"}]}
Timing-Allow-Origin: *
X-Cache: TCP_HIT
X-MSEdge-Ref: Ref A: 8B5CE1F545EF423BB71B5FAE403C694C Ref B: LTSEDGE1915 Ref C: 2023-03-08T10:24:38Z



+-----------------------------------------+
| NOTE: binary data not shown in terminal |
+-----------------------------------------+

So I'm rather none the wiser...

[neffo]

Thanks for bug report @yrro this could be a new bug in libsoup (if you are running unstable) or as you said some strange network configuration.

Do you want to try this very simple JS script to test if it's the extension, libsoup or something else we are hitting.

Just save the snip below as test.js, then in a command line just run:
gjs test.js

#!/usr/bin/env gjs

imports.gi.versions.Gtk = '3.0';

const { Gtk, GLib, Soup } = imports.gi;
const ByteArray = imports.byteArray;

let sessionSync = new Soup.SessionSync();

function synchttp_get_data(uri) {
    let msg = Soup.Message.new('GET', uri);
    sessionSync.send_message(msg);
    if (msg.status != 200) {
        log('message code: '+msg.status_code);
    }
    let data = ByteArray.toString(msg.response_body_data.get_data());
    return data;
}

let url = 'https://www.bing.com/HPImageArchive.aspx?format=js&idx=0&n=1&mbl=1&mkt=';

let data = synchttp_get_data(url);

log(data);

This is what I see:

# gjs test.js
Gjs-Message: 21:30:50.964: JS LOG: message code: 200
Gjs-Message: 21:30:50.964: JS LOG: {"market":{"mkt":"en-AU"},"images":[{"startdate":"20230308","fullstartdate":"202303080800","enddate":"20230309","url":"/th?id=OHR.WhitehorseAurora_ROW2466181142_1920x1080.jpg&rf=LaDigue_1920x1080.jpg&pid=hp","urlbase":"/th?id=OHR.WhitehorseAurora_ROW2466181142","copyright":"Aurora display, Whitehorse, Yukon, Canada (© John Hyde/plainpicture/Design Pics)","copyrightlink":"https://www.bing.com/search?q=Aurora+phenomenon&form=hpcapt&filters=HpDate%3a%2220230308_0800%22","title":"Info","quiz":"/search?q=Bing+homepage+quiz&filters=WQOskey:%22HPQuiz_20230308_WhitehorseAurora%22&FORM=HPQUIZ","wp":true,"hsh":"757236c1260a8eb9be1bb97818ad3d8e","drk":1,"top":1,"bot":1,"hs":[]}],"tooltips":{"loading":"Loading...","previous":"Previous image","next":"Next image","walle":"This image is not available to download as wallpaper.","walls":"Download this image. Use of this image is restricted to wallpaper only."}}

[yrro]

Thanks for that. I have to add:

imports.gi.versions.Soup = '2.4';

Then I get:

$ ./bing.js 
Gjs-Message: 11:55:48.890: JS LOG: message code: 200
Gjs-Message: 11:55:48.890: JS LOG: {"market":{"mkt":"en-GB"},"images":[{"startdate":"20230308","fullstartdate":"202303080000","enddate":"20230309","url":"/th?id=OHR.IntlWomensDayChange_EN-GB0996253952_1920x1080.jpg&rf=LaDigue_1920x1080.jpg&pid=hp","urlbase":"/th?id=OHR.IntlWomensDayChange_EN-GB0996253952","copyright":"Cibeles Fountain and Madrid City Hall lit for International Women's Day, Madrid, Spain (© dpa picture alliance/Alamy)","copyrightlink":"https://www.bing.com/search?q=International+Womens+Day&form=hpcapt&filters=HpDate%3a%2220230308_0000%22","title":"Celebrating women","quiz":"/search?q=Bing+homepage+quiz&filters=WQOskey:%22HPQuiz_20230308_IntlWomensDayChange%22&FORM=HPQUIZ","wp":false,"hsh":"d32bd45aa27ccee81b6b5f15d9e0e449","drk":1,"top":1,"bot":1,"hs":[]}],"tooltips":{"loading":"Loading...","previous":"Previous image","next":"Next image","walle":"This image is not available to download as wallpaper.","walls":"Download this image. Use of this image is restricted to wallpaper only."}}

So maybe this is a problem with Soup 3...

[yrro]

Looks like it:

>>> import gi

>>> gi.require_version("Soup", "3.0"

>>> from gi.repository import Soup

>>> mes = Soup.Message.new_from_encoded_form("GET", "https://www.bing.com/HPImageArchive.aspx", Soup.form_encode_hash({"format": "js", "idx": "0", "n": "1", "mbl": "1", "mkt": "auto"}))

>>> ses = Soup.Session()

>>> ses.send_and_read(mes)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
gi.repository.GLib.GError: g-io-error-quark: Error receiving data: Connection reset by peer (44)

So this is not the extension's fault, thanks!

[neffo]

Oh thanks for confirming, I've been testing in Fedora 37 (I think) but perhaps debian unstable is a bit more bleeding edge. Any idea which version it might be? 3.0, 3.1 and 3.2 all had point releases in September.

[yrro]

I see this with 3.2.2 and 3.3.1 but not 2.74.3 and I created some test cases: https://github.com/yrro/soup3-debugging

Further testing shows that libsoup 3 only fails on my work network. Since 2.4 (and curl, Python's urllib, etc.) work fine it may be something weird that libsoup 3 is doing with GnuTLS, that 2.4 doesn't do.

I filed a libsoup issue, will see what they think...

[yrro]

I've found the cause! Does this work for you?

On my Debian testing/unstable system:

$ curl --tlsv1.3 https://www.bing.com/
curl: (35) Recv failure: Connection reset by peer

On a RHEL 8 system:

$ $ curl --tlsv1.3 https://www.bing.com/
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to www.bing.com:443 

So I guess www.bing.com can't speak TLSv1.3; and Soup 3 uses TLSv1.3. I had a quick look through the Soup documentation and it doesn't look like there's any way for a client to influence the TLS version used by Soup... but I only had a quick look at the documentation so I may be wrong.

[yrro]

Sorry I should have said why I reopened this. I wonder if your system is using libsoup 3 and if so how come you're able to fetch from www.bing.com at all?

If libsoup doesn't have any way for you to influence/request TLSv1.2 then I guess there's not much you can do assuming you want to keep using libsoup (which is fine, it's what GNOME Shell gives you to work with after all...)

It appears libsoup relies on glib-networking, which is (in Debian) using GnuTLS.

GnuTLS allows a "priority string" to influence the choice of TLS version, ciphersuites etc.

$ gnutls-cli --priority=NORMAL:-VERS-ALL:+VERS-TLS1.3 www.bing.com:443 
Processed 132 CA certificate(s).
Resolving 'www.bing.com:443'...
Connecting to '204.79.197.200:443'...
*** Fatal error: Error in the pull function.

$ gnutls-cli --priority=NORMAL:-VERS-ALL:+VERS-TLS1.2 www.bing.com:443 
Processed 132 CA certificate(s).
Resolving 'www.bing.com:443'...
Connecting to '131.253.33.200:443'...
- Certificate type: X.509
- Got a certificate list of 2 certificates.
[...]

I guess libsoup doesn't allow clients to specify this?

The weird thing is that gnutls-cli without overriding the priority works fine for me. It's specifically when libsoup -> glib-networking use GnuTLS that I see this behaviour. I wonder if libsoup or glib-networking are specifying a non-default priority string that causes GnuTLS to try to use TLSv1.3...

[yrro]

Ok well while experimenting with various things I decided to upgrade GLib-Networking from 2.66 to 2.74 and now libsoup 3 is able to fetch stuff from www.bing.com!

I'll have to log out and in again to confirm that the extension is able to fetch wallpaper again but I would be very surprised if that doesn't fix it so I'll close this again on the assumption that it'll work.

[neffo]

Happy to keep this open, this is quite interesting. I'll have to think about how we deal with this, I think. The existence of the GNOME introspection files for libsoup3 is not guaranteed even if the library is installed as best I can tell, Ubuntu 22.04 might have just stuck with 2.74.

Soup does appear to have a message priority flag, but not documented to suggest it's the equivalent to the GnuTLS one.

[neffo]

Definitely seeing the same thing with latest Ubuntu 22.04.2 LTS:

➜  ~ curl --tlsv1.3 https://www.bing.com/
curl: (35) OpenSSL SSL_connect: Connection reset by peer in connection to www.bing.com:443 
➜  ~