Conversation (+attachments) not working: NGINX problem ($query_string missing / Redirect loop?: https?)

[mo-kuh]

Hey, thanks you so much for making freescout available for yunohost (of which I am a great fan of 💯)! Sorry if my message is akwards, i pst anything on github before ...

Unfortunately i spent 2 days to find bugs in the yunohost app. Most is working, so I want to share the solutions. I hope it helps others who benefit from you bringing freescout to yunohost 🥇

My setup:
MxLinux = Debian 11 (update to date)
Yunohost 11.2.8.2 (up to date)
Freescout : 1.8.113 (original app installed version) + 1.8.115 (updated via freescout web console)
Browsers: Firefox 121 (+ Chromium = for testing)

My problems:

1) Problem in NGINX config ("freescout.conf")

I dont know why, but the nginx config of the yunohost app differs from the "standard" nginx config as stated in the freescout install guide.

1.1) Wrong nginx location path (root)?

The nginx access logs for my domain recorded alot of errors ("No such file or directory") involving "/polycast/..."

/var/log/nginx/subdomain.domain.end-access.log 
2023/12/26 21:31:38 [error] 69716#69716: *3664 open() "/usr/share/nginx/htmlindex.php" failed (2: No such file or directory), client: IP, server: subdomain.domain.end, request: "POST /polycast/connect HTTP/2.0", host: "subdomain.domain.end", referrer: "subdomain.domain.end"

On my server there is no "/usr/share/nginx/htmlindex.php" (or "/usr/share/nginx/html/..."). Nothing that freescout was looking for existed under this path. The correct path root is "/var/www/html/public".

I think i solved this problem by using the code from the freescout install guide:

Change
alias /var/www/html/public;

to
root /var/www/html/public;

As I understood it: in some situations nginx couldnt find any matching location in the freescout.conf and did fall back to higher locations with their corresponding roots. In the end the last fallback is the default root ("/usr/share/nginx"). The default root can be changed, but I dont know if this is helpful if you are running other software. You can find a bit more info on this (nginx root) topic here: https://serverfault.com/questions/920880/nginx-defaults-to-usr-share-nginx-html

1.2) Conversation = "Too many redirects"

Most of freescout worked out of the box but I couldnt do anything with "conversations", besides creating new emails. Meaning: I couldnt click on emails titles to see their content or reply to them. Everything the went to "/conversation/..." did end in firefox and chrome aborting the request after some time (1+ minute) or some tries (around 20) with the statement: "The page isn’t redirecting properly" (Firefox) or "ERR_TOO_MANY_REDIRECTS" (Chromium). You can read more about this behaviour here: https://blog.hubspot.com/website/too-many-redirects.

My nginx access logs recorded alot of HTTP "302" errors:

freescoutsubdomain.domain.end access log: 
IP - - [27/Dec/2023:13:36:32 +0100] "GET /conversation/12?folder_id=1 HTTP/2.0" 302 452 "https://freescoutsubdomain.domain.end/mailbox/1" "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"

Its often assumed that this is due to https (certificate) problems and thats why I had alot of trial and error trying to force freescout to use HTTPs:

• this didn't work for me: Users cannot sign in with Google's SAML SSO after upgrade to FS 1.8.79 + SAML SSO 1.0.7 freescout-help-desk/freescout#3075
• this didn't work for me either: Redirect loop after upgrade freescout-help-desk/freescout#3070

In the end i added the following code to 3 of my files (A+B+C):

A) freescout environment variables file (".env" as I found it in /var/www/freescout/.env"):

# Application URL (https = important!)
APP_URL=https://subdomain.domain.end 
APP_FORCE_HTTPS=true
ENABLE_SSL_PROXY=true
NGINX_ENABLE_FASTCGI_HTTPS=true

# If you are using HTTPS, feel free to uncomment this line to improve se>
SESSION_SECURE_COOKIE=true

After changing the ".env" file you have to reload freescout. You can do this fro the webconsole (Manage/System/Tools/"Clear Cache" Button) or command line:
/bin/php8.2 /var/www/freescout/artisan freescout:clear-cache

I cant tell if any of this helped :(

B) freescout.conf (as found in /etc/nginx/conf.d/freescoutsubdomain.domain.end.d/freescout.conf)

I added "fastcgi_param HTTPS on;" in the location block dealing with ".php" files:

  location ~ [^/]\.php(/|$) {
    fastcgi_split_path_info ^(.+?\.php)(/.*)$;
    fastcgi_pass unix:/var/run/php/php8.2-fpm-freescout.sock;

    fastcgi_index index.php;
    include fastcgi_params;
    fastcgi_param REMOTE_USER $remote_user;
    fastcgi_param PATH_INFO $fastcgi_path_info;

# yunohost app original
#    fastcgi_param SCRIPT_FILENAME $request_filename;
# freescout install guide
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param HTTPS on;
  }

C) If you are using nginx as a reverse proxy, then some said that the following code is helpful:

    proxy_set_header    Host $host;
    proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto https;

I added this to the nginx configuration of the domain where freescout is running on ( /etc/nginx/conf.d/freescoutsubdomain.domain.end.conf)

Nothing worked. I almost gave up and was looking at the nginx conf from the freescout install guide "one last time" :)

D) I saw a difference to my servers nginx conf and thought: i did try to change this at the beginning but it didnt work back then. I tried it again, and the conversations where working!

nginx conf from Freescout install guide:

location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

IMPORTANT: You have had to add "$query_string;" to 2 locations:

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

  location ~* ^/storage/attachment/ {
        expires 1M;
        access_log off;
        try_files $uri $uri/ /index.php?$query_string;

NOTE: There are more difference in the nginx configurations used by the yunohost app and the freescout install guide.

2) Conversation attachments not working :(

While writing this I found out that email attachments produce HTTP error "404 Not Found" :(

I guess it has something to do with this part of the nginx freescout.conf:

  location ~* ^/storage/attachment/ {
        expires 1M;
        access_log off;
        try_files $uri $uri/ /index.php$query_string;
  }

• More on this (adding the symbolik link DID fix the attachments issue for me!): 1.5.0 Upgrade 404 to Attachments freescout-help-desk/freescout#522

I hope i get it fixed.

[stepcellwolf]

Hi @mo-kuh,

I can confirm that the above solution is working, for the conversation and for the attachments. Thanks a lot for the workaround and great work.

[stepcellwolf]

However, just noticed that I have another issue now. I get the 'Whoops, looks like something went wrong.' page.

Troubleshooting was as described issue-31, but it is not sustainable. The page comes back again after everything working properly.
I did the following steps:

1. Added the APP_KEY manually to .env file
2. /bin/php8.2 /var/www/freescout/artisan freescout:clear-cache

The problem is solved every time after I run the /bin/php8.2 /var/www/freescout/artisan freescout:clear-cache but it is not for a long term solution.

Any ideas what might be or does anyone else is having the same issue?

[stepcellwolf]

There was another issue with permissions on /var/www/freescout/storage/, I fixed it with
chmod -R 775 /var/www/freescout/storage/ and now it is working without any issues.

[harcesz]

@mo-kuh thank you very much, you gave me the necessary bits to fix a loop that killed conversations for me.

what I used was;

• fixing path to source, which in my case is
  root /var/www/freescout/public/;
• fixing the urls with

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

reloading nginx and it works like a charm. I'm very happy to have Freescout in yunohost, looking forward to it's future development and further integration (LDAP users and so on).

[mo-kuh]

Just a small info:

Updating FreeScout via YUNOHOST did break it again. I tried to fix it but couldn't do it quickly. Menu (clicking on it didnt do anything) + Conversation (404 again: "/var/www/freescout/public/index.phpfolder_id=1" failed (2: No such file or directory)") where not working :(

Workaround:

• Restore the pre-update backup
• Update FreeScout from within FreeScout itself

Whenever there is time i will try to make a fresh install of FreeScout and see its still broken out of the box.

[stepcellwolf]

Hi, this is happening again after updating. Is there a plan to fix it for once?

[stepcellwolf]

No more working I always get the following error: <404 Not Found>. the only solution is the following comment

[W1BTR]

I wanted to simplify the steps for fixing the too many redirects issue as it took me a bit to figure out what exactly the above folks had done.

In /etc/nginx/conf.d/yourdomain.com.d/freescout.conf

• Replace alias /var/www/freescout/public/; near the top with root /var/www/freescout/public/;
• Find the two places with try_files $uri $uri and replace them with: try_files $uri $uri/ /index.php?$query_string;
• run chmod -R 775 /var/www/freescout/storage/

In /var/www/freescout/.env

• ensure APP_URL is set to your domain (i.e. APP_URL=https://yoursite.com)
• IF YOU HAVE A REVERSE PROXY FORWARDING TO YUNOHOST Uncomment #APP_TRUSTED_PROXIES by removing the # and ensure it is set to the ip address of your reverse proxy.
• IF YOU HAVE A REVERSE PROXY FORWARDING TO YUNOHOST add the following the the config on the reverse proxy:

    proxy_set_header    Host $host;
    proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto https;

• restart nginx with sudo service nginx restart
• restart freescout with /bin/php8.2 /var/www/freescout/artisan freescout:clear-cache
• It should now be working (verify) but youre not done yet! Any updates to yunohost or the app will break this fix, so make a backup of /etc/nginx/conf.d/yourwebsitename.com.d/freescout.conf with:
• cp /etc/nginx/conf.d/yourwebsitename.com.d/freescout.conf ~/freescout.conf-backup
• You can then restore it in the future with:
• sudo cp -i ~/freescout.conf-backup /etc/nginx/conf.d/yourwebsitename.com.d/freescout.conf

For reference, my freescout.conf looks like this now that it is fixed:

location / {
  # Path to source
  root /var/www/freescout/public/;

  index index.php index.html index.htm;

  # Common parameter to increase upload size limit in conjunction with dedicated php-fpm file
  client_max_body_size 50M;

  try_files $uri $uri/ /index.php?$query_string;
  # try_files $uri $uri/ /index.php;

  # Add proxy headers here to handle reverse proxy properly
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Proto $scheme;
  proxy_set_header X-Forwarded-Host $host;
  proxy_set_header Host $host;

  location ~ [^/]\.php(/|$) {
    fastcgi_split_path_info ^(.+?\.php)(/.*)$;
    fastcgi_pass unix:/var/run/php/php8.2-fpm-freescout.sock;

    fastcgi_index index.php;
    include fastcgi_params;
    fastcgi_param REMOTE_USER $remote_user;
    fastcgi_param PATH_INFO $fastcgi_path_info;
    fastcgi_param SCRIPT_FILENAME $request_filename;
  }

  location ~* ^/storage/attachment/ {
    expires 1M;
    access_log off;
    try_files $uri $uri/ /index.php?$query_string;
  }

  location ~* ^/(?:css|js)/.*\.(?:css|js)$ {
    expires 2d;
    access_log off;
    more_set_headers "Cache-Control: public, must-revalidate";
  }

  # The list should be in sync with /storage/app/public/uploads/.htaccess and /config/app.php
  location ~* ^/storage/.*\.((?!(jpg|jpeg|jfif|pjpeg|pjp|apng|bmp|gif|ico|cur|png|tif|tiff|webp|pdf|txt|diff|patch|json|mp3|wav|ogg|wma)).)*$ {
    more_set_headers "Content-disposition: attachment; filename=$2";
    default_type application/octet-stream;
  }

  location ~* ^/(?:css|fonts|img|installer|js|modules|[^\\\]+\..*)$ {
    expires 1M;
    access_log off;
    more_set_headers "Cache-Control: public";
  }

  location ~ /\. {
    deny  all;
  }

  # Include SSOWAT user panel.
  # include conf.d/yunohost_panel.conf.inc;
}```

[W1BTR]

But now I have this issue...

[ericgaspar]

will you be able to provide the permissions applied to /var/www/freescout/storage file?

Edit. Is this with the update from testing branch?

[W1BTR]

I have not been able to get the testing branch to install as of yet.

root@yunohost:/var/www/freescout/storage# ls -l
total 16
drwxrwxr-x 5 freescout www-data 4096 Oct  3 00:30 app
drwxrwxr-x 2 freescout www-data 4096 Aug 28 02:15 debugbar
drwxrwxr-x 6 freescout www-data 4096 Oct  3 00:15 framework
drwxrwxr-x 2 freescout www-data 4096 Oct  3 20:00 logs
root@yunohost:/var/www/freescout/storage#

also running /bin/php8.2 /var/www/freescout/artisan freescout:clear-cache fixes the above php permission error temporarily, usually for a couple of hours.

[ericgaspar]

I have included some of the fixes suggested here in this PR: #95
Not being a Freescout user myself, I will welcome your feedback.