ERROR: /app or /app/.scrapy has incorrect ownership

[jchutian24]

[root@bl1 sycamore]# docker compose run sycamore_crawler_http https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5385863/pdf/cro-0010-0008.pdf
WARN[0000] /home/chutian/sycamore/compose.yaml: version is obsolete
[+] Running 18/18
✔ sycamore_crawler_http 17 layers [⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿] 0B/0B Pulled 21.0s
✔ 71215d55680c Already exists 0.0s
✔ 3cb8f9c23302 Already exists 0.0s
✔ 5f899db30843 Already exists 0.0s
✔ 567db630df8d Already exists 0.0s
✔ d68cd2123173 Already exists 0.0s
✔ 4d2ec28189ce Already exists 0.0s
✔ ac10cbb83311 Already exists 0.0s
✔ fe3fbe6c8a5b Already exists 0.0s
✔ db125a339c84 Already exists 0.0s
✔ c921ba5ae274 Already exists 0.0s
✔ aea4c55ccbb8 Already exists 0.0s
✔ ac9d704f369b Pull complete 6.0s
✔ 3c4c2f02ebe8 Pull complete 0.3s
✔ a7e4a7b132c8 Pull complete 3.6s
✔ 78cf07ec6546 Pull complete 11.0s
✔ 83558cf25b43 Pull complete 11.0s
✔ 24167cdc7d45 Pull complete 11.0s
Version-Info, Sycamore Crawler HTTP Branch: main
Version-Info, Sycamore Crawler HTTP Commit: 8b7190b--2024-04-02_00:28:10_-0700
Version-Info, Sycamore Crawler HTTP Diff: clean
ERROR: /app or /app/.scrapy has incorrect ownership
TO FIX: docker compose run fixuser

When I try to run the sycamore_crawler, it always show this error. I try to run the "docker compose run fixuser", its not work

[bsowell]

Can you let us know what version of sycamore you are using and what platform you are running on? Did you do docker compose up in a separate console? I confirmed that after running docker compose up, docker compose run sycamore_crawler_http https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5385863/pdf/cro-0010-0008.pdf worked for me, but it could be some platform specific issue.

[eric-anderson]

I tracked the problem down, Running as root causes the issue. What I'm unable to reproduce is it still not working after fixuser.

The sequence I did was:

sudo docker compose run sycamore_crawler_http https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5385863/pdf/cro-0010-0008.pdf
# fails, will investigate how to fix it.
sudo docker compose run fixuser
# works
sudo docker compose run sycamore_crawler_http https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5385863/pdf/cro-0010-0008.pdf
# works

Can you try running the second command (docker compose run fixuser)? I don't see it in your output.

You can also fully reset the volumes if that doesn't work. The following sequence worked for me:

docker compose down --volumes
# gets rid of most things if it reports some volumes as undeletable:
docker rm -vf $(docker ps -a | grep sycamore | awk '{print $1}')
# you can make that remove fewer things by adjusting the grep
docker volume rm -f sycamore_crawl_data
# that's probably the name you have, the docker compose down --volumes command may also work at this point

[HenryL27]

@jchutian24 with eric's fix is this still an issue?
@eric-anderson is there anything we need to do (docs, code?) for this?