[🐛 Bug]: Selenium [:driver_service] polling for socket on ["127.0.0.1", 9515] on Docker with Ruby on Rails (webscraping) #15042
Comments
|
@kevinher7, thank you for creating this issue. We will troubleshoot it as soon as we can. Info for maintainersTriage this issue by using labels.
If information is missing, add a helpful comment and then
If the issue is a question, add the
If the issue is valid but there is no time to troubleshoot it, consider adding the
If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C),
add the applicable
After troubleshooting the issue, please add the Thank you! |
|
Hello again! After some more debugging I found out it's not really an issue with Selenium itself but rather some sort configuration set by Rails, but this is just my guess. For the moment here is what I found. The work aroundThe work around I found is to use Docker Compose with two services: the rails app and a selenium image. The resulting image was surprisingly light so it's not a bad work around. I add this services:
website:
build: .
entrypoint: ./bin/docker-entrypoint
command: ["./bin/thrust", "./bin/rails", "server"]
ports:
- "80:3000"
depends_on:
- selenium
environment:
- RAILS_MASTER_KEY=${RAILS_MASTER_KEY}
- SELENIUM_URL=http://selenium:4444/wd/hub
tty: true # you may not need this
stdin_open: true # or this, but I forgot what they do
selenium:
image: selenium/standalone-chrome
ports:
- "4444:4444"The important things are
And then modify the scraper service class to use a remote driver require "selenium-webdriver"
class Scraper
def initialize(url)
@options = Selenium::WebDriver::Chrome::Options.new
@options.add_argument("--headless")
@selenium_url = ENV.fetch("SELENIUM_URL", "http://localhost:4444/wd/hub")
@article_url = url
end
def scrape
@driver = Selenium::WebDriver.for :remote, url: @selenium_url, options: @options
@driver.navigate.to @article_url
# Do Some Scraping
@driver.quit
# Return Scraped Data
end
endAfter making these changes, build with Hope this helps someone out there! Why tho?After some more debugging I managed to pinpoint the cause to being some sort of compatibility issue between the Rails and Chrome/Chromedriver (maybe the way Rails is made it prevents the process to directly interact with other Chrome instances), since I could run a scraping script from the shell of the same docker container given that the ruby script was outside the rails app directory (although I didn't try running a file inside the rails directory from the docker bash). I did try a lot of things like running dbus or trying to set up a display, but my best guess is that Rails (at least in production mode) prevents ruby files from opening or connecting with Chrome/Chromedriver. However, in the end I guess is some sort of issue with rails and not selenium itself, so I suppose it's okay to close this issue. Thanks for reading. |
and others
What happened?
Hello everyone! It's my first bug report so sorry if the format is bad.
I've been looking for a solution to this for the last couple of days but the few things I found were about testing with selenium and haven't been of much help.
I am making a Rails app that uses selenium to do web scrapping. Although locally it works just fine, once I run a docker container I get the following log messages
Even then, I can run
chromedriverfrom the terminal just fineI do get the following socket related error when I try to execute chrome and I think the problem might lie here but I am not sure how to go around it. I do intend to use chromedriver headless.
I would appreciate if I could get some help with this problem.
How can we reproduce the issue?
Build the Image and then after running it go to the endpoint that triggers the scrapping get the logs mentioned above.
I also played changing to the normal (not slim) version of the ruby image but the problem persisted
Operating System
It says that the Ruby image is based on Debian but I am not sure.
Selenium version
Ruby 3.4.1 and selenium-webdriver 4.27.0
What are the browser(s) and version(s) where you see this issue?
Chrome 131.0.6778.204 (stable version atm)
What are the browser driver(s) and version(s) where you see this issue?
ChromeDriver 131.0.6778.204
Are you using Selenium Grid?
No (I think)
The text was updated successfully, but these errors were encountered: