using chrome CDP

[jansengorilla]

Using docker. Having installed browserless/chromium, i get the following error, having set 'driver / useCDP: true' in the yml file of the scraper.

failed to modify wsURL: invalid character '<' looking for beginning of value

What am i missing?

Many thanks!

[feederbox826]

browserless-chromium is not CDP or a CDP equivalent.
You need to use https://hub.docker.com/r/chromedp/headless-shell/ instead.

[cytec]

Well yes and no.
browserless offers the same features as CDP + some extras and they do support chromedp which is used by stash see https://docs.browserless.io/libraries/chromedp

imho the main reason why browserless doesn't work is that stash allows chromedp to modify the url here

stash/pkg/scraper/url.go

Line 149 in 0621d87

ctx, cancelAct = chromedp.NewRemoteAllocator(ctx, remote)

which could be "fixed" by adding chromedp.NoModifyURL

a simple way to support both would be to add the chromedp.NoModifyURL flag if the url was given with ws:// or wss:// (or maybe make it an option to disable url modification)

also: the isCDPathWS function doesn't check wss://

BTW: when you put https://hub.docker.com/r/chromedp/headless-shell/ behind a reverse proxy (to get rid of the port, add basic auth or whatever) chromedp fails with failed to modify wsURL: address XXXX missing port in address

i've got a remote instance of browserless running and it would be nice to be able to use it. spinning up another container that does the same but is completely open doesn't seem very appealing to me tbh