Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Playwright connectOverCDP() not working #9911

Open
jasonwillschiu opened this issue Apr 4, 2024 · 16 comments · May be fixed by microsoft/playwright#34546
Open

Playwright connectOverCDP() not working #9911

jasonwillschiu opened this issue Apr 4, 2024 · 16 comments · May be fixed by microsoft/playwright#34546
Assignees
Labels

Comments

@jasonwillschiu
Copy link

What version of Bun is running?

1.1.0+5903a6141

What platform is your computer?

Darwin 23.4.0 arm64 arm

What steps can reproduce the bug?

I'm trying to use Playwright's connectOverCDP to access an existing Chrome browser in debug mode. This is for a web scraping project. I've returned to Nodejs and everything works fine, but keen to try bun more.

My setup

  • M1 Arm Mac
  • Bun 1.1
  • Playwright [email protected]
  • Chrome Version 123.0.6312.107 (Official Build) (arm64)

I run chrome using:
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222 --no-first-run --no-default-browser-check --user-data-dir=chrome-user-data-dir --profile-directory=Default

Code:

import { chromium } from 'playwright';
const browser = await chromium.connectOverCDP('http://localhost:9222');
const defaultContext = browser.contexts()[0];
const page = await defaultContext.newPage();
page.goto('https://google.com')
await page.waitForTimeout(5000);
await browser.close();

Bun Error message:

ConnectionRefused: overCDP: Unable to connect. Is the computer able to access the url?
Call log:
  - <ws preparing> retrieving websocket url from http://localhost:9222

In Nodejs it connects and works as expected.

Thanks for all your great work!
Jason

What is the expected behavior?

A new Chrome tab is opened to google.com and then closed after 5 seconds.

What do you see instead?

Bun Error message:

ConnectionRefused: overCDP: Unable to connect. Is the computer able to access the url?
Call log:
  - <ws preparing> retrieving websocket url from http://localhost:9222

Additional information

No response

@jasonwillschiu jasonwillschiu added the bug Something isn't working label Apr 4, 2024
@rossanmol
Copy link

Experiencing same issue :/

@sohaieb
Copy link

sohaieb commented Apr 9, 2024

I faced the same issue here as well

@blackfan23
Copy link

blackfan23 commented Apr 20, 2024

I have set up a repo here:
https://github.com/blackfan23/bun-test-playwright

It uses nx and has 3 test cases (see readme)

  1. node + playwright (connects fine)
  2. bun + puppeteer (connects fine)
  3. bun + playwright (does not work)

Edit 2024-11-03:
Updated again, latest bun and deps. Same error persists. I assume it is not being worked on and won't be fixed by accident. Puppeteer connect works flawlessly. Maybe that's a starting point?

Edit 2024-07-28:
I have updated to the latest Bun and deps. Now the bun-playwright connect throws an error:

`error: overCDP: WebSocket error: ws://localhost:9555/devtools/browser/8b68f908-be47-4a64-99d1-81c37df5fccb 200 OK
Call log:

  • retrieving websocket url from http://localhost:9555
  • ws://localhost:9555/devtools/browser/8b68f908-be47-4a64-99d1-81c37df5fccb
  • ws://localhost:9555/devtools/browser/8b68f908-be47-4a64-99d1-81c37df5fccb 200 OK
  • ws://localhost:9555/devtools/browser/8b68f908-be47-4a64-99d1-81c37df5fccb error WebSocket was closed before the connection was established
  • ws://localhost:9555/devtools/browser/8b68f908-be47-4a64-99d1-81c37df5fccb WebSocket was closed before the connection was established
  • ws://localhost:9555/devtools/browser/8b68f908-be47-4a64-99d1-81c37df5fccb code=1006 reason=`

@nemya9066
Copy link

Same issue with the latest version bun throw an error message but the problem still persist...

@ohroy
Copy link

ohroy commented Aug 8, 2024

any news ?

@NicolaiSchmid
Copy link

Does anyone know where exactly the issue lies?

@ozeron
Copy link

ozeron commented Oct 30, 2024

Still persist, any updates?

@vlatkodimeski
Copy link

vlatkodimeski commented Nov 11, 2024

Any update on this?
I am still getting this issue

message: 'TimeoutError: browserType.connectOverCDP: Timeout 30000ms exceeded.\n' +
'Call log:\n' +
' \x1B[2m- retrieving websocket url from http://localhost:9222\x1B[22m\n' +
'\x1B[2m - ws://localhost:9222/devtools/browser/b9b16310-4a05-4c9c-a6fe-087db633d583\x1B[22m\n' +
'\x1B[2m - ws://localhost:9222/devtools/browser/b9b16310-4a05-4c9c-a6fe-087db633d583\x1B[22m\n',
stack: 'TimeoutError: browserType.connectOverCDP: Timeout 30000ms exceeded.\n' +
'Call log:\n' +
' \x1B[2m- retrieving websocket url from http://localhost:9222\x1B[22m\n' +
'\x1B[2m - ws://localhost:9222/devtools/browser/b9b16310-4a05-4c9c-a6fe-087db633d583\x1B[22m\n' +
'\x1B[2m - ws://localhost:9222/devtools/browser/b9b16310-4a05-4c9c-a6fe-087db633d583\x1B[22m\n' +
'\n' +

@IamAlexandros
Copy link

IamAlexandros commented Nov 12, 2024

I have set up a repo here: https://github.com/blackfan23/bun-test-playwright

It uses nx and has 3 test cases (see readme)

  1. node + playwright (connects fine)
  2. bun + puppeteer (connects fine)
  3. bun + playwright (does not work)

Edit 2024-11-03: Updated again, latest bun and deps. Same error persists. I assume it is not being worked on and won't be fixed by accident. Puppeteer connect works flawlessly. Maybe that's a starting point?

Edit 2024-07-28: I have updated to the latest Bun and deps. Now the bun-playwright connect throws an error:

`error: overCDP: WebSocket error: ws://localhost:9555/devtools/browser/8b68f908-be47-4a64-99d1-81c37df5fccb 200 OK Call log:

  • retrieving websocket url from http://localhost:9555
  • ws://localhost:9555/devtools/browser/8b68f908-be47-4a64-99d1-81c37df5fccb
  • ws://localhost:9555/devtools/browser/8b68f908-be47-4a64-99d1-81c37df5fccb 200 OK
  • ws://localhost:9555/devtools/browser/8b68f908-be47-4a64-99d1-81c37df5fccb error WebSocket was closed before the connection was established
  • ws://localhost:9555/devtools/browser/8b68f908-be47-4a64-99d1-81c37df5fccb WebSocket was closed before the connection was established
  • ws://localhost:9555/devtools/browser/8b68f908-be47-4a64-99d1-81c37df5fccb code=1006 reason=`

Bump. Having exactly the same issue. Example:

import { chromium } from 'playwright';
const wsUrl = 'the_ws_url'
browser = await chromium.connect(wsUrl, {
	timeout: 120000
}); // this is throwing with bun, same with "connectOverCDP"

Using:

  • bun: 1.1.34
  • playwright: ^1.48.2
  • os: Fedora 40

@iSuslov
Copy link

iSuslov commented Nov 25, 2024

@Jarred-Sumner @dylan-conway @paperdave @nektro @cirospaciari

I can't help but notice the abundance of reproduction examples here. Could someone at least remove the needs repro tag? 🤔

It would be truly wonderful if someone from the team could spare their precious time to grace us with even a single comment on this issue. This is Playwright after all - not some obscure package - and clearly many developers are running into this problem.

But instead, what do we get? @Electroid going through related issues and marking them as duplicates... of themselves? Seriously? Look at that - issue that comes up first in google results.

Perhaps we could get some actual attention on this issue instead of this circular duplicate-marking dance?

@wxs77577
Copy link

wxs77577 commented Jan 9, 2025

Playwright support plz...
error when use bun, but it's ok to use tsx or node

@rossanmol
Copy link

@Jarred-Sumner any updates on this? Playwright is widely used, and websockets too. Having this solved would allow us to switch our tooling to bun.

@geminigeek
Copy link

looking for Playwright with websockets

@DonIsaac
Copy link
Contributor

Update: I'm actively investigating this. Fixes will likely coincide with the rest of node:net compatibility improvements.

I'll provide more update as I have them.

@DonIsaac
Copy link
Contributor

Update: I've tracked down the problem.

Since bun's HTTP client implementation does not support websocket upgrades; instead, websocket clients must use globalThis.WebSocket. However, a lot of code uses ws (a from-scratch websocket implementation). To keep existing code from breaking, we override ws imports with our own shim.

This normally works fine. Until you try to bundle ws, as Playwright does.

I'll be making a PR into their codebase shortly, which should fix this.

@DonIsaac
Copy link
Contributor

If you want this fix now, without waiting for our PR into playwright to land, you can use bun patch.

First, prepare playwright-core for patching

bun patch node_modules/playwright-core
# or, to patch it everywhere, run `bun patch playwright-core`

Then, edit lib/utilsBundle.js with this change:

-const ws = exports.ws = require('./utilsBundleImpl').ws;
+const ws = exports.ws = 'Bun' in globalThis ? require('ws') : require('./utilsBundleImpl').ws;

By running:

pushd node_modules/playwright-core
# Open lib/utilsBundle.js in your favorite editor. Make the above change
popd

Finally, complete the patch by running

# if you want this patch applied everywhere, use `playwright-core` instead
bun patch --commit node_modules/playwright-core --patches-dir=.patches

You should see this patch in .patches/ that looks something like this:

diff --git a/lib/utilsBundle.js b/lib/utilsBundle.js
index 9d4258e5f0edc3447214d37ea28651675ba19e30..5e0ed26ace3c3ca70155c5b43331ab4f2ea2bc19 100644
--- a/lib/utilsBundle.js
+++ b/lib/utilsBundle.js
@@ -43,7 +43,7 @@ const program = exports.program = require('./utilsBundleImpl').program;
 const progress = exports.progress = require('./utilsBundleImpl').progress;
 const SocksProxyAgent = exports.SocksProxyAgent = require('./utilsBundleImpl').SocksProxyAgent;
 const yaml = exports.yaml = require('./utilsBundleImpl').yaml;
-const ws = exports.ws = require('./utilsBundleImpl').ws;
+const ws = exports.ws = 'Bun' in globalThis ? require('ws') : require('./utilsBundleImpl').ws;
 const wsServer = exports.wsServer = require('./utilsBundleImpl').wsServer;
 const wsReceiver = exports.wsReceiver = require('./utilsBundleImpl').wsReceiver;
 const wsSender = exports.wsSender = require('./utilsBundleImpl').wsSender;

And your package.json should now have a new field in patchedDependencies:

"patchedDependencies": {
  // 1.50.0 will be whatever version of playwright-core you have installed
  "[email protected]": ".patches/[email protected]"
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.