Differences between PhantomJS & Chrome

[aral]

Results differ between PhantomJS and Chrome while testing Stripe iframe functionality with the local version of the page at http://indietech.org/summit …

(Not a showstopper for us at the moment as we’re just going to use Chrome for this, but thought you’d like to know. I can only imagine they should both return the same result.) :)

With PhantomJS:

RUNNING TEST - "Stripe"
▶ OPEN http://localhost:8000/summit
▶ TYPE #name-1
▶ TYPE #email-1
✔ ENABLED Purchase tickets button is enabled.
▶ CLICK #purchase-tickets-button
▶ TOFRAME true
▶ WAITFORELEMENT
✘ EXISTS
0 EXPECTED: undefined
0 FOUND: false
0 MESSAGE: App view exists
▶ WAITFORELEMENT
✘ EXISTS
0 EXPECTED: undefined
0 FOUND: false
0 MESSAGE: Test Mode exists.
✘ TEST - "Stripe" FAILED

1/3 assertions passed. Elapsed Time: 1 min 26.7 sec

---

With Chrome (35.0.x)

UNNING TEST - "Stripe"
▶ OPEN http://localhost:8000/summit
▶ TYPE #name-1
▶ TYPE #email-1
✔ ENABLED Purchase tickets button is enabled.
▶ CLICK #purchase-tickets-button
▶ TOFRAME true
▶ WAITFORELEMENT
✔ EXISTS App view exists
▶ WAITFORELEMENT
✔ EXISTS Test Mode exists.
✔ 3 Assertions run
✔ TEST - "Stripe" SUCCEEDED

---

The test:

module.exports =
    'Stripe': (test) ->
        test
            .open('http://localhost:8000/summit')
            .type('#name-1', 'Aral Balkan')
            .type('#email-1', 'aral@indiephone.eu')
            .assert.enabled('#purchase-tickets-button', 'Purchase tickets button is enabled.')
            .click('#purchase-tickets-button')
            .toFrame('.stripe_checkout_app')
            .waitForElement('.appView', 1000)
            .assert.exists('.appView', 'App view exists')
            .waitForElement('.testMode', 1000)
            .assert.exists('.testMode', 'Test Mode exists.')
            .done()

[asciidisco]

Hey @aral,
thank you for reporting the issue :)

The problem is that the differences between Chrome & PhantomJS are even small, compared to the differences between iOS-Safari & Firefox for example.

I´m going to set up a docs page that outlines the differences; I think it will be in the style of https://github.com/admc/wd/blob/master/doc/jsonwire-unsupported-mapping.md

[stulzer]

I am not so sure, if the problem is Dalek here... I am running a similar issue with a test suite using CasperJS with PhantomJS...

I am migrating those tests to Dalek now, and guess what, I got the same problems when I switch the test context to an iframe. When using Dalek with PhantomJS it fails, but using Chrome, everything goes well...

[aral]

Ah, interesting. Sounds like the issue is with PhantomJS. Will you raise with them or shall we?

Aral

Sent from my iPhone

On 8 Jul 2014, at 19:14, Rubens Stulzer notifications@github.com wrote:

I am not so sure, if the problem is Dalek here... I am running a similar issue with a test suite using CasperJS with PhantomJS...

I am migrating those tests to Dalek now, and guess what, I got the same problems when I switch the test context to an iframe. When using Dalek with PhantomJS it fails, but using Chrome, everything goes well...

—
Reply to this email directly or view it on GitHub.

[stulzer]

I will, I have lots of PhantomJS dump files here... Because they were crashing pretty badly with CasperJS.

I will send them properly.

[ruanyl]

The first time I run test, all passed. but every time I run a second time immediately, it will fail. But it's ok if I wait for a while.

here is the result:
first time:
Running tests
Running Browser: Google Chrome
OS: Mac OS X 10.9.4 x86_64
Browser Version: 36.0.1985.143

RUNNING TEST - "Checking Toggle Button"
▶ OPEN http://localhost/smart/
▶ TYPE #edit-name
▶ TYPE #edit-pass
▶ CLICK #edit-submit
▶ OPEN http://localhost/smart/retail_monitoring?run=/maps
▶ CLICK .toolbar a:last-child
✔ URL Metro toggle button goes to the right page
▶ CLICK .toolbar a:first-child
✔ URL Store toggle button goes to the right page
✔ 2 Assertions run
✔ TEST - "Checking Toggle Button" SUCCEEDED

2/2 assertions passed. Elapsed Time: 5.65 sec

second time:
Running tests
Running Browser: Google Chrome
OS: Mac OS X 10.9.4 x86_64
Browser Version: 36.0.1985.143

RUNNING TEST - "Checking Toggle Button"
▶ OPEN http://localhost/smart/
▶ TYPE #edit-name
▶ TYPE #edit-pass
▶ CLICK #edit-submit
▶ OPEN http://localhost/smart/retail_monitoring?run=/maps
▶ CLICK .toolbar a:last-child
✔ URL Metro toggle button goes to the right page
▶ CLICK .toolbar a:first-child
✘ URL
0 EXPECTED: http://localhost/smart/retail_monitoring?run=/maps
0 FOUND: http://localhost/smart/retail_monitoring?run=/maps/metro
0 MESSAGE: Store toggle button goes to the right page
✘ TEST - "Checking Toggle Button" FAILED

1/2 assertions passed. Elapsed Time: 4.11 sec

and here is test script:

var testToggleBtn = function(test) {
    test.open('http://localhost/smart/')
          .type('#edit-name', 'username')
          .type('#edit-pass', 'password')
          .click('#edit-submit')
          .open('http://localhost/smart/retail_monitoring?run=/maps')
          .click('.toolbar a:last-child')
          .assert.url('http://localhost/smart/retail_monitoring?run=/maps/metro', 'Metro toggle button goes to the right page')
          .click('.toolbar a:first-child')
          .assert.url('http://localhost/smart/retail_monitoring?run=/maps', 'Store toggle button goes to the right page')
          .done();
}; 

[asciidisco]

@ruanyl Are you sure that this is not your application messing things up? It would help to track this down if you could set up a reduced testcase that reproduces this issue, but tests a site that I can access. And btw. even if that is just an application running on your local machine, please mask critical things like passwords in your code.

For future issues, GH can highlight code using markdown (even in issues), that improves readability a lot. Check this guide for more information :)

Thank you.

[ruanyl]

@asciidisco Thank you for your answer. The problem only happens when I testing with chrome, everything is ok with phantomJS. For chrome, if I wait ~15 seconds between two tests, then it's fine.

really thank you for your tips:)