Skip to content

test: is sequential/test-fs-watch flaky on macOS #12900

Closed
Closed
test: is sequential/test-fs-watch flaky on macOS#12900
Labels
fsIssues and PRs related to file-system APIs and the fs module.macosIssues and PRs related to the macOS platform.testIssues and PRs related to Node.js core tests and test infrastructure.

Description

@refack
refack
opened on May 8, 2017
  • Version: master
  • Platform: macOS
  • Subsystem: test

https://ci.nodejs.org/job/node-test-commit-osx/9629/nodes=osx1010/tapResults/

1393	sequential/test-fs-watch	
duration_ms	
60.74
severity	
fail
stack	
timeout

/cc @nodejs/platform-macos

Activity

  1. refack
    added
    macosIssues and PRs related to the macOS platform.
    testIssues and PRs related to Node.js core tests and test infrastructure.
    on May 8, 2017

  2. aashil commented on May 8, 2017

    @aashil
    aashil
    on May 8, 2017
    Contributor

    I would like to work on this issue.

  3. refack
    self-assigned this
    on May 8, 2017

  4. refack commented on May 8, 2017

    @refack
    refack
    on May 8, 2017
    ContributorAuthor

    @aashil Go for it 🥇

  5. Trott
    added
    fsIssues and PRs related to file-system APIs and the fs module.
    on May 8, 2017

  6. aashil commented on May 15, 2017

    @aashil
    aashil
    on May 15, 2017
    Contributor

    @refack I tried running this test using ./node test/sequential/test-fs-watch.js and I got nothing. I assume the test ran with no errors?

  7. aqrln commented on May 15, 2017

    @aqrln
    aqrln
    on May 15, 2017
    Contributor

    @aashil it did. You can try something like python tools/test.py --mode=release --repeat=10000 sequential/test-fs-watch and see if it starts failing after a couple of minutes of running. However, a better strategy might be to examine its source code and try to find out the reason analytically and not empirically.

  8. refack commented on May 15, 2017

    @refack
    refack
    on May 15, 2017
    ContributorAuthor

    Since it was a timeout issue that happens only occasionally, it might be hard to reproduce empirically...

  9. Trott commented on May 15, 2017

    @Trott
    Trott
    on May 15, 2017
    Member

    I can replicate the issue reasonably reliably by doing the following:

    • copy the test into test/parallel
    • run with tools/test.py --mode=release --repeat=64 -j 16 test/parallel/test-fs-watch.js

    When I run it that way, it runs successfully a lot but also times out a lot.

    I have not investigated further at this time, but this all suggests that the test is flaky when the system is under load and/or when the test is competing with other processes for specific resources.

  10. aashil commented on May 15, 2017

    @aashil
    aashil
    on May 15, 2017
    Contributor

    I was able to replicate the issue after suggestions from @aqrln and @Trott I will look into this in more detail.

  11. aashil commented on May 16, 2017

    @aashil
    aashil
    on May 16, 2017
    Contributor

    @Trott @refack @aqrln Shouldn't the behavior of the test be the same when running via tools/test.py and running via a script which runs ./node test/sequential/test-fs-watch.js given number of times? With the latter approach I have console.log statements set up at different points which would help me identify which command in the test times out but, the tests never time out, which makes me think if the timeout is specific to calling via tools/test.py Do you have an alternative or a better approach to debug the test?

  12. Trott commented on May 16, 2017

    @Trott
    Trott
    on May 16, 2017
    Member

    @Trott @refack @aqrln Shouldn't the behavior of the test be the same when running via tools/test.py and running via a script which runs ./node test/sequential/test-fs-watch.js given number of times?

    @aashil No. tools/test.py --mode=release --repeat=64 -j 16 test/parallel/test-fs-watch.js will run those tests in parallel. ./node test/sequential/test-fs-watch.js a given number of times will run them sequentially.

  13. aqrln commented on May 16, 2017

    @aqrln
    aqrln
    on May 16, 2017
    Contributor

    @aashil besides that, right, it is tools/test.py watching that a test does not exceed the maximal timeframe. However, you can see if it would fail because of that even without the test harness, because if a test fails with timeout, it usually means that it has hung.

  14. MylesBorins commented on May 16, 2017

    @MylesBorins
    MylesBorins
    on May 16, 2017
    Contributor

    looks like we might have an AIX failure for this on v6.x

    https://ci.nodejs.org/job/node-test-commit-aix/5888/nodes=aix61-ppc64/console

    not ok 1317 sequential/test-fs-watch
  ---
  duration_ms: 0.430
  severity: fail
  stack: |-
    
    assert.js:81
      throw new assert.AssertionError({
      ^
    AssertionError: Got unwanted exception..
        at _throws (assert.js:356:5)
        at Function.assert.doesNotThrow (assert.js:374:3)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/test/sequential/test-fs-watch.js:35:8)
        at Module._compile (module.js:570:32)
        at Object.Module._extensions..js (module.js:579:10)
        at Module.load (module.js:487:32)
        at tryModuleLoad (module.js:446:12)
        at Function.Module._load (module.js:438:3)
        at Module.runMain (module.js:604:10)
        at run (bootstrap_node.js:389:7)

  15. gibfahn commented on May 18, 2017

    @gibfahn
    gibfahn
    on May 18, 2017
    Member

    looks like we might have an AIX failure for this on v6.x

    For posterity this turned out to be unrelated (see #13070)

  16. refack commented on Jun 27, 2017

    @refack
    refack
    on Jun 27, 2017
    ContributorAuthor

    Stress macOS: https://ci.nodejs.org/job/node-stress-single-test/1312/nodes=osx1010/
    Stress AIX: https://ci.nodejs.org/job/node-stress-single-test/1311/nodes=aix61-ppc64/

  17. refack commented on Jun 28, 2017

    @refack
    refack
    on Jun 28, 2017
    ContributorAuthor

    Stress passed (when run as parallel with refack@63ef897):

    macOS:

    + python tools/test.py -p tap --mode=release -j 16 sequential/test-fs-watch
TAP version 13
1..1
ok 1 sequential/test-fs-watch
  ---
  duration_ms: 0.134
  ...
+ OK=99
+ echo '99   OK: 99   NOT OK: 0   TOTAL: 100'
99   OK: 99   NOT OK: 0   TOTAL: 100
+ for i in '`seq $RUN_TIMES`'
+ python tools/test.py -p tap --mode=release -j 16 sequential/test-fs-watch
TAP version 13
1..1
ok 1 sequential/test-fs-watch
  ---
  duration_ms: 0.399
  ...
+ OK=100
+ echo '100   OK: 100   NOT OK: 0   TOTAL: 100'
100   OK: 100   NOT OK: 0   TOTAL: 100
+ '[' 0 '!=' 0 ']'
Notifying upstream projects of job completion
Finished: SUCCESS

    AIX:

    + python tools/test.py -p tap --mode=release -j 16 sequential/test-fs-watch
TAP version 13
1..1
ok 1 sequential/test-fs-watch
  ---
  duration_ms: 0.423
  ...
+ OK=99
+ echo '99   OK: 99   NOT OK: 0   TOTAL: 100'
99   OK: 99   NOT OK: 0   TOTAL: 100
+ for i in '`seq $RUN_TIMES`'
+ python tools/test.py -p tap --mode=release -j 16 sequential/test-fs-watch
TAP version 13
1..1
ok 1 sequential/test-fs-watch
  ---
  duration_ms: 0.489
  ...
+ OK=100
+ echo '100   OK: 100   NOT OK: 0   TOTAL: 100'
100   OK: 100   NOT OK: 0   TOTAL: 100
+ '[' 0 '!=' 0 ']'
Notifying upstream projects of job completion
Finished: SUCCESS

    Closing for now

  18. refack
    closed this as completedon Jun 28, 2017
  19. refack
    removed their assignment
    on Oct 24, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    fsIssues and PRs related to file-system APIs and the fs module.macosIssues and PRs related to the macOS platform.testIssues and PRs related to Node.js core tests and test infrastructure.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions