Firestore Tests sporadically failing

[danielpeintner]

see https://github.com/eclipse/thingweb.node-wot/actions/runs/3054165973/jobs/4925717361 for example

  2 failing

  1) Firestore client basic test implementation
       [client] subscribe and unsubscribe event with object:
     AssertionError: expected null to deeply equal { eventStr: 'event1', eventNum: 123 }
      at FirestoreClientBasicTest.<anonymous> (test/firestore-client-basic-test.ts:237:16)
      at Generator.next (<anonymous>)
      at fulfilled (test/firestore-client-basic-test.ts:30:58)

  2) Firestore client basic test implementation
       [client] observe and unobserve property:
     AssertionError: expected null to equal 'test-string-888'
      at FirestoreClientBasicTest.<anonymous> (test/firestore-client-basic-test.ts:264:16)
      at Generator.next (<anonymous>)
      at fulfilled (test/firestore-client-basic-test.ts:30:58)

I think it is a timing issue but I am not sure.

@hidetak do you know more?

BTW, we (I) have one more strange issue... not really related to Firestore but still strange, see #826

[danielpeintner]

Moreover, we have very often timeout issues with the firestore test setup which make the CI build/test fail.
Note: Most of the time this happens on MacOS.

see https://github.com/eclipse/thingweb.node-wot/actions/runs/3060186473/jobs/4938400649

⚠  emulators: You are not currently authenticated so some features may not work correctly. Please run firebase login to authenticate the CLI.
i  emulators: Starting emulators: auth, firestore
⚠  It appears you are running in a CI environment. You can avoid downloading the Firestore Emulator repeatedly by caching the /Users/runner/.cache/firebase/emulators directory.
i  firestore: downloading cloud-firestore-emulator-v1.13.1.jar...

i  firestore: Firestore Emulator logging to firestore-debug.log
i  emulators: Shutting down emulators.
i  firestore: Stopping Firestore Emulator
i  hub: Stopping emulator hub

Error: TIMEOUT: Port 8088 on localhost was not active within 60000ms

[hidetak]

@danielpeintner
Thank you for your research.
I am new to the phenomenon and do not know the solution at this time.
I will look into it this weekend.

[danielpeintner]

We now have failing PRs due to Firestore in almost any PR (see #850 (comment)).

@hidetak do you have a change to check this.

Moreover, what can/should we do for now:

• disable firestore tests? (rather drastic)
• is there a way to mark certain tests as non stable so that they don't show up as failures
• other proposals ???

[hidetak]

@danielpeintner

Sorry for the very late reply.
My work has become very busy and I have not been able to devote much time to this development.

I tried to get the master branch and test it, but the npm ci for windows-latest, 18.x failed and I could not see any errors in the Firebase emulator.
https://github.com/hidetak/thingweb.node-wot/actions/runs/3387282040/jobs/5628076208

In any case, I am hoping that you will temporarily disable the test as it is difficult for me to take the time to investigate.
My apologies.

[danielpeintner]

@hidetak thanks for letting us know.
I think we should temporarily disable testing for binding-firestore till we have a solution (see #865).

[danielpeintner]

Firestore testing has been disabled in #878

[danielpeintner]

Check whether node-fetch/node-fetch#1624 (comment) solves the issue.

[danielpeintner]

Issue can be closed since firestore binding has been removed.

[hidetak]

I apologize for any inconvenience caused by our inability to correct the test code.
I believe that the WoT will be used even more in the future because of the work of everyone in this community.
I hope to find time in the future to modify Binding Firebase or find other things to contribute.
Thank you for all your support.

[danielpeintner]

Thank you @hidetak for your work and your contribution.

The firestore code is still there (see https://github.com/eclipse-thingweb/node-wot/tree/v0.8.6/packages/binding-firestore) and you or others can start the work to align in the future again.