yarn test failures without cached Daffodil CLI downloads

[stevedlawrence]

Running yarn test sometimes results in these failures:

2 failing
  1) Daffodil Debugger
       should output xml infoset - debugger version 3.10.0:
     Error: DialogService: refused to show dialog in tests. Contents: DialogService: refused to show dialog in tests. Contents: connect ECONNREFUSED 127.0.0.1:4711
      at w.prompt (vscode-file://vscode-app/home/slawrence/owl/daffodil/vscode.git/.vscode-test/vscode-linux-x64-1.90.0/resources/app/out/vs/workbench/workbench.desktop.main.js:1446:1842)
      at se.mb (vscode-file://vscode-app/home/slawrence/owl/daffodil/vscode.git/.vscode-test/vscode-linux-x64-1.90.0/resources/app/out/vs/workbench/workbench.desktop.main.js:2155:66042)
      at se.hb (vscode-file://vscode-app/home/slawrence/owl/daffodil/vscode.git/.vscode-test/vscode-linux-x64-1.90.0/resources/app/out/vs/workbench/workbench.desktop.main.js:2155:59358)
      at async se.startDebugging (vscode-file://vscode-app/home/slawrence/owl/daffodil/vscode.git/.vscode-test/vscode-linux-x64-1.90.0/resources/app/out/vs/workbench/workbench.desktop.main.js:2155:57446)

  2) Daffodil Debugger
       should output json infoset - debugger version 3.10.0:
     Error: DialogService: refused to show dialog in tests. Contents: DialogService: refused to show dialog in tests. Contents: connect ECONNREFUSED 127.0.0.1:4712
      at w.prompt (vscode-file://vscode-app/home/slawrence/owl/daffodil/vscode.git/.vscode-test/vscode-linux-x64-1.90.0/resources/app/out/vs/workbench/workbench.desktop.main.js:1446:1842)
      at se.mb (vscode-file://vscode-app/home/slawrence/owl/daffodil/vscode.git/.vscode-test/vscode-linux-x64-1.90.0/resources/app/out/vs/workbench/workbench.desktop.main.js:2155:66042)
      at se.hb (vscode-file://vscode-app/home/slawrence/owl/daffodil/vscode.git/.vscode-test/vscode-linux-x64-1.90.0/resources/app/out/vs/workbench/workbench.desktop.main.js:2155:59358)
      at async se.startDebu

Steps to reproduced referenced from another issue:

If if I run git clean -xdf && yarn install && yarn test, then those two tests consistently fail, and for a good portion of the test the VS Code GUI is slowly downloading a Daffodi 3.10.0 CLI.

After yarn test finishes, every time I run yarn test without git clean first, then the tests consistently pass.

So it seem like some tests might need to download a new version of the Daffodil CLI, and these tests are sensitive to that taking a while? Once they are downloaded and cached that's not an issue, but maybe running git clean deletes the cache and requires the CLI to be downloaded again and leads to the failure?

Originally posted by @stevedlawrence in #1557

[JeremyYao]

Some thoughts about potential approaches:

1. Move the downloading of the Daffodil CLIs away from the yarn test steps. Could we incorporate it for yarn install or yarn package?
2. Should we have checks in there to see if a port is available before Daffodil-VSCode relevant tests begin? It'll help with the debugging process of seeing if he issue is port availability or if it's something else.

[stevedlawrence]

1. I believe we do already download and bundle a specific version of Daffodil when we build the VS Code extension. But I think these specific tests are testing the ability for the extension to download and use a different version of daffodil than is already bundled. Downloading is a feature of the extension that we probably should verify that it works correctly, so I don't think we want to pre-download things.
2. I'm not familiar with the port issue but if testing depends on a specific port being availble then it probably does need to be checked, or alternatively use a random ephemeral port if testing specific ports is not important.

[JeremyYao]

1. I believe we do already download and bundle a specific version of Daffodil when we build the VS Code extension. But I think these specific tests are testing the ability for the extension to download and use a different version of daffodil than is already bundled. Downloading is a feature of the extension that we probably should verify that it works correctly, so I don't think we want to pre-download things.
2. I'm not familiar with the port issue but if testing depends on a specific port being availble then it probably does need to be checked, or alternatively use a random ephemeral port if testing specific ports is not important.

Good catch on point 1. Pre-downloading doesn't align with the current flow of the extension. It sounds like a potential avenue then could be to pre-download required CLIs at the beginning of yarn test? What are your thoughts? It shouldn't affect the extension from a user's perspective or use-case.

[stevedlawrence]

I would assume the purpose of these failing tests is to make sure the extension can download daffodil versions when trying to run a schema with a non-bundled Daffodil version. In that case trying to pre-download the extension doesn't actually test that.

Or if testing downloading isn't the goal then what you suggest is probably fine. But it does feel like download Daffodil versions important functionality that should be tested.

It might work to add a new test that makes sure downloading works and caches that, and then additional tests can use that cache, as long as the code paths are essentially the same. I'm not faimiliar enough with the extension and how testing works to know if that makes sense/is possible.

[stevedlawrence]

Note, it feels to me like the right fix it to remove the timeout or whatever is preventing the test from working. The test is testing is a legitimate code path so changing the test to avoid that because of something wrong with the test harness doesn't feel like the right fix to me.