Actions gets stuck when starting a step #571
Description
Edit: Feel free to edit the title of this issue to something better. I'm not super well-versed in the vocabulary of this.
Hey, so I've been helping to migrate the repositories for an org I'm in to actions. I did our CircuitPython libraries in November and December, which went very well. I'm currently working with a few other people to move all of our Arduino libraries to actions, which is almost complete.
So, on to the issue I've been having:
Running a workflow on Github's servers on 2 repositories gets stuck before the directive gets run. It will run for a few hours and then I'll just stop it because there's no reason to just keep it running. In this case, I only let it run a few minutes since I was grabbing the logs after turning them on. I don't think this is on our end, but I don't know nearly enough about this to actually know for sure.
Environment:
OS: Ubuntu 18.04.4
Runner version: 2.263.0
Relevant part of the workflow file
- name: test platforms
run: python3 ci/build_platform.py main_platforms
Comparing it to another library that isn't having this issue, it seems like the issue is happening before build_platform.py is actually run. The odd thing is that there are close to 270 libraries in the org I'm a part of that are running the same workflow file but not having these issues.
From the normal log:
2020-06-26T20:14:06.2946965Z ##[debug]Starting: test platforms
2020-06-26T20:14:06.2955266Z ##[debug]Loading inputs
2020-06-26T20:14:06.2956924Z ##[debug]Loading env
2020-06-26T20:14:06.2961499Z ##[group]Run python3 ci/build_platform.py main_platforms
2020-06-26T20:14:06.2961729Z python3 ci/build_platform.py main_platforms
2020-06-26T20:14:06.3002596Z shell: /bin/bash -e {0}
2020-06-26T20:14:06.3002754Z env:
2020-06-26T20:14:06.3002874Z pythonLocation: /opt/hostedtoolcache/Python/3.8.3/x64
2020-06-26T20:14:06.3003034Z ##[endgroup]
2020-06-26T20:14:06.3044169Z ##[debug]/bin/bash -e /home/runner/work/_temp/c9f30b10-b927-4812-bc2d-0201ff1b5c11.sh
2020-06-26T20:14:06.5850548Z Installing in /home/runner/work/Adafruit_DAP/Adafruit_DAP/bin
2020-06-26T20:14:06.5859543Z ARCH=64bit
2020-06-26T20:14:06.5868052Z OS=Linux
2020-06-26T20:14:06.5868608Z Using curl as download tool
2020-06-26T20:14:07.0060729Z TAG=0.11.0
2020-06-26T20:14:07.0062607Z CLI_DIST=arduino-cli_0.11.0_Linux_64bit.tar.gz
2020-06-26T20:14:07.0063338Z Downloading https://downloads.arduino.cc/arduino-cli/arduino-cli_0.11.0_Linux_64bit.tar.gz
2020-06-26T20:14:07.5816373Z arduino-cli Version: 0.11.0 Commit: 0296f4d installed successfully in /home/runner/work/Adafruit_DAP/Adafruit_DAP/bin
2020-06-26T20:14:13.4573858Z Library Adafruit DAP library is not installed
2020-06-26T20:18:15.7838524Z ##[debug]Re-evaluate condition on job cancellation for step: 'test platforms'.
2020-06-26T20:18:28.3004843Z ##[error]The operation was canceled.
From the diagnostic log:
[2020-06-26 20:14:06Z INFO ProcessInvokerWrapper] Starting process:
[2020-06-26 20:14:06Z INFO ProcessInvokerWrapper] File name: '/bin/bash'
[2020-06-26 20:14:06Z INFO ProcessInvokerWrapper] Arguments: '-e /home/runner/work/_temp/c9f30b10-b927-4812-bc2d-0201ff1b5c11.sh'
[2020-06-26 20:14:06Z INFO ProcessInvokerWrapper] Working directory: '/home/runner/work/Adafruit_DAP/Adafruit_DAP'
[2020-06-26 20:14:06Z INFO ProcessInvokerWrapper] Require exit code zero: 'False'
[2020-06-26 20:14:06Z INFO ProcessInvokerWrapper] Encoding web name: ; code page: ''
[2020-06-26 20:14:06Z INFO ProcessInvokerWrapper] Force kill process on cancellation: 'False'
[2020-06-26 20:14:06Z INFO ProcessInvokerWrapper] Redirected STDIN: 'False'
[2020-06-26 20:14:06Z INFO ProcessInvokerWrapper] Persist current code page: 'False'
[2020-06-26 20:14:06Z INFO ProcessInvokerWrapper] Keep redirected STDIN open: 'False'
[2020-06-26 20:14:06Z INFO ProcessInvokerWrapper] High priority process: 'False'
[2020-06-26 20:14:06Z INFO ProcessInvokerWrapper] Updated oom_score_adj to 500 for PID: 5935.
[2020-06-26 20:14:06Z INFO ProcessInvokerWrapper] Process started with process id 5935, waiting for process exit.
[2020-06-26 20:14:06Z INFO JobServerQueue] Try to append 1 batches web console lines for record '26f1c419-bc2c-554c-71b6-9aa612d1c993', success rate: 1/1.
[2020-06-26 20:14:06Z INFO JobServerQueue] Try to append 1 batches web console lines for record '64195e42-44eb-5e07-91ab-2f7354f198e9', success rate: 1/1.
[2020-06-26 20:14:06Z INFO JobServerQueue] Try to append 1 batches web console lines for record '64195e42-44eb-5e07-91ab-2f7354f198e9', success rate: 1/1.
[2020-06-26 20:14:06Z INFO JobServerQueue] Try to upload 1 log files or attachments, success rate: 1/1.
[2020-06-26 20:14:07Z INFO JobServerQueue] Try to append 1 batches web console lines for record '64195e42-44eb-5e07-91ab-2f7354f198e9', success rate: 1/1.
[2020-06-26 20:14:07Z INFO JobServerQueue] Try to append 1 batches web console lines for record '64195e42-44eb-5e07-91ab-2f7354f198e9', success rate: 1/1.
[2020-06-26 20:14:13Z INFO JobServerQueue] Try to append 1 batches web console lines for record '64195e42-44eb-5e07-91ab-2f7354f198e9', success rate: 1/1.
[2020-06-26 20:14:48Z INFO JobServerQueue] Stop aggressive process web console line queue.
[2020-06-26 20:18:15Z INFO Worker] Cancellation/Shutdown message received.
So it seems to me that the issue is presenting itself in the second to last line. That's the point it will hang until I cancel the workflow.
Here are the two PRs on the libraries that were doing this:
adafruit/TFTLCD-Library#47
adafruit/Adafruit_DAP#11 (where the above logs came from)
Also semi-related, I reorganized some #includes and it hung at the same spot. I put it back to where it was before when it was failing because of the #includes and it no longer failed due to the issue above.
adafruit/Adafruit_nRF8001#27
Let me know if there's anything I can do to help. Thanks!