Random failure: Copy local build to Node [FAILED "copyTo", failed with code=1006, reason=] #1381
Assignees
Labels
Bug
A error that causes the feature to behave differently than what was expected based on design docs
P1
High priority issue. Required to be completed in the assigned milestone.
released on @0.35.x
released
Comments
alex-kuzmin-hg
added
Bug
jeromy-cannon
added
P1
|
🎉 This issue has been resolved in version 0.35.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
|
🎉 This issue has been resolved in version 0.35.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
I see sticky error deploying Hedera with Solo.
[solo:node:setup] ✔ Check network pod: node6 [solo:node:setup] ✔ Check network pod: node2 [solo:node:setup] ✔ Check network pod: node8 [solo:node:setup] ✔ Check network pod: node1 [solo:node:setup] ✔ Check network pod: node5 [solo:node:setup] ✔ Check network pod: node3 [solo:node:setup] ✔ Check network pod: node9 [solo:node:setup] ✔ Check network pod: node10 [solo:node:setup] ✔ Check network pod: node4 [solo:node:setup] ✔ Check network pod: node7 [solo:node:setup] ✔ Identify network pods [solo:node:setup] ❯ Fetch platform software into network nodes [solo:node:setup] ❯ Copy local build to Node: node1 from /home/runner/_work/performance-analysis-automation/performance-analysis-automation/hedera-services/hedera-node/data [solo:node:setup] ✖ Copy local build to Node: node1 from /home/runner/_work/performance-analysis-automation/performance-analysis-automation/hedera-services/hedera-node/data [FAILED: copyTo[network-node1-0,ed951421-cade-4757-9ff8-ea1d1a690b54]: failed with code=1006, reason=] [solo:node:setup] ✖ Fetch platform software into network nodes [FAILED: copyTo[network-node1-0,ed951421-cade-4757-9ff8-ea1d1a690b54]: failed with code=1006, reason=] [solo:node:setup] *********************************** ERROR ***************************************** [solo:node:setup] Error in setting up nodes: copyTo[network-node1-0,ed951421-cade-4757-9ff8-ea1d1a690b54]: failed with code=1006, reason= [solo:node:setup] SoloError: Error in setting up nodes: copyTo[network-node1-0,ed951[421](https://github.com/swirldslabs/performance-analysis-automation/actions/runs/13277159102/job/37068760687#step:20:422)-cade-4757-9ff8-ea1d1a690b54]: failed with code=1006, reason= [solo:node:setup] at file:///home/runner/_work/performance-analysis-automation/performance-analysis-automation/solo/dist/src/commands/base.js:166:23 [solo:node:setup] at process.processTicksAndRejections (node:internal/process/task_queues:105:5) [solo:node:setup] at async NodeCommandHandlers.setup (file:///home/runner/_work/performance-analysis-automation/performance-analysis-automation/solo/dist/src/commands/node/handlers.js:576:9) [solo:node:setup] Caused by: SoloError: copyTo[network-node1-0,ed951421-cade-4757-9ff8-ea1d1a690b54]: failed with code=1006, reason= [solo:node:setup] at K8ClientContainer.exitWithError (file:///home/runner/_work/performance-analysis-automation/performance-analysis-automation/solo/dist/src/core/kube/k8_client/k8_client_container.js:335:36) [solo:node:setup] at WebSocket.<anonymous> (file:///home/runner/_work/performance-analysis-automation/performance-analysis-automation/solo/dist/src/core/kube/k8_client/k8_client_container.js:155:41) [solo:node:setup] at WebSocket.emit (node:events:519:28) [solo:node:setup] at WebSocket.emitClose (/home/runner/_work/performance-analysis-automation/performance-analysis-automation/solo/node_modules/ws/lib/websocket.js:272:10) [solo:node:setup] at Receiver.receiverOnFinish (/home/runner/_work/performance-analysis-automation/performance-analysis-automation/solo/node_modules/ws/lib/websocket.js:1209:20) [solo:node:setup] at Receiver.emit (node:events:519:28) [solo:node:setup] at finish (node:internal/streams/writable:955:10) [solo:node:setup] at node:internal/streams/writable:936:13 [solo:node:setup] at process.processTicksAndRejections (node:internal/process/task_queues:90:21) [solo:node:setup] [solo:node:setup] Caused by: copyTo[network-node1-0,ed951421-cade-4757-9ff8-ea1d1a690b54]: failed with code=1006, reason= [solo:node:setup] SoloError: copyTo[network-node1-0,ed951421-cade-4757-9ff8-ea1d1a690b54]: failed with code=1006, reason= [solo:node:setup] at K8ClientContainer.exitWithError (file:///home/runner/_work/performance-analysis-automation/performance-analysis-automation/solo/dist/src/core/kube/k8_client/k8_client_container.js:335:36) [solo:node:setup] at WebSocket.<anonymous> (file:///home/runner/_work/performance-analysis-automation/performance-analysis-automation/solo/dist/src/core/kube/k8_client/k8_client_container.js:155:41) [solo:node:setup] at WebSocket.emit (node:events:519:28) [solo:node:setup] at WebSocket.emitClose (/home/runner/_work/performance-analysis-automation/performance-analysis-automation/solo/node_modules/ws/lib/websocket.js:272:10) [solo:node:setup] at Receiver.receiverOnFinish (/home/runner/_work/performance-analysis-automation/performance-analysis-automation/solo/node_modules/ws/lib/websocket.js:1209:20) [solo:node:setup] at Receiver.emit (node:events:519:28) [solo:node:setup] at finish (node:internal/streams/writable:955:10) [solo:node:setup] at node:internal/streams/writable:936:13 [solo:node:setup] at process.processTicksAndRejections (node:internal/process/task_queues:90:21) [solo:node:setup] [solo:node:setup] *********************************************************************************** task: Failed to run task "default": exit status 1all logs for this case: https://perf.analytics.eng.hashgraph.io/ephemeral/test_runs/main_Latitude4_326/solo_deploy/
Another case: https://github.com/swirldslabs/performance-analysis-automation/actions/runs/13276666450/job/37067391868#step:20:424
with logs: https://perf.analytics.eng.hashgraph.io/ephemeral/test_runs/main_Latitude3_24/solo_deploy/
The sequential mode one-by-one does not help.
If retried, it may fail with another node
Describe the expected behavior
Solo deploy local build of Hedera must deploy with exit code=0
To Reproduce
Reproducible on any random Latitude cluster
Additional Context
No response
The text was updated successfully, but these errors were encountered: