Upgraded Xen-tools from 4.15 to 4.19.0 #4186

roja-zededa · 2024-08-29T19:41:22Z

Able to add ninja through apk in xen-tools/Docker and xen/Docker
Changed the seabios and Xen version to 1.16.3 and 4.19.0 respectively
Added 12-remove-vanillaqemu4.19-cpupinning.patch which removes new qemu_thread_set_affinity implementation (QEMU 8.0.4), also retained Nikolay CPU Pinning PatchNo.15
Warning treated as error flag is available by default so removed 12-disable-Werror-to-build-under-gcc-11.2.patch
NetdevTapOptions doesn't have has_br member so changed 10-bridge-helper-support.patch
Bydefault vhost-vsock and vhost-scsi is enabled so removing the corresponding enable flags from xen-tools/Dockerfile
Removed 11-char-socket-revert.patch as it's unnecessary
Removed [realtime] option from kvm.go and replaced it with [overcommit], hypervisor.go unit test need to be changed to reflect [overcommit]
08-Revert__Revert__vfio_pci-quirks_c__Disable_stolen_memory_for_igd_VFIO__.patch looked super messy, so cleaned it.
Replaced [realtime] with [overcommit] in kvm_test.go for the unit test case to pass

roja-zededa · 2024-08-29T20:33:30Z

@OhmSpectator @rene @rouming Fixed unit test case, looks good to me. Please merge this when you can.

P.S. Had to move #4133 here because of the Request Code Owners Review / auto_request_review error.

shjala · 2024-08-29T20:52:49Z

@OhmSpectator @rene @rouming Fixed unit test case, looks good to me. Please merge this when you can.

P.S. Had to move #4133 here because of the Request Code Owners Review / auto_request_review error.

There are still unanswered question/change request in the original PR.

OhmSpectator · 2024-08-29T23:59:13Z

@OhmSpectator @rene @rouming Fixed unit test case, looks good to me. Please merge this when you can.
P.S. Had to move #4133 here because of the Request Code Owners Review / auto_request_review error.

There are still unanswered question/change request in the original PR.

Do you mean this one? #4133 (comment)
or something else?

shjala · 2024-08-30T05:02:26Z

@OhmSpectator @rene @rouming Fixed unit test case, looks good to me. Please merge this when you can.
P.S. Had to move #4133 here because of the Request Code Owners Review / auto_request_review error.

There are still unanswered question/change request in the original PR.

Do you mean this one? #4133 (comment) or something else?

#4133 (comment)

rouming · 2024-08-30T07:16:20Z

@roja-zededa we usually don't merge all the patches in one commit: difficult to review, difficult to revert, difficult to maintain. Each logical change goes to its own commit. For example test fixes is ideal candidate for a separate commit with its own good description why the test needs a fix; or removal of old patches, or docker changes - those are good candidates for a separate commits. We try to follow Linux kernel best practices and those are well described here: https://github.com/torvalds/linux/blob/master/Documentation/process/submitting-patches.rst#separate-your-changes

Update: your PR description consists of 10 sentences, which perfectly illustrate the "logical change" approach. Of course there should not be exact 10 commits, but some of them definitely "want" to be separated.

pkg/xen/Dockerfile

pkg/pillar/hypervisor/kvm.go

eriknordmark

There are still some comments here and from #4133 to respond to, but kicking off tests.

OhmSpectator · 2024-09-03T09:09:00Z

There are still some comments here and from #4133 to respond to, but kicking off tests.

It makes sense to rebase on the latest master before kicking the tests.

roja-zededa · 2024-09-03T18:06:30Z

@OhmSpectator @rene @rouming Fixed unit test case, looks good to me. Please merge this when you can.
P.S. Had to move #4133 here because of the Request Code Owners Review / auto_request_review error.

There are still unanswered question/change request in the original PR.

Do you mean this one? #4133 (comment) or something else?

#4133 (comment)

@shjala I have only added libgcc so I didn't know why there were sudo, bash, yajl. But, I can successfully build an image without these utils. @rene Can I remove sudo, bash, yajl from Dockerfile?

roja-zededa · 2024-09-03T20:09:31Z

There are still some comments here and from #4133 to respond to, but kicking off tests.

It makes sense to rebase on the latest master before kicking the tests.

Tested with the latest master, and no issues were found.

rouming · 2024-09-23T19:58:14Z

@roja-zededa This is the source of the error: https://github.com/qemu/qemu/blob/01dc65a3bc262ab1bec8fe89775e9bbfa627becb/ui/vnc.c#L4069

The problem should be reproduced by using vnc and password. Try to follow build recommendations from here: https://bugs.gentoo.org/832494

eriknordmark · 2024-09-24T05:22:23Z

@roja-zededa This is the source of the error: https://github.com/qemu/qemu/blob/01dc65a3bc262ab1bec8fe89775e9bbfa627becb/ui/vnc.c#L4069

Is that "Cipher backend does not support DES algorithm" check new in the new version of qemu?

rouming · 2024-09-24T05:41:31Z

Apparently something was changed, for example qemu/qemu@83bee4b
Either in the code, either in qemu build system, which (only a guess) does not enable a few libraries by default needed by vnc. But having an explicit error it is easy to prove: configure vnc with a password, run, tweak qemu build, repeat.

milan-zededa · 2024-09-24T09:32:23Z

@Roja-Eswaran You can run locally these commands to reproduce:

make clean && make build-tests
./eden config add default
./eden config set default --key eve.tag --value 0.0.0-xentools-bump-8843ff1f
./eden config set default --key eve.log-level --value debug
./eden setup
./dist/bin/eden+ports.sh 2223:2223 2224:2224 
./eden start
./eden eve onboard
./eden test tests/vnc -e vnc_test -v debug

# later run:
./eden pod logs vnc-app

roja-zededa · 2024-09-24T21:15:16Z

Apparently something was changed, for example qemu/qemu@83bee4b Either in the code, either in qemu build system, which (only a guess) does not enable a few libraries by default needed by vnc. But having an explicit error it is easy to prove: configure vnc with a password, run, tweak qemu build, repeat.

This issue has been resolved by adding gnutls and gnutls-dev in Xen-tools/Dockerfile.

eriknordmark

Kick off tests

eriknordmark · 2024-09-25T10:13:21Z

@Roja-Eswaran please rebase on master (to pick up #4290) so we can get the build to work again.

Upgraded xen/Dockerfile and xen-tools/Dockerfile to point new version (1) Removed the following unused patches: 11-char-socket-revert.patch, 12-disable-Werror-to-build-under-gcc-11.2.patch, 15-qemu-Set-the-affinity-of-QEMU-threads-according-to-t.patch, 16-imammedo_x86_acpi_use_offset_instead_of_pointer_when_using_build_header.patch, 0003-arch-arm-small-hack-for-rpi4-usb.patch Signed-off-by: Roja Eswaran <[email protected]>

Patch 12-remove-vanillaqemu4.19-cpupinning.patch which removes new qemu_thread_set_affinity implementation (QEMU 8.0.4) Patch 20-return_zero_close_if_special_file.patch which removes a CVE fix Signed-off-by: Roja Eswaran <[email protected]>

Replacing [realtime] with [overcommit] as it's deprecated in QEMU 8.0.4 provided by xen-tools 4.19.0 Signed-off-by: Roja Eswaran <[email protected]>

Adding gnutls and gnutls-dev in xen-tools/Dockerfile for VNC error Signed-off-by: Roja Eswaran <[email protected]>

roja-zededa · 2024-09-25T16:08:02Z

@Roja-Eswaran please rebase on master (to pick up #4290) so we can get the build to work again.

Done! Please kickoff the tests.

roja-zededa · 2024-09-25T21:26:39Z

I am unable to launch native containers. The eden test failed again! I was able to reproduce it locally and here is the log:

content: {"file":"/pillar/cmd/zedagent/handleconfig.go:824","func":"github.com/lf-edge/eve/pkg/pillar/cmd/zedagent.requestConfigByURL","level":"error","msg":"inhaleDeviceConfig failed: 4","pid":2022,"source":"zedagent","time":"2024-09-25T21:23:05.974509884Z"}
severity: error

content: {"file"`:"/pillar/hypervisor/qmp.go:202","func":"github.com/lf-edge/eve/pkg/pillar/hypervisor.qmpEventHandler","level":"error","msg":"qmpEventHandler: Exception while stopping domain with socket: /run/hypervisor/kvm/1c868ce6-f417-4281-90ab-f98aec092c81.1.1/qmp. write unix @-\u003e/run/hypervisor/kvm/1c868ce6-f417-4281-90ab-f98aec092c81.1.1/qmp: write: broken pipe","pid":2022,"source":"zedbox","time":"2024-09-25T21:23:06.521514741Z"}
severity: error

content: {"file":"/pillar/hypervisor/qmp.go:205","func":"github.com/lfedge/eve/pkg/pillar/hypervisor.qmpEventHandler","level":"error","msg":"qmpEventHandler: Exception while quitting domain with socket: /run/hypervisor/kvm/1c868ce6-f417-4281-90ab-f98aec092c81.1.1/qmp. dial unix /run/hypervisor/kvm/1c868ce6-f417-4281-90ab-f98aec092c81.1.1/qmp: connect: no such file or directory","pid":2022,"source":"zedbox","time":"2024-09-25T21:23:10.527364286Z"}
severity: error

content: {"file":"/pillar/hypervisor/qmp.go:193","func":"github.com/lf-edge/eve/pkg/pillar/hypervisor.qmpEventHandler","level":"error","msg":"qmpEventHandler: Exception while accessing listenerSocket: /run/hypervisor/kvm/1c868ce6-f417-4281-90ab-f98aec092c81.1.1/listener.qmp. stat /run/hypervisor/kvm/1c868ce6-f417-4281-90ab-f98aec092c81.1.1/listener.qmp: no such file or directory","pid":2022,"source":"zedbox","time":"2024-09-25T21:23:10.527630294Z"}
severity: error

content: {"file":"/pillar/hypervisor/hypervisor.go:155","func":"github.com/lf-edge/eve/pkg/pillar/hypervisor.logError","level":"error","msg":"containerd looking up domain 1c868ce6-f417-4281-90ab-f98aec092c81.1.1 resulted in CtrContainerInfo: couldn't load container 1c868ce6-f417-4281-90ab-f98aec092c81.1.1: CtrLoadContainer: Exception while loading container: container \"1c868ce6-f417-4281-90ab-f98aec092c81.1.1\" in namespace \"eve-user-apps\": not found","pid":2022,"source":"zedbox","time":"2024-09-25T21:23:26.053377236Z"}
severity: error

content: {"file":"/pillar/hypervisor/hypervisor.go:155","func":"github.com/lf-edge/eve/pkg/pillar/hypervisor.logError","level":"error","msg":"containerd looking up domain 1c868ce6-f417-4281-90ab-f98aec092c81.1.1 resulted in CtrContainerInfo: couldn't load container 1c868ce6-f417-4281-90ab-f98aec092c81.1.1: CtrLoadContainer: Exception while loading container: container \"1c868ce6-f417-4281-90ab-f98aec092c81.1.1\" in namespace \"eve-user-apps\": not found","pid":2022,"source":"zedbox","time":"2024-09-25T21:23:41.852764882Z"}

FWIW: AMD64/KVM, Sometimes the zedcontroller throws me this error while launching native container:

failed to create task: new task failed: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: brctl: bridge bn1: Resource busy
brctl: bridge bn1: Resource busy
brctl: bridge bn1: Resource busy
Cannot find device "nbu1x2.1": unknown

rene · 2024-09-26T07:59:47Z

I am unable to launch native containers. The eden test failed again! I was able to reproduce it locally and here is the log:

content: {"file":"/pillar/cmd/zedagent/handleconfig.go:824","func":"github.com/lf-edge/eve/pkg/pillar/cmd/zedagent.requestConfigByURL","level":"error","msg":"inhaleDeviceConfig failed: 4","pid":2022,"source":"zedagent","time":"2024-09-25T21:23:05.974509884Z"}
severity: error

content: {"file"`:"/pillar/hypervisor/qmp.go:202","func":"github.com/lf-edge/eve/pkg/pillar/hypervisor.qmpEventHandler","level":"error","msg":"qmpEventHandler: Exception while stopping domain with socket: /run/hypervisor/kvm/1c868ce6-f417-4281-90ab-f98aec092c81.1.1/qmp. write unix @-\u003e/run/hypervisor/kvm/1c868ce6-f417-4281-90ab-f98aec092c81.1.1/qmp: write: broken pipe","pid":2022,"source":"zedbox","time":"2024-09-25T21:23:06.521514741Z"}
severity: error

content: {"file":"/pillar/hypervisor/qmp.go:205","func":"github.com/lfedge/eve/pkg/pillar/hypervisor.qmpEventHandler","level":"error","msg":"qmpEventHandler: Exception while quitting domain with socket: /run/hypervisor/kvm/1c868ce6-f417-4281-90ab-f98aec092c81.1.1/qmp. dial unix /run/hypervisor/kvm/1c868ce6-f417-4281-90ab-f98aec092c81.1.1/qmp: connect: no such file or directory","pid":2022,"source":"zedbox","time":"2024-09-25T21:23:10.527364286Z"}
severity: error

content: {"file":"/pillar/hypervisor/qmp.go:193","func":"github.com/lf-edge/eve/pkg/pillar/hypervisor.qmpEventHandler","level":"error","msg":"qmpEventHandler: Exception while accessing listenerSocket: /run/hypervisor/kvm/1c868ce6-f417-4281-90ab-f98aec092c81.1.1/listener.qmp. stat /run/hypervisor/kvm/1c868ce6-f417-4281-90ab-f98aec092c81.1.1/listener.qmp: no such file or directory","pid":2022,"source":"zedbox","time":"2024-09-25T21:23:10.527630294Z"}
severity: error

content: {"file":"/pillar/hypervisor/hypervisor.go:155","func":"github.com/lf-edge/eve/pkg/pillar/hypervisor.logError","level":"error","msg":"containerd looking up domain 1c868ce6-f417-4281-90ab-f98aec092c81.1.1 resulted in CtrContainerInfo: couldn't load container 1c868ce6-f417-4281-90ab-f98aec092c81.1.1: CtrLoadContainer: Exception while loading container: container \"1c868ce6-f417-4281-90ab-f98aec092c81.1.1\" in namespace \"eve-user-apps\": not found","pid":2022,"source":"zedbox","time":"2024-09-25T21:23:26.053377236Z"}
severity: error

content: {"file":"/pillar/hypervisor/hypervisor.go:155","func":"github.com/lf-edge/eve/pkg/pillar/hypervisor.logError","level":"error","msg":"containerd looking up domain 1c868ce6-f417-4281-90ab-f98aec092c81.1.1 resulted in CtrContainerInfo: couldn't load container 1c868ce6-f417-4281-90ab-f98aec092c81.1.1: CtrLoadContainer: Exception while loading container: container \"1c868ce6-f417-4281-90ab-f98aec092c81.1.1\" in namespace \"eve-user-apps\": not found","pid":2022,"source":"zedbox","time":"2024-09-25T21:23:41.852764882Z"}

Only these logs are not enough to know the root cause of the crash.

FWIW: AMD64/KVM, Sometimes the zedcontroller throws me this error while launching native container:

failed to create task: new task failed: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: brctl: bridge bn1: Resource busy
brctl: bridge bn1: Resource busy
brctl: bridge bn1: Resource busy
Cannot find device "nbu1x2.1": unknown

This error message happens a lot while deploying native containers when running EVE device on QEMU (I think it takes too long to setup the bridge device in this case). However, the virt. ethernet hook script retries a couple of times, so usually it succeeds after the second retry. If it manages to run the container, you don't need to worry about it....

rene

Kicking off Eden Tests...

milan-zededa · 2024-09-26T16:34:45Z

Eden tests are green. Should we merge?
@rouming You will need to remove your "changes requested" for merge to be allowed.

eriknordmark

Re-running the same qemu device as before
make -e HV=xen ACCEL=y QEMU_MEMORY=16384 MEDIA_SIZE=65536 SSH_PORT=2323 pkg/pillar pkg/watchdog rootfs live run

where the controller is deploying two VMs and two containers, the alpine container fails after 5 minutes, This is not happening when I run the same thing on master, nor when I run it with KVM on this PR. So that failure needs to be investigated.

roja-zededa · 2024-09-27T17:51:20Z

@eriknordmark Could you please share the error trace?

roja-zededa · 2024-09-27T20:21:44Z

@eriknordmark I don't face any issue while launching three VMs and three native containers at the same time on this PR. https://zedcontrol.alpha.zededa.net/edge-nodes/223c376d-242c-4da2-8487-253c6d599c07/details/status. Could you please share the error log messages so that I can reproduce the issue locally?

eriknordmark · 2024-09-30T12:04:59Z

@eriknordmark I don't face any issue while launching three VMs and three native containers at the same time on this PR. https://zedcontrol.alpha.zededa.net/edge-nodes/223c376d-242c-4da2-8487-253c6d599c07/details/status. Could you please share the error log messages so that I can reproduce the issue locally?

I sent you a pointer to the Kibana logs when running xen-amd64.

I kicked off a run with the kvm image from your PR and today I see some new qmp errors (didn't see those a week ago)
FailureInfo: Error OperState. Error Details: [{'description': "Giving up waiting to connect to QEMU Monitor Protocol socket /run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp from VM 3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1, error: [attempt 1] qmp status failed for QMP socket '/run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp': err: 'dial unix /run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp: connect: connection refused'; (JSON response: ''); [attempt 2] qmp status failed for QMP socket '/run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp': err: 'dial unix /run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp: connect: connection refused'; (JSON response: ''); [attempt 3] qmp status failed for QMP socket '/run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp': err: 'dial unix /run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp: connect: connection refused'; (JSON response: '')", 'severity': 'SEVERITY_ERROR', 'timestamp': '2024-09-30T09:17:07.254882353Z', 'entities': [], 'retryCondition': ''}]

Do you want a kibana saved search for those as well? The kvm error is for a device called sc-supermicro-zc2 in the alpha cluster.

roja-zededa · 2024-09-30T13:59:20Z

@eriknordmark I don't face any issue while launching three VMs and three native containers at the same time on this PR. https://zedcontrol.alpha.zededa.net/edge-nodes/223c376d-242c-4da2-8487-253c6d599c07/details/status. Could you please share the error log messages so that I can reproduce the issue locally?

I sent you a pointer to the Kibana logs when running xen-amd64.

I kicked off a run with the kvm image from your PR and today I see some new qmp errors (didn't see those a week ago) FailureInfo: Error OperState. Error Details: [{'description': "Giving up waiting to connect to QEMU Monitor Protocol socket /run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp from VM 3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1, error: [attempt 1] qmp status failed for QMP socket '/run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp': err: 'dial unix /run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp: connect: connection refused'; (JSON response: ''); [attempt 2] qmp status failed for QMP socket '/run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp': err: 'dial unix /run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp: connect: connection refused'; (JSON response: ''); [attempt 3] qmp status failed for QMP socket '/run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp': err: 'dial unix /run/hypervisor/kvm/3c9007f2-4907-4722-ac23-0ed61a0111fe.1.1/qmp: connect: connection refused'; (JSON response: '')", 'severity': 'SEVERITY_ERROR', 'timestamp': '2024-09-30T09:17:07.254882353Z', 'entities': [], 'retryCondition': ''}]

Do you want a kibana saved search for those as well? The kvm error is for a device called sc-supermicro-zc2 in the alpha cluster.

@eriknordmark I don't think so my PR is responsible for sc-supermicro-zc2 failure as I have seen a similar error in PR:4259 as well.
Thanks for sending the pointers w.r.t. Xen-amd64, I'll take a look at it.

eriknordmark

It does look like some container (the alpine-cont we use in ztest) can have issues with a 9p mount when run on the xen hypervisor. But let's debug that in parallel with merging this (assuming no test issues discovered overnight for the kvm image).

I've seen a panic in the guest VM kernel (in p9_xen_create) which looks like an old guest kernel (missing a check for addr == NULL which is in the 6.1 kernel). So not an issue with this PR.

roja-zededa requested review from OhmSpectator, rene and rouming as code owners August 29, 2024 19:41

github-actions bot requested review from eriknordmark, rucoder and shjala August 29, 2024 19:41

roja-zededa force-pushed the xentools-bump branch 4 times, most recently from 6d45ae7 to 7280a02 Compare August 29, 2024 20:01

roja-zededa mentioned this pull request Aug 29, 2024

Upgraded Xen-tools from 4.15 to 4.19.0 #4133

Closed

rene reviewed Aug 30, 2024

View reviewed changes

pkg/xen/Dockerfile Show resolved Hide resolved

eriknordmark reviewed Sep 3, 2024

View reviewed changes

pkg/pillar/hypervisor/kvm.go Show resolved Hide resolved

eriknordmark approved these changes Sep 3, 2024

View reviewed changes

github-actions bot requested review from eriknordmark and rene September 3, 2024 18:08

roja-zededa force-pushed the xentools-bump branch from 1b25649 to 7280a02 Compare September 3, 2024 18:13

roja-zededa closed this Sep 3, 2024

roja-zededa force-pushed the xentools-bump branch from 7280a02 to 80611a1 Compare September 3, 2024 18:30

roja-zededa reopened this Sep 3, 2024

roja-zededa force-pushed the xentools-bump branch 2 times, most recently from df6005f to 6d9e7fa Compare September 3, 2024 20:34

roja-zededa force-pushed the xentools-bump branch 3 times, most recently from 038f0df to 949b192 Compare September 24, 2024 20:36

eriknordmark approved these changes Sep 25, 2024

View reviewed changes

roja-zededa added 4 commits September 25, 2024 09:06

Replace: Deprecated QEMU parameters

310b155

Replacing [realtime] with [overcommit] as it's deprecated in QEMU 8.0.4 provided by xen-tools 4.19.0 Signed-off-by: Roja Eswaran <[email protected]>

Add: pkgs gnutls

9647550

Adding gnutls and gnutls-dev in xen-tools/Dockerfile for VNC error Signed-off-by: Roja Eswaran <[email protected]>

roja-zededa force-pushed the xentools-bump branch from 58630ff to 9647550 Compare September 25, 2024 16:06

github-actions bot requested a review from eriknordmark September 25, 2024 16:07

rene approved these changes Sep 26, 2024

View reviewed changes

eriknordmark requested changes Sep 26, 2024

View reviewed changes

eriknordmark approved these changes Sep 30, 2024

View reviewed changes

rouming approved these changes Oct 1, 2024

View reviewed changes

eriknordmark merged commit 8776e5d into lf-edge:master Oct 1, 2024
69 of 77 checks passed

rouming mentioned this pull request Oct 1, 2024

Bump up XEN version to 4.18.0 #3534

Closed

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgraded Xen-tools from 4.15 to 4.19.0 #4186

Upgraded Xen-tools from 4.15 to 4.19.0 #4186

roja-zededa commented Aug 29, 2024

roja-zededa commented Aug 29, 2024

shjala commented Aug 29, 2024

OhmSpectator commented Aug 29, 2024

shjala commented Aug 30, 2024

rouming commented Aug 30, 2024 •

edited

Loading

eriknordmark left a comment

OhmSpectator commented Sep 3, 2024

roja-zededa commented Sep 3, 2024

roja-zededa commented Sep 3, 2024

rouming commented Sep 23, 2024

eriknordmark commented Sep 24, 2024

rouming commented Sep 24, 2024

milan-zededa commented Sep 24, 2024

roja-zededa commented Sep 24, 2024

eriknordmark left a comment

eriknordmark commented Sep 25, 2024

roja-zededa commented Sep 25, 2024 •

edited

Loading

roja-zededa commented Sep 25, 2024 •

edited

Loading

rene commented Sep 26, 2024 •

edited

Loading

rene left a comment

milan-zededa commented Sep 26, 2024

eriknordmark left a comment

roja-zededa commented Sep 27, 2024

roja-zededa commented Sep 27, 2024

eriknordmark commented Sep 30, 2024

roja-zededa commented Sep 30, 2024

eriknordmark left a comment •

edited

Loading

Upgraded Xen-tools from 4.15 to 4.19.0 #4186

Upgraded Xen-tools from 4.15 to 4.19.0 #4186

Conversation

roja-zededa commented Aug 29, 2024

roja-zededa commented Aug 29, 2024

shjala commented Aug 29, 2024

OhmSpectator commented Aug 29, 2024

shjala commented Aug 30, 2024

rouming commented Aug 30, 2024 • edited Loading

eriknordmark left a comment

Choose a reason for hiding this comment

OhmSpectator commented Sep 3, 2024

roja-zededa commented Sep 3, 2024

roja-zededa commented Sep 3, 2024

rouming commented Sep 23, 2024

eriknordmark commented Sep 24, 2024

rouming commented Sep 24, 2024

milan-zededa commented Sep 24, 2024

roja-zededa commented Sep 24, 2024

eriknordmark left a comment

Choose a reason for hiding this comment

eriknordmark commented Sep 25, 2024

roja-zededa commented Sep 25, 2024 • edited Loading

roja-zededa commented Sep 25, 2024 • edited Loading

rene commented Sep 26, 2024 • edited Loading

rene left a comment

Choose a reason for hiding this comment

milan-zededa commented Sep 26, 2024

eriknordmark left a comment

Choose a reason for hiding this comment

roja-zededa commented Sep 27, 2024

roja-zededa commented Sep 27, 2024

eriknordmark commented Sep 30, 2024

roja-zededa commented Sep 30, 2024

eriknordmark left a comment • edited Loading

Choose a reason for hiding this comment

rouming commented Aug 30, 2024 •

edited

Loading

roja-zededa commented Sep 25, 2024 •

edited

Loading

roja-zededa commented Sep 25, 2024 •

edited

Loading

rene commented Sep 26, 2024 •

edited

Loading

eriknordmark left a comment •

edited

Loading