Connection to libvirt via SSH fails with provider version 0.6.9

[soerenschneider]

System Information

Linux distribution

Fedora 34

Terraform version

Terraform v1.0.1
on linux_amd64

Provider and libvirt versions

provider: v0.6.9
libvirt (remote, debian stable): 5.0.0-4+deb10u1

---

Checklist

• Is your issue/contribution related with enabling some setting/option exposed by libvirt that the plugin does not yet support, or requires changing/extending the provider terraform schema?

  • Make sure you explain why this option is important to you, why it should be important to everyone. Describe your use-case with detail and provide examples where possible.
  • If it is a very special case, consider using the XSLT support in the provider to tweak the definition instead of opening an issue
  • Maintainers do not have expertise in every libvirt setting, so please, describe the feature and how it is used. Link to the appropriate documentation

• Is it a bug or something that does not work as expected? Please make sure you fill the version information below:

Description of Issue/Question

Connecting to a remote libvirt provider via SSH fails after updating to v0.6.9 with the error message Error: failed to dial libvirt: failed to read ssh key: open /home/soeren/.ssh/id_rsa: no such file or directory. I'm indeed not using RSA keypairs anymore, instead I've been using ed25519 keypairs. Reverting back to v0.6.3 of the provider makes it work again. For the given remote host there's no additional settings in the local ~/.ssh/config file.

Setup

terraform {
  required_version = ">= 0.15"
  required_providers {
    libvirt = {
      source  = "dmacvicar/libvirt"
      version = "0.6.9"
    }
  }
}

provider "libvirt" {
  uri = "qemu+ssh://[email protected]/system"
}

resource "libvirt_domain" "terraform_test" {
  name = "terraform_test"
}

Steps to Reproduce Issue

➜ terraform plan -no-color

Error: failed to dial libvirt: failed to read ssh key: open /home/soeren/.ssh/id_rsa: no such file or directory

  with provider["registry.terraform.io/dmacvicar/libvirt"],
  on providers.tf line 12, in provider "libvirt":
  12: provider "libvirt" {

...after changing the provider version to 0.6.3 again

➜ terraform init -upgrade

Initializing the backend...

Initializing provider plugins...
- Finding dmacvicar/libvirt versions matching "~> 0.6.3"...
- Using previously-installed dmacvicar/libvirt v0.6.3

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
➜ terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
  + create

Terraform will perform the following actions:

  # libvirt_domain.terraform_test will be created
  + resource "libvirt_domain" "terraform_test" {
      + arch        = (known after apply)
      + emulator    = (known after apply)
      + fw_cfg_name = "opt/com.coreos/config"
      + id          = (known after apply)
      + machine     = (known after apply)
      + memory      = 512
      + name        = "terraform_test"
      + qemu_agent  = false
      + running     = true
      + vcpu        = 1
    }

Plan: 1 to add, 0 to change, 0 to destroy.

---

Additional information:

SELinux is enabled on the host running the terraform code, however, disabling SELinux doesn't fix the issue.

[pmaene]

I ran into similar issues yesterday. You can (temporarily?) specify the location of the private as a query parameter in the connection URI. Assuming you are using the default filename for an ed25519 key, your URI would become something like this.

qemu+ssh://[email protected]/system?keyfile=/home/soeren/.ssh/id_ed25519

[soerenschneider]

@pmaene Thanks for the workaround, you guessed the filename correctly, unfortunately I get another error: Error: failed to dial libvirt: failed to parse ssh key: ssh: cannot decode encrypted private keys.

My best guess is that it's because my key has a passphrase set, but terraform isn't asking for it, instead it just fails. I can maybe try to test it later with a temporary ed25519 key without a passphrase. Is your key having a passphrase set?

[pmaene]

You're right that this error has to do with the fact that your key has a passphrase set. Using a temporary key without one set should allow you to continue.

[oranenj]

Can the golang SSH client not use the SSH agent? SSH keys without passphrases are a bad idea, and generating a temporary one just to run Terraform makes for an unpleasant user experience.

[pmaene]

I don't necessarily agree that keys without a passphrase are always a bad idea (see for instance this tweet). However, I was also thinking that adding SSH agent support would resolve both the private key discovery and passphrase issues. Go's x/crypto/ssh package does have APIs for this.

[oranenj]

@pmaene that particular tweet only applies to situations where the SSH keys don't give you access anywhere interesting :-) I regularly use systems (including my current desktop, mostly) where I wouldn't care much about the contents of my home directory leaking except for the SSH keys because they give access to the places where all the interesting data is stored.

To get back on topic, you can also work around this by using netcat manually. First run something like this:
nc -kl -c 'ssh vmhost sudo nc -U /var/run/libvirt/libvirt-sock' localhost 5000 and then use qemu+tcp://localhost:5000/system as the URI.

[pmaene]

@oranenj As with anything security-related, it always depends on your attacker model ☺️. My usage aligns with the one described in that tweet, so I've never really seen the value of password-protected SSH keys. Sorry if I my comment came across differently.

Thanks for the netcat-based workaround, that's a great tip.

[dmacvicar]

When I got rid of the libvirt client library, I had to write the ssh transport from scratch. I followed the documentation.

I don't think I implemented the sshauth part as described here, but with some guidance I think we can get it done:
https://libvirt.org/uri.html

[oranenj]

@dmacvicar Looking at the documentation for the ssh library, it doesn't seem too difficult to add support for specifying SSH auth methods so I might take a stab at it if you didn't already get to it.

[dmacvicar]

Give it a try. Everything is in the libvirt/uri subdirectory, with one file per transport.

Only thing I'd like to keep in mind is to be as close as possible in behavior to the C libvirt client.

[limburgher]

Seeing this on Fedora 34 with 0.6.10.

[pahansen95]

I'm also seeing similar behavior on MacOS 11.5.

My URI Connection string is qemu+ssh://[email protected]/system?socket=/var/run/libvirt/libvirt-sock&keyfile=/Users/peter/.ssh/id_ed25519.

That connection string works with virsh but not when using the Terraform provider.

Here is the Terraform Output

╷
│ Error: failed to dial libvirt: failed to connect to libvirt on the remote host: ssh: rejected: connect failed (open failed)
│
│   with provider["registry.terraform.io/dmacvicar/libvirt"],
│   on main.tf line 10, in provider "libvirt":
│   10: provider "libvirt" {
│
╵

WorkAround

I forwarded the unix port using SSH.

note for some you might need to enable Agent Forwarding in your /etc/ssh/sshd_conf file. I had to comment out my DisableForwarding yes line.

Here is the script I use.

#!/usr/bin/env bash

set -xe

cleanup() {
    for pid in $(pgrep ssh \-fnNT); do kill -3 "$pid"; done
    rm $1
}

libvirt_sock="$TMPDIR/libvirt-sock"
ssh -fnNT -L "$libvirt_sock":/var/run/libvirt/libvirt-sock -i ~/.ssh/id_ed25519 [email protected]
trap "cleanup $libvirt_sock" ERR EXIT
export LIBVIRT_DEFAULT_URI="qemu+unix:///session?socket=$libvirt_sock"

#Verify Connection
virsh -c "$LIBVIRT_DEFAULT_URI" sysinfo >/dev/null

#Use terraform
for cmd in init plan apply; do
    terraform "$cmd"
done
trap "terraform destroy" EXIT ERR 

echo "Press Return to Quit & Cleanup..."
read

[putchar]

Hello.
I think i am facing a similar issue as of now

Here is my tf configuration

terraform {
  required_version = "1.0.4"

  backend "http" {
    address = "https://gitlab.com/api/v4/projects/PROJECT_ID/terraform/state/dedibox-1"
    lock_address="https://gitlab.com/api/v4/projects/PROJECT_ID/terraform/state/dedibox-1/lock"
    unlock_address="https://gitlab.com/api/v4/projects/PROJECT_ID/terraform/state/dedibox-1/lock"
    username="putchar"
    password="MYPASS"
    lock_method="POST"
    unlock_method="DELETE"
    retry_wait_min="5"
  }

  required_providers {
    libvirt = {
      source = "dmacvicar/libvirt"
      version = "0.6.10"
    }
  }
}

provider "libvirt" {
  uri = "qemu+ssh://putchar@dedibox-1/system"
}


resource "libvirt_domain" "terraform_test" {
  name = "terraform_test"
}

and this is what i see

[ λ infra/terraform/dedibox-1 ]
19:05 $ terraform  init -reconfigure

Initializing the backend...

Successfully configured the backend "http"! Terraform will automatically
use this backend unless the backend configuration changes.
2021/08/13 19:08:10 [DEBUG] GET https://gitlab.com/api/v4/projects/PROJECT_ID/terraform/state/dedibox-1

Initializing provider plugins...
- Reusing previous version of dmacvicar/libvirt from the dependency lock file
- Using previously-installed dmacvicar/libvirt v0.6.10

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

19:08 $ terraform plan -out tf.plan
2021/08/13 19:08:17 [DEBUG] POST https://gitlab.com/api/v4/projects/PROJECT_ID/terraform/state/dedibox-1/lock
2021/08/13 19:08:17 [DEBUG] GET https://gitlab.com/api/v4/projects/PROJECT_ID/terraform/state/dedibox-1
╷
│ Error: failed to dial libvirt: ssh: handshake failed: knownhosts: key mismatch
│
│   with provider["registry.terraform.io/dmacvicar/libvirt"],
│   on 00-terraform.tf line 23, in provider "libvirt":
│   23: provider "libvirt" {
│
╵
2021/08/13 19:08:18 [DEBUG] DELETE https://gitlab.com/api/v4/projects/PROJECT_ID/terraform/state/dedibox-1/lock

my ssh key has no passphrase and i can ssh properly on the remote-host
virsh / virt-manager are working fine aswell

[pdcrom]

I see this:

provider "libvirt" {
uri = "qemu+ssh://root@kvm/system?socket=/var/run/libvirt/libvirt-sock"
}

terraform {
required_providers {
libvirt = {
source = "dmacvicar/libvirt"
version = "0.6.10"
}
}
}

---

❯ terraform init

Initializing the backend...

Initializing provider plugins...

• Finding dmacvicar/libvirt versions matching "0.6.10"...
• Installing dmacvicar/libvirt v0.6.10...
• Installed dmacvicar/libvirt v0.6.10 (self-signed, key ID 96B1FE1A8D4E1EAB)

Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/cli/plugins/signing.html

Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

---

❯ terraform plan

Error: failed to dial libvirt: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

on main.tf line 1, in provider "libvirt":
1: provider "libvirt" {

---

older version works correctly, can also ssh into the 'kvm' machine with private key, because that is specified in the ssh config

[sspreitzer]

Dear Duncan <@dmacvicar>

When I got rid of the libvirt client library, I had to write the ssh transport from scratch. I followed the documentation.

I don't think I implemented the sshauth part as described here, but with some guidance I think we can get it done: https://libvirt.org/uri.html

A lot of the connection functionality, especially ssh, worked nicely prior this design change. Why did you make the decision to drop the dependency to the libvirt library?

Is there anything that can make you consider the use of the libvirt library again?

Personally I prefer the use of the libvirt client and not to implement the libvirt protocol and the ssh protocol all by myself. The software would have to be kept always compatible, the efforts of maintenance would be high. Client libraries like the libvirt or ssh libraries are explicitly created from the other projects, so I don't have to think about the protocol and can stay compatible over multiple versions.

Hoping for the best and thank you for creating this wonderful piece of software, which I use everyday ❤️
Sascha <@sspreitzer>

[dmacvicar]

A lot of the connection functionality, especially ssh, worked nicely prior this design change. Why did you make the decision to drop the dependency to the libvirt library?

Because we needed a pure Go provider in order to be able to distribute using the Hashicorp Provider Registry which means people can install it just mentioning the provider.

Distributing a binary linking to a C library meant we have to provide different binaries per distribution.

[sspreitzer]

Because we needed a pure Go provider in order to be able to distribute using the Hashicorp Provider Registry which means people can install it just mentioning the provider.

I could not find a policy or rule made by Hashicorp, that would reject providers with external dependencies. But there is a statement about the compatibility towards Terraform Cloud here: https://www.terraform.io/docs/registry/providers/os-arch.html#terraform-cloud-compatibility

Citing:

To ensure your provider can run in Terraform Cloud, please include a Linux / AMD64 binary. This binary should also not have CGO enabled and should not depend on command line execution of any external tools or binaries. We cannot guarantee availibility of any package/library/binary within the Terraform Cloud images.

Hashicorp is stating that a provider should not have external dependencies, because it can not guarantee those dependencies available on Terraform cloud. Hashicorp is not stating all provider must not have external dependencies in general.

Distributing a binary linking to a C library meant we have to provide different binaries per distribution.

From my knowledge, in C there is the possibility instead to link to a library to dynamically load the library on runtime. dlopen() dlsym() and dlclose()
My golang skills are zero, but researching the dlopen interface I came across that it is part as a plugin in Go. And other search results used CGO to use the functions.

The libvirt library is available as libvirt.so on any Linux distribution that has it installed.

[root@voyager ~]# ldconfig -p|grep -i libvirt|grep 'libvirt.so '
	libvirt.so (libc6,x86-64) => /usr/lib64/libvirt.so

Putting those two information together, we can conclude, that a Hashicorp provider could dynamically load libvirt, if it is availble on the system, instead of linking to it. If it is not available it could fallback to a functionality limited builtin.

@dmacvicar, could it be a compromise of functionality and efforts to offer one provider that has full functionality for the non Terraform Cloud scenario and a limited builtin functionally for the Terraform Cloud scenario?

PS.: Or having two providers, one dynamically linked libvirt.so and named as is and another one with builtin functionality and named terraform-provider-libvirt-tc, where tc marks it for the Terraform Cloud scenario.

[dmacvicar]

I am not sure what you are suggesting. When we linked against libvirt, we used the official bindings https://github.com/libvirt/libvirt-go which require CGO and therefore can't be used in the Terraform registry. Maintaining a different set of bindings is considerable effort (and I am not even sure if it is technically possible).

The only reason why we can rely on a different approach is because we have alternative bindings, maintained outside the scope of this project.

Now, that does not mean we will ignore these feature gaps. I will try to fix them, inside our limited capacity of few developers and many users.

[dmacvicar]

This is fixed and will be part of 0.6.11. Thanks @oranenj for contributing the fix.

I tested locally using encrypted ed25519 keys and the GNOME ssh agent.

[mariogarcia]

After reading your last comment @dmacvicar I've realized that the "qemu+ssh" connection, at least in the latest release:

• It does work with encrypted ed25519 keys
• But IT DOESN'T with RSA keys

Versions:

Terraform v1.0.6
on linux_amd64
+ provider registry.terraform.io/dmacvicar/libvirt v0.6.11
+ provider registry.terraform.io/hashicorp/template v2.2.0

Now I'm extremely happy deploying remotely my virtual machines :)
Cheers

[frgomes]

Hello,

I'm hesitant to open a new issue without asking for help here in this thread.

I was using v0.6.10 and the connection string below was working just fine:

qemu+ssh://[email protected]/system?keyfile=$HOME/.ssh/id_ed25519_laptop.example.com

Now with 0.6.11 it is failing with message:

Error: failed to dial libvirt: ssh: handshake failed: ssh: unable to authenticate, attempted methods [publickey none], no supported methods remain

I've removed the keyfile part but it is still failing. If I ssh to the host it works just fine.

Any ideas?

Thanks

[dmacvicar]

Is your key encrypted with password?

If yes, you need a ssh agent.

[dmacvicar]

After reading your last comment @dmacvicar I've realized that the "qemu+ssh" connection, at least in the latest release:

* It does work with encrypted ed25519 keys

* But **IT DOESN'T** with RSA keys

Versions:

Terraform v1.0.6
on linux_amd64
+ provider registry.terraform.io/dmacvicar/libvirt v0.6.11
+ provider registry.terraform.io/hashicorp/template v2.2.0

Now I'm extremely happy deploying remotely my virtual machines :)
Cheers

@mariogarcia what is the error with RSA keys?

[frgomes]

@dmacvicar : No, my key does not have a password. It was working just fine with 0.6.10.

[mariogarcia]

@dmacvicar these are the steps I followed to reproduce it

Generate ssh RSA key

mylocalsystem:~$ ssh-keygen -t rsa -b 4096 -C "[email protected]"
Generating public/private rsa key pair.
Enter file in which to save the key (/home/myuser/.ssh/id_rsa): /home/myuser/.ssh/id_rsa_todelete
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/myuser/.ssh/id_rsa_todelete
Your public key has been saved in /home/myuser/.ssh/id_rsa_todelete.pub
The key fingerprint is:
SHA256:1ayb5UmYFecSw0grNidiBR1XH382QqhyU1bq9KkMeIM [email protected]
The key's randomart image is:
+---[RSA 4096]----+
|      .oooo+Boo  |
|       ....B+B o |
|      o = Bo=.o.+|
|     . +oXo=..o.o|
|       ES+=.oo   |
|        . +*..   |
|          ooo    |
|                 |
|                 |
+----[SHA256]-----+

Copied public key to remote KVM server

Changing ~/.ssh/config file in local

Host mykvmserver
  HostName 192.168.1.35
  User maestro
  IdentityFile ~/.ssh/id_rsa_to_delete

Executing terraform and getting the error

terraform apply

And the error

2021-09-17T12:35:49.501+0200 [INFO]  Terraform version: 1.0.6
2021-09-17T12:35:49.501+0200 [INFO]  Go runtime version: go1.16.4
2021-09-17T12:35:49.501+0200 [INFO]  CLI args: []string{"/home/myuser/.bin/terraform", "apply"}
...
2021-09-17T12:35:50.057+0200 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/dmacvicar/libvirt/0.6.11/linux_amd64/terraform-provider-libvirt_v0.6.11 args=[.terraform/providers/registry.terraform.io/dmacvicar/libvirt/0.6.11/linux_amd64/terraform-provider-libvirt_v0.6.11]
2021-09-17T12:35:50.057+0200 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/dmacvicar/libvirt/0.6.11/linux_amd64/terraform-provider-libvirt_v0.6.11 pid=10153
2021-09-17T12:35:50.057+0200 [DEBUG] provider: waiting for RPC address: path=.terraform/providers/registry.terraform.io/dmacvicar/libvirt/0.6.11/linux_amd64/terraform-provider-libvirt_v0.6.11
2021-09-17T12:35:50.062+0200 [INFO]  provider.terraform-provider-libvirt_v0.6.11: configuring server automatic mTLS: timestamp=2021-09-17T12:35:50.062+0200
2021-09-17T12:35:50.087+0200 [DEBUG] provider.terraform-provider-libvirt_v0.6.11: plugin address: address=/tmp/plugin182159544 network=unix timestamp=2021-09-17T12:35:50.087+0200
2021-09-17T12:35:50.087+0200 [DEBUG] provider: using plugin: version=5
2021-09-17T12:35:50.120+0200 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio"
2021-09-17T12:35:50.120+0200 [INFO]  ReferenceTransformer: reference not found: "var.cloudinit_net_conf"
2021-09-17T12:35:50.120+0200 [DEBUG] ReferenceTransformer: "module.hadoop-metis.data.template_file.network_config" references: []
2021-09-17T12:35:50.120+0200 [INFO]  ReferenceTransformer: reference not found: "var.cloudinit_user_conf"
2021-09-17T12:35:50.120+0200 [INFO]  ReferenceTransformer: reference not found: "var.ssh_public_key_path"
2021-09-17T12:35:50.120+0200 [DEBUG] ReferenceTransformer: "module.hadoop-jupyter.data.template_file.user_data" references: []
2021-09-17T12:35:50.120+0200 [DEBUG] No provider meta schema returned
2021-09-17T12:35:50.121+0200 [INFO]  ReferenceTransformer: reference not found: "var.cloudinit_net_conf"
2021-09-17T12:35:50.121+0200 [DEBUG] ReferenceTransformer: "module.hadoop-adrastea.data.template_file.network_config" references: []
2021-09-17T12:35:50.121+0200 [INFO]  ReferenceTransformer: reference not found: "var.cloudinit_net_conf"
2021-09-17T12:35:50.121+0200 [DEBUG] ReferenceTransformer: "module.hadoop-io.data.template_file.network_config" references: []
2021-09-17T12:35:50.122+0200 [INFO]  ReferenceTransformer: reference not found: "var.cloudinit_user_conf"
2021-09-17T12:35:50.122+0200 [INFO]  ReferenceTransformer: reference not found: "var.ssh_public_key_path"
2021-09-17T12:35:50.122+0200 [DEBUG] ReferenceTransformer: "module.hadoop-metis.data.template_file.user_data" references: []
2021-09-17T12:35:50.122+0200 [INFO]  ReferenceTransformer: reference not found: "var.cloudinit_user_conf"
2021-09-17T12:35:50.122+0200 [INFO]  ReferenceTransformer: reference not found: "var.ssh_public_key_path"
2021-09-17T12:35:50.122+0200 [DEBUG] ReferenceTransformer: "module.hadoop-io.data.template_file.user_data" references: []
2021-09-17T12:35:50.122+0200 [INFO]  ReferenceTransformer: reference not found: "var.cloudinit_net_conf"
2021-09-17T12:35:50.122+0200 [DEBUG] ReferenceTransformer: "module.hadoop-jupyter.data.template_file.network_config" references: []
2021-09-17T12:35:50.123+0200 [INFO]  ReferenceTransformer: reference not found: "var.cloudinit_user_conf"
2021-09-17T12:35:50.123+0200 [INFO]  ReferenceTransformer: reference not found: "var.ssh_public_key_path"
2021-09-17T12:35:50.123+0200 [DEBUG] ReferenceTransformer: "module.hadoop-adrastea.data.template_file.user_data" references: []
2021-09-17T12:35:50.124+0200 [DEBUG] provider.terraform-provider-libvirt_v0.6.11: 2021/09/17 12:35:50 [DEBUG] Configuring provider for 'qemu+ssh://maestro@mykvmserver/system': &{map[uri:0xc00015e3c0] <nil> <nil> 0xc000146260 map[] <nil> 0xc0001462e0 0xc00012e498 0xc000150410 false map[] {1 {0 0}} false false}
2021-09-17T12:35:50.126+0200 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/template/2.2.0/linux_amd64/terraform-provider-template_v2.2.0_x4 pid=10140
2021-09-17T12:35:50.126+0200 [DEBUG] provider: plugin exited
╷
│ Error: failed to dial libvirt: ssh: handshake failed: EOF
│
│   with provider["registry.terraform.io/dmacvicar/libvirt"],
│   on main.tf line 10, in provider "libvirt":
│   10: provider "libvirt" {

Now changing to an ed25519 key

• Created an ed25519 key
• Copied it to remote kvm server
• Modified the entry in ssh config file
• Executed terraform
• It worked

When it's working it seems that the call is the same, I didn't see anything that could make the difference. In the success case once it does the call it keeps going.

...
2021-09-17T12:49:26.080+0200 [DEBUG] No provider meta schema returned
2021-09-17T12:49:26.081+0200 [DEBUG] provider.terraform-provider-libvirt_v0.6.11: 2021/09/17 12:49:26 [DEBUG] Configuring provider for 'qemu+ssh://maestro@mykvmserver/system': &{map[uri:0xc0005163c0] <nil> <nil> 0xc0005580c0 map[] <nil> 0xc000558140 0xc000502210 0xc0005c4280 false map[] {1 {0 0}} false false}
...

I hope it helps

[dmacvicar]

@mariogarcia can you somehow get the list of ciphers supported by your server?

$ sudo /usr/sbin/sshd -T | grep "\(ciphers\|macs\|kexalgorithms\)"

(run on the server)

or

sudo nmap --script ssh2-enum-algos -sV -p 22 yourhost

remotely

I want to compare those with https://github.com/golang/crypto/blob/master/ssh/cipher.go#L99

[mariogarcia]

@dmacvicar here you have it:

[user@server ~]$ sudo /usr/sbin/sshd -T | grep "\(ciphers\|macs\|kexalgorithms\)"
gssapikexalgorithms gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-
ciphers [email protected],[email protected],aes256-ctr,[email protected],aes128-ctr
macs [email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha1,[email protected],hmac-sha2-512
kexalgorithms curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512

More information that could be relevant about the server:

• Fedora 34 server
• OpenSSH_8.6p1, OpenSSL 1.1.1l FIPS 24 Aug 2021

[ACampbell12]

Now changing to an ed25519 key

• Created an ed25519 key
• Copied it to remote kvm server
• Modified the entry in ssh config file
• Executed terraform
• It worked

Same here. Replicated on server with OpenSSH_8.8p1, OpenSSL 1.1.1l 24 Aug 2021

[jli-cparta]

Possibly related:
The following works with 0.6.10, but not with 0.6.11 (ed25519 private key):

provider "libvirt" {
  uri = "qemu+ssh://[email protected]/system?keyfile=./admin-example-com"
}

[dmacvicar]

@jli-cparta @frgomes I continue to try to reproduce this without success. Lets continue in #886 886

[strokovok]

Setting this on server-side ssh config fixed it for me:

AllowPortForwarding yes
AllowTcpForwarding yes
AllowStreamLocalForwarding yes

[kideg20]

Works with this config:
On the libvirt host (ubuntu22) in /etc/ssh/sshd_config:

AllowAgentForwarding yes
AllowTcpForwarding yes
AllowStreamLocalForwarding yes
DisableForwarding no

in provider config I use: (i use ssh agent so i just put user@host )
uri = "qemu+ssh://[email protected]/system"

I think if you specify ssh private key, it will also work.
Try:
uri = "qemu+ssh://[email protected]/system?keyfile=/home/ubuntu/.ssh/ubuntu.key"