Failing to authenticate to FreeNAS

[jhughes2112]

I have a new FreeNAS 11.3-U3.2 setup. Have the provisioner 2.6 installed. Haven't been able to get any NFS volumes provisioned.

I went into the provisioner and verified that my freenas.local box does resolve properly, so I dug into the tcpdump. Curl works fine, but the provisioner does not. Here's the curl I used from inside the provisioner bash shell:

curl --user root:mypassword http://freenas.local/api/v1.0/storage/dataset/metalgods/k8s/

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
17:30:52.752752 IP freenas-nfs-provisioner-cffbb8f44-dpvp7.57348 > freenas.local.80: Flags [P.], seq 487283753:487283908, ack 2211867418, win 219, options [nop,nop,TS val 1957179417 ecr 4132469310], length 155: HTTP: GET /api/v1.0/storage/dataset/metalgods/k8s/ HTTP/1.1
E....~@.@...
bp.
......P..\)..k......u.....
t.0..Pz>GET /api/v1.0/storage/dataset/metalgods/k8s/ HTTP/1.1
Host: freenas.local
Authorization: Basic cm9vdDpteXBhc3N3b3JkCg==
User-Agent: curl/7.52.1
Accept: */*


17:30:52.809679 IP freenas.local.80 > freenas-nfs-provisioner-cffbb8f44-dpvp7.57348: Flags [P.], seq 1:809, ack 155, win 1028, options [nop,nop,TS val 4132469366 ecr 1957179417], length 808: HTTP: HTTP/1.1 200 OK
E..\..@.?...
...
bp..P....k...\............
.Pzvt.0.HTTP/1.1 200 OK
Server: nginx
Date: Tue, 23 Jun 2020 17:30:52 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept, Accept-Language, Cookie
Cache-Control: no-cache
Content-Language: en
Strict-Transport-Security: max-age=0
X-Content-Type-Options: nosniff
X-XSS-Protection: 1

1cf
{"atime": "on", "avail": 1529620118136, "comments": "Used by the cluster as remote PV storage", "compression": "lz4", "dedup": "off", "exec": "on", "inherit_props": ["compression", "aclinherit", "org.freebsd.ioc:active"], "mountpoint": "/mnt/metalgods/k8s", "name": "metalgods/k8s", "pool": "metalgods", "quota": 0, "readonly": "off", "recordsize": 131072, "refer": 253704, "refquota": 0, "refreservation": 0, "reservation": 0, "sync": "standard", "used": 253704}
0

Unfortunately, this is what I see when snooping the provisioner:

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
17:19:33.667899 IP freenas-nfs-provisioner-cffbb8f44-dpvp7.56334 > freenas.local.80: Flags [P.], seq 2892095849:2892096082, ack 4282234469, win 219, options [nop,nop,TS val 1956500347 ecr 1027750863], length 233: HTTP: GET /api/v1.0/storage/dataset/metalgods/k8s/ HTTP/1.1
E....C@.@...
bp.
......P.a.i.=.e...........
t..{=B;.GET /api/v1.0/storage/dataset/metalgods/k8s/ HTTP/1.1
Host: freenas.local:80
User-Agent: Go-http-client/1.1
Accept: application/json
Authorization: Basic cm9vdDpteXBhc3N3b3JkCg==
Content-Type: application/json
Accept-Encoding: gzip


17:19:33.704049 IP freenas.local.80 > freenas-nfs-provisioner-cffbb8f44-dpvp7.56334: Flags [P.], seq 1:281, ack 233, win 1028, options [nop,nop,TS val 1027750899 ecr 1956500347], length 280: HTTP: HTTP/1.1 401 Unauthorized
E..L..@.?...
...
bp..P...=.e.a.R...........
=B;.t..{HTTP/1.1 401 Unauthorized
Server: nginx
Date: Tue, 23 Jun 2020 17:19:33 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
WWW-Authenticate: Basic Realm="django-tastypie"
Vary: Accept-Language, Cookie
Content-Language: en

0


17:19:33.819787 IP freenas-nfs-provisioner-cffbb8f44-dpvp7.56340 > freenas.local.80: Flags [P.], seq 442110070:442110303, ack 4167529060, win 219, options [nop,nop,TS val 1956500499 ecr 65777863], length 233: HTTP: GET /api/v1.0/storage/dataset/metalgods/k8s/ HTTP/1.1
E....=@.@...
bp.
......P.Z.v.grd...........
t.......GET /api/v1.0/storage/dataset/metalgods/k8s/ HTTP/1.1
Host: freenas.local:80
User-Agent: Go-http-client/1.1
Accept: application/json
Authorization: Basic cm9vdDpteXBhc3N3b3JkCg==
Content-Type: application/json
Accept-Encoding: gzip


17:19:33.848877 IP freenas.local.80 > freenas-nfs-provisioner-cffbb8f44-dpvp7.56340: Flags [P.], seq 1:281, ack 233, win 1028, options [nop,nop,TS val 65777892 ecr 1956500499], length 280: HTTP: HTTP/1.1 401 Unauthorized
E..L..@.?...
...
bp..P...grd.Z._...........
....t...HTTP/1.1 401 Unauthorized
Server: nginx
Date: Tue, 23 Jun 2020 17:19:33 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
WWW-Authenticate: Basic Realm="django-tastypie"
Vary: Accept-Language, Cookie
Content-Language: en

0

It's unclear to me how I have set it up wrong, I'm guessing the more restrictive Accept and Accept-Encoding are causing problems, but I'm far from an authentication expert. Any ideas how to fix this?

JH

[jhughes2112]

Stranger, still, that I added the exact same headers to the curl line as was present, in exactly the same order, and it worked fine. I have no clue why the server would reject the one from the provisioner but work fine for curl.