Upstairs write stats are no longer sent to oximeter.

[leftwo]

I noticed on dogfood that read/flush metrics are showing updates, but write metrics are empty.

Over in ack_job() we have this:

            IOop::Write { .. } => {
                cdt::gw__write__done!(|| (ds_id.0));
                self.stats.add_write(io_size as i64);
            }

Checking on a running propolis instance that is doing writes, I see the probe "gw__write__done" is no longer firing:

BRM44220011 # dtrace -n 'crucible_upstairs*:::gw-write-done { trace("fds"); }'
dtrace: description 'crucible_upstairs*:::gw-write-done ' matched 8 probes
^C

The read/flush probes do fire.

[leftwo]

The ack for a write now happens in compute_deferred_write(), so it never travels through the ack_job() in upstairs/src/downstairs.rs. So, the accounting never happens.

Also, it seems like the gw__*__start dtrace probes fire after a bit of work has happened already in the upstairs, so those probes may have lost some of their accuracy. In the case of a write, we have already acked that back to the guest before we even get to the place where we set gw__write__start