You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Profiling and debugging remote execution is currently hard; we have some logging, but nothing structured.
Let's modify the process_execution::remote::CommandRunner so that it tracks timings of key events, and outputs a zipkin trace for the execution.
Key events:
Making the execute RPC (and getting a final)
Starting and finishing uploading any files/digests that may be needed, both per-digest and for batches
I'm imagining a new struct along the lines of:
struct TimingEvent {
execution_id: String, // just a uuid
description: String, // something like "Uploading digest abc123 for execution of action with digest def456" or "Waiting for action with digest def456 to complete"
start_time: std::time::SystemTime,
finish_time: std::time::SystemTime,
}
the remote::CommandRunner having a Vec of these things, and the run method passing around an incomplete one, then adding it to the Vec when it's complete.
We can punt on formatting these as zipkin traces in the short term - just dumping these as JSON is better than nothing, but let's try to stay as close as possible to zipkin so that it's easy to zipkin-ise these in the future.
The text was updated successfully, but these errors were encountered:
(We will also, at some point, probably want to work out how to merge this into WorkUnit hierarchy, but passing that information over the Rust-Python boundary probably isn't worthwhile while this is in flux)
Profiling and debugging remote execution is currently hard; we have some logging, but nothing structured.
Let's modify the
process_execution::remote::CommandRunner
so that it tracks timings of key events, and outputs a zipkin trace for the execution.Key events:
execute
RPC (and getting a final)I'm imagining a new struct along the lines of:
the
remote::CommandRunner
having aVec
of these things, and therun
method passing around an incomplete one, then adding it to theVec
when it's complete.We can punt on formatting these as zipkin traces in the short term - just dumping these as JSON is better than nothing, but let's try to stay as close as possible to zipkin so that it's easy to zipkin-ise these in the future.
The text was updated successfully, but these errors were encountered: