Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add job creator CLI run trace #418

Merged
merged 2 commits into from
Nov 1, 2024
Merged

feat: Add job creator CLI run trace #418

merged 2 commits into from
Nov 1, 2024

Conversation

bgins
Copy link
Contributor

@bgins bgins commented Oct 28, 2024

Summary

This pull request makes the following changes:

  • Add job creator CLI run trace
  • Remove a line of commented code

This pull requests implements a trace over jobs initiated with the CLI run command.

Task/Issue reference

Closes: #407

Test plan

Start the observability stack (https://github.com/Lilypad-Tech/observability) with:

./stack observe-up dev

Run a CLI job. Open the observability dashboard at localhost:3000 and run the following TraceQL query:

{resource.service.name="job_creator" && name="run_job"}

Details

We would also like to implement traces over jobs received onchain, but we will take on this work in a future PR.

Related issues or PRs (optional)

Depends on #416. This PR has a temporary pending: Add download and accept result error handling commit that will be dropped when the dependency is fulfilled.

@cla-bot cla-bot bot added the cla-signed label Oct 28, 2024
@bgins bgins force-pushed the bgins/feat-add-run-trace branch from ea152fb to 5e757b8 Compare October 28, 2024 19:01
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes in this file are from the temporary pending: Add download and accept result error handling commit. They will be dropped and replaced by the changes from #416.

attribute.String("job_offer.job_creator", offer.JobCreator),
attribute.String("job_offer.module.repo", offer.Module.Repo),
attribute.String("job_offer.module.hash", offer.Module.Hash),
attribute.String("job_offer.mode", string(offer.Mode)),
Copy link
Contributor Author

@bgins bgins Oct 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will likely want to add more fields around pricing when we continue our work here.

Comment on lines +77 to +78
span.SetAttributes(attribute.String("job_offer.id", jobOfferContainer.JobOffer.ID),
attribute.String("deal.id", jobOfferContainer.DealID))
Copy link
Contributor Author

@bgins bgins Oct 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting span attributes this far into a span is discouraged, but we need to submit the job offer before we can establish a deal ID. We want the deal ID to query on and associate with traces from other services.

@bgins bgins marked this pull request as ready for review October 29, 2024 16:22
@bgins bgins requested a review from a team as a code owner October 29, 2024 16:22
@@ -46,20 +49,42 @@ func RunJob(
// wait a short period because we've just started the job creator service
time.Sleep(100 * time.Millisecond)

// Start run job trace
c, span := jobCreatorService.controller.tracer.Start(ctx.Ctx, "run_job",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this span shares the same name as the resource provider run_job span. A TraceQL query like {name="run_job"} will include spans from both the job creator and resource provider.

This behavior is considered acceptable because one can narrow their search by adding the service name. For example {resource.service.name="job_creator" && name="run_job"}.

Copy link
Collaborator

@walkah walkah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for breaking this up into smaller PRs!

Tested locally

@bgins bgins force-pushed the bgins/feat-add-run-trace branch from df1be79 to 70db13d Compare October 31, 2024 21:29
@bgins bgins merged commit f9ce8a7 into main Nov 1, 2024
4 checks passed
@bgins bgins deleted the bgins/feat-add-run-trace branch November 1, 2024 02:17
walkah pushed a commit that referenced this pull request Dec 10, 2024
* feat: Add job creator run trace

* chore: Remove commented code
@bgins bgins self-assigned this Dec 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add job creator run job trace
2 participants