Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented sending logs to systemd-journald #1975

Merged
merged 3 commits into from
May 31, 2023

Conversation

yihuaf
Copy link
Collaborator

@yihuaf yihuaf commented May 27, 2023

@utam0k mentioned a usecase where when test youki inside k8s, it is hard to track logs of each youki implementation. A workaround was using bpf tracing the syscalls. However, it would be much better to see the actual youki logs. This is where journald comes in. It is one of the common ways for linux services and application to collect logs into a central place. In this PR, we implemented a different tracing layer that sends youki logs to journald. All logs are tagged as youki and we can obtain the logs with journalctl -t youki.

Note. I am not sure how best to enable this feature. I decided to implement this as a flag --systemd-log on the command line. However, this is not part of the OCI spec, so I added as a youki extension in the struct. However, this may not be the best way.

Another way to implement this is through features.

Tag along in this PR is a small refactor of the existing logging code. This refactors the logging into using different layers where we can get ready for the opentelelmetry work.

@yihuaf yihuaf requested review from utam0k and a team May 27, 2023 07:13
@codecov-commenter
Copy link

Codecov Report

Merging #1975 (05305af) into main (dc5d021) will increase coverage by 0.26%.
The diff coverage is 63.09%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1975      +/-   ##
==========================================
+ Coverage   65.02%   65.28%   +0.26%     
==========================================
  Files         129      129              
  Lines       14731    14784      +53     
==========================================
+ Hits         9579     9652      +73     
+ Misses       5152     5132      -20     

Copy link
Member

@utam0k utam0k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you write how to test?

@yihuaf
Copy link
Collaborator Author

yihuaf commented May 30, 2023

Could you write how to test?

In one terminal, run journalctl -f -t youki. The -t means tag. We tag all the logs using youki tag, which we hardcoded in this PR. The -f means follow, so you wait on the logs as they arrive. On another terminal, run normal youki commands with --systemd-log flag, such as ./youki --debug --systemd-log create -b tutorial test. You should see logs shows up in systemd log through journalctl.

Currently, we are exposing this feature using a flag --systemd-log. We may consider exposing this as a feature flag at compile time, so for k8s testing, no runtime flag needs to be added.

Give it a try first to see how you like how this prototype works currently. Note, journalctl itself has a lot of features, so we don't do much other than shipping the log/trace to it. For example, journalctl can present the log as json format, so our implementation don't need to do it.

Comment on lines +35 to +36
#[clap(flatten)]
youki_extend: YoukiExtendOpts,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love this idea ❤️

@utam0k utam0k merged commit cb75d26 into youki-dev:main May 31, 2023
@yihuaf yihuaf deleted the yihuaf/journald branch May 31, 2023 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants