Add Fluentd v1.13.0 images #270

kenhys · 2021-06-03T09:12:42Z

No description provided.

kenhys · 2021-06-03T09:13:32Z

kenhys · 2021-06-03T11:33:49Z

It stalls with *** Removed specs cache ***...

kenhys · 2021-06-04T02:30:19Z

It stalls with *** Removed specs cache ***...

I doubt that the image may be built but docker images doesn't show it. Hmm.
(Windows 10 21H1, Docker Engine v20.10.6, and still dockerd.exe consume 30% continuously)

https://github.com/fluent/fluentd-docker-image/pull/270/files#diff-28b7554bb0449c765034d46df17564c613771856b561847673eeb44d88a25da7R31-R40

There may be something wrong after gem sources --clear-all.

# Remove gem cache and chocolatey
RUN powershell -Command "Remove-Item -Force C:\ruby26\lib\ruby\gems\2.6.0\cache\*.gem; Remove-Item -Recurse -Force 'C:\ProgramData\chocolatey'"

COPY fluent.conf /fluent/conf/fluent.conf


ENV FLUENTD_CONF="fluent.conf"

EXPOSE 24224 5140

ENTRYPOINT ["cmd", "/k", "fluentd", "-c", "C:\\fluent\\conf\\fluent.conf"]

kenhys · 2021-06-04T04:12:32Z

hmm, hit the microsoft/hcsshim#696 or docker/for-win#3229?

How about @cosmo0920 ?

cosmo0920 · 2021-06-04T08:52:05Z

hmm, hit the microsoft/hcsshim#696 or docker/for-win#3229?

How about @cosmo0920 ?

It's curious.
As usual, this issue is caused by old MSYS2 and updating huge amount of packages within ridk exec pacman -Syu --noconfirm step.
But, MSYS2 20210419.0.0 is the latest stable released version.... 🤔

cosmo0920 · 2021-06-04T08:53:28Z

And we should update using Dockerfile paths on GitHub Actions:
https://github.com/fluent/fluentd-docker-image/blob/master/.github/workflows/linux.yml#L12-L13

cosmo0920 · 2021-06-04T09:42:16Z

There may be something wrong after gem sources --clear-all.

I tested with removed gem sources --clear-all step but no luck.

cosmo0920 · 2021-06-04T10:00:55Z

I split installing MSYS2 and gems into 2 steps as follows:

RUN refreshenv \
&& ridk exec pacman -Syu --noconfirm \
&& ridk install 2 3
RUN echo gem: --no-document >> C:\ProgramData\gemrc \
&& gem install cool.io -v 1.5.4 --platform ruby \
&& gem install oj -v 3.3.10 \
&& gem install json -v 2.2.0 \
&& gem install fluentd -v 1.13.0 \
&& gem install win32-service -v 2.1.6 \
&& gem install win32-ipc -v 0.7.0 \
&& gem install win32-event -v 0.6.3 \
&& gem install windows-pr -v 1.2.6 \
&& gem sources --clear-all

But it is no luck, too. 😖

kenhys · 2021-06-07T02:00:17Z

Updated .github/workflows/linux.yml and it is ok as expected.

kenhys · 2021-06-07T02:09:18Z

I've checked whether rest of file related operations such as gem sources --clear-all , powershell -Command "Remove-Item ... and COPY fluent.conf /fluent/conf/fluent.conf affects (by comment out them), but it does not change the situation.
(As Removed specs cache is shown by gem sources --clear-all, so something wrong in more early stage....)

cosmo0920 · 2021-06-07T04:22:58Z

but it does not change the situation.
(As Removed specs cache is shown by gem sources --clear-all, so something wrong in more early stage....)

Yes. I think so, too.
I'd got similar situations for several times, it was suddenly resolved by upgrading MSYS2.
But Currently, our using MSYS2 installer is still the previous latest stable version.
How about using the current very fresh stable version of MSYS2 installer?
https://github.com/msys2/msys2-installer/releases/tag/2021-06-04
https://community.chocolatey.org/packages/msys2/20210604.0.0

If with the current very fresh stable version of MSYS2 makes to be buildable on Windows Server Core images, we should use this fresh stable version of MSYS2.

cosmo0920 · 2021-06-07T04:24:15Z

(And we also should push Fluentd v1.12.4 images for Windows Server Core. I'd forgotten to build them. 😖)

cosmo0920 · 2021-06-07T04:48:49Z

I'm confirming whether upgrading MSYS2 is effective to be able to build Windows images....

cosmo0920 · 2021-06-07T05:15:26Z

I'd registered docker building stall issue patch for Windows images: #271

cosmo0920 · 2021-06-07T06:02:19Z

@kenhys Could you rebase off master? I think that #271 is effective to resolve stucking of Windows docker images building.

ashie · 2021-06-07T06:36:25Z

We should add resolv 0.2.1, or downgrade Ruby to 2.6.6 to avoid fluent/fluentd#3382

ashie · 2021-06-07T06:39:24Z

BTW ruby 2.6 ends the normal maintenance phase: https://www.ruby-lang.org/en/news/2021/04/05/ruby-2-6-7-released/
Should we use ruby 2.7.2? (2.7.3 has also broken resolv).

cosmo0920 · 2021-06-07T06:41:24Z

BTW ruby 2.6 ends the normal maintenance phase: https://www.ruby-lang.org/en/news/2021/04/05/ruby-2-6-7-released/
Should we use ruby 2.7.2? (2.7.3 has also broken resolv).

Our fluentd-docker-image is used by fluentd-kubernetes-daemonset images.
If we upgrade Ruby version to v2.7, we should confirm that Ruby 2.7 is workable on kubernetes daemonset images...

cosmo0920 · 2021-06-07T06:42:14Z

We should add resolv 0.2.1, or downgrade Ruby to 2.6.6 to avoid fluent/fluentd#3382

I guess that adding resolv v0.2.1 is better than rollback Ruby version to 2.6.6....

ashie · 2021-06-07T06:44:00Z

I see. Let's go with ruby 2.6.7 & resolv 0.2.1 for this release.

ashie · 2021-06-07T06:45:33Z

Our fluentd-docker-image is used by fluentd-kubernetes-daemonset images.

BTW I think v1.12 image should be also updated to fix resolv's issue.
What do you think about it? @cosmo0920

cosmo0920 · 2021-06-07T06:47:48Z

Our fluentd-docker-image is used by fluentd-kubernetes-daemonset images.

BTW I think v1.12 image should be also updated to fix resolv's issue.
What do you think about it? @cosmo0920

Yes, we should do.
But Windows images still use Ruby 2.6.5 because chocolatey does not package Ruby 2.6.6 and 2.6.7:
https://community.chocolatey.org/packages/ruby#versionhistory
We should do that work for Linux images(alpine/debian).

ashie · 2021-06-07T06:49:05Z

Yes, I've checked it.

cosmo0920 · 2021-06-07T06:52:37Z

@kenhys Could you rebase off master? I think that #271 is effective to resolve stucking of Windows docker images building.

#271 is effective for stucking to build Windows images.
I'd confirmed that Windows images can be built with MSYS2 20210604.0.0.

cosmo0920 · 2021-06-07T07:20:21Z

Our fluentd-docker-image is used by fluentd-kubernetes-daemonset images.

BTW I think v1.12 image should be also updated to fix resolv's issue.
What do you think about it? @cosmo0920

Yes, we should do.
But Windows images still use Ruby 2.6.5 because chocolatey does not package Ruby 2.6.6 and 2.6.7:
https://community.chocolatey.org/packages/ruby#versionhistory
We should do that work for Linux images(alpine/debian).

I'd sent in #272.

cosmo0920 · 2021-06-07T08:27:03Z

Could you resolve conflicts?
#272 makes merging conflicts against Makefile....

kenhys · 2021-06-08T00:57:08Z

Ok, I'll do it.

kenhys · 2021-06-08T01:40:12Z

Now, v1.13.0 is ready to build 🎉

ashie · 2021-06-08T07:26:29Z

We need to check whether resolv gem is correctly loaded or not.
It seems that v1.12 image doesn't load it: #272 (comment)

kenhys · 2021-06-09T02:46:16Z

We need to check whether resolv gem is correctly loaded or not.
It seems that v1.12 image doesn't load it: #272 (comment)

I'll create another PR to fix it.

kenhys · 2021-06-09T02:57:27Z

I've created PR for loading resolv issue for v1.12
#275

kenhys · 2021-06-10T01:18:48Z

I'll resolve the conflict.

Closes: fluent#269 Signed-off-by: Kentaro Hayashi <[email protected]>

cosmo0920 · 2021-06-14T07:20:52Z

Any blocker to merge and publish Fluentd v1.13.0 images?

ashie · 2021-06-14T07:25:03Z

Sorry, just forgot to merge.

cosmo0920 · 2021-06-14T07:49:04Z

No problem. Let's publish the new images.
But currently, DockerHub reached 25 rules on its limitation of defined build rules.
Sould we remove v0.12 building definitions from DockerHub?

ashie · 2021-06-14T07:52:57Z

Sould we remove v0.12 building definitions from DockerHub?

👍

cosmo0920 · 2021-06-14T07:57:40Z

I removed the definitions for Fluentd v0.12 and kicked Fluentd v1.13 image buildings.

cosmo0920 · 2021-06-14T10:05:51Z

@kenhys Could you publish your built Windows Server images?

kenhys · 2021-06-14T12:47:33Z

ok, I'll try.

kenhys · 2021-06-14T13:30:14Z

done!

kenhys · 2021-06-14T13:30:31Z

make DOCKERFILE=v1.13/windows-20H2 VERSION=v1.13.0-windows-20H2-1.0
make DOCKERFILE=v1.13/windows-20H2 VERSION=v1.13-windows-20H2-1

make DOCKERFILE=v1.13/windows-2004 VERSION=v1.13.0-windows-2004-1.0
make DOCKERFILE=v1.13/windows-2004 VERSION=v1.13-windows-2004-1

make DOCKERFILE=v1.13/windows-ltsc2019 VERSION=v1.13.0-windows-ltsc2019-1.0
make DOCKERFILE=v1.13/windows-ltsc2019 VERSION=v1.13-windows-ltsc2019-1



docker push fluent/fluentd:v1.13.0-windows-20H2-1.0
docker push fluent/fluentd:v1.13-windows-20H2-1
docker push fluent/fluentd:v1.13.0-windows-2004-1.0
docker push fluent/fluentd:v1.13-windows-2004-1
docker push fluent/fluentd:v1.13.0-windows-ltsc2019-1.0
docker push fluent/fluentd:v1.13-windows-ltsc2019-1

kenhys force-pushed the v1-13-0 branch from 965d4ca to 47c4e41 Compare June 4, 2021 04:20

kenhys force-pushed the v1-13-0 branch from 47c4e41 to e42f45b Compare June 7, 2021 01:30

kenhys force-pushed the v1-13-0 branch from e42f45b to 43c67d1 Compare June 7, 2021 06:06

ashie mentioned this pull request Jun 7, 2021

alpine: debian: Use the latest resolv gem #272

Merged

kenhys force-pushed the v1-13-0 branch from 43c67d1 to 9468e47 Compare June 8, 2021 01:06

kenhys marked this pull request as ready for review June 8, 2021 01:24

cosmo0920 approved these changes Jun 8, 2021

View reviewed changes

Add Fluentd v1.13.0 images

1a2fff0

Closes: fluent#269 Signed-off-by: Kentaro Hayashi <[email protected]>

kenhys force-pushed the v1-13-0 branch from 9468e47 to 1a2fff0 Compare June 10, 2021 01:41

ashie merged commit 2b70eeb into fluent:master Jun 14, 2021

kenhys deleted the v1-13-0 branch June 21, 2021 09:10

kenhys mentioned this pull request Jun 28, 2021

Add Fluentd v.1.13.1 images #281

Merged

Add Fluentd v1.13.0 images #270

Add Fluentd v1.13.0 images #270

Conversation

kenhys commented Jun 3, 2021

kenhys commented Jun 3, 2021 • edited Loading

kenhys commented Jun 3, 2021

kenhys commented Jun 4, 2021

kenhys commented Jun 4, 2021

cosmo0920 commented Jun 4, 2021

cosmo0920 commented Jun 4, 2021

cosmo0920 commented Jun 4, 2021

cosmo0920 commented Jun 4, 2021 • edited Loading

kenhys commented Jun 7, 2021

kenhys commented Jun 7, 2021

cosmo0920 commented Jun 7, 2021 • edited Loading

cosmo0920 commented Jun 7, 2021

cosmo0920 commented Jun 7, 2021

cosmo0920 commented Jun 7, 2021

cosmo0920 commented Jun 7, 2021

ashie commented Jun 7, 2021

ashie commented Jun 7, 2021

cosmo0920 commented Jun 7, 2021

cosmo0920 commented Jun 7, 2021

ashie commented Jun 7, 2021 • edited Loading

ashie commented Jun 7, 2021

cosmo0920 commented Jun 7, 2021

ashie commented Jun 7, 2021

cosmo0920 commented Jun 7, 2021

cosmo0920 commented Jun 7, 2021

cosmo0920 commented Jun 7, 2021

kenhys commented Jun 8, 2021

kenhys commented Jun 8, 2021

ashie commented Jun 8, 2021

kenhys commented Jun 9, 2021

kenhys commented Jun 9, 2021

kenhys commented Jun 10, 2021

cosmo0920 commented Jun 14, 2021

ashie commented Jun 14, 2021

cosmo0920 commented Jun 14, 2021

ashie commented Jun 14, 2021 • edited Loading

cosmo0920 commented Jun 14, 2021

cosmo0920 commented Jun 14, 2021

kenhys commented Jun 14, 2021

kenhys commented Jun 14, 2021

kenhys commented Jun 14, 2021

kenhys commented Jun 3, 2021 •

edited

Loading

cosmo0920 commented Jun 4, 2021 •

edited

Loading

cosmo0920 commented Jun 7, 2021 •

edited

Loading

ashie commented Jun 7, 2021 •

edited

Loading

ashie commented Jun 14, 2021 •

edited

Loading