in_tail does not filter directories when the ** pattern is used

[angry-cellophane]

• td-agent 2.3.3
• OS: redhat 7.3
• Configuration:

<source>
   @type tail
   path /tmp/fluentd/**
   tag tail.tag
   format none
   pos_file /var/log/td-agent/tmp/tail-tag.pos
   read_from_head true
</source>

<match tail.tag>
  @type stdout
</match>   

ls -R /tmp/fluentd/
/tmp/fluentd/:
dir1

/tmp/fluentd/dir1:
dir2    dir3

/tmp/fluentd/dir1/dir2:
file1.log    file2.log

/tmp/fluentd/dir1/dir3:
dir4    file3.log

/tmp/fluentd/dir1/dir3/dir4:
file4.log    file5.log    file6

Our case: there is a folder that contains log files. The log files can have different extensions, even nothing (like file6 in the example above). We want fluentd to tail all the files in the given directory and its subdirectories. It can be done with the in_tail plugin and the ** pattern in the path. However, in_tail can't distinguish files and folders, therefore fluentd doens't tail the files but shows the following error:

2017-01-25 22:54:31 +0000 [error]: Is a directory @ io_getpartial - /some/path

I looked through the code and found no solutions for this issue except adding a filter for directories in in_tail.rb.
I would appreciate if you could recommend a workaround for this issue without code change.
If you think a code change is needed to solve this issue I could create a patch.

[repeatedly]

path /tmp/fluentd/**/*.log doesn't work?

[angry-cellophane]

path /tmp/fluentd/**/*.log works fine, but we're having an issue with files that don't have extensions like /tmp/fluentd/dir1/dir4/file6

[repeatedly]

For such cases, separating in_tail setting is better.

[repeatedly]

To ignore directory, adding file type check is one way.

fluentd/lib/fluent/plugin/in_tail.rb

Line 187 in 4dab161

if File.readable?(p)

[angry-cellophane]

It's what I did: I added a method that removes all directories from the result of expand_paths. However, I was wondering if there was another way to solve this issue with a configuration change only.

[repeatedly]

Fixed at master