Contribute to Drill?

[cgivre]

Hello there,
Nice work on your Drill storage plugin! I was wondering if you might consider submitting this as a pull request to be included in the next version of Drill?
If you are open to it, I can assist with the review process.
Thanks!

[dbw9580]

Hi, thank you for contacting! We would love to see Minerva become part of Drill!
We had been working on other projects and haven't updated it for quite a while. There are still performance issues and bugs here and there. So I'm not sure if it meets the standards of Drill to be merged, and if there is anything else that should be done on our side?

[cgivre]

@dbw9580
I can help you through the review process. It's ok if it isn't perfect. We can get it into Drill and then refine it. One thing we will need however are unit tests. There is a current PR for a storage plugin for Apache Druid 1 where the author wrote a very thorough suite of unit tests. You can use that as a reference for how to do it.

[dbw9580]

Thanks! We'll look into it asap! ⁣祝好， Best Regards, 博文 Bowen 2020年3月8日 14:11, 14:11，在 "Charles S. Givre" <[email protected]> 已写: @dbw9580 I can help you through the review process. It's ok if it isn't perfect. We can get it into Drill and then refine it. One thing we will need however are unit tests. There is a current PR for a storage plugin for Apache Druid 1 where the author wrote a very thorough suite of unit tests. You can use that as a reference for how to do it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[bmcnabb25]

@cgivre @dbw9580 out of curiosity, was this plugin ever included in Drill?

[cgivre]

@bmcnabb25 There was a pull request started (apache/drill#2084) but it hasn't been merged yet. My sense is that it was close and if someone was willing to pick up the work, we could get it merged.

[nashshapley]

@cgivre Thanks for the invitation. Would you mind listing the remaining questions that need to be solved? Bowen and our new team members will work together to get it merged. If possible, would you please send the todo list to my email: [email protected].

[cgivre]

Hi @nashshapley
Please take a look here: apache/drill#2084.

The reason this wasn't merged was that in its current implementation, the IPFS/Drill plugin was creating new Drillbits and tearing them down as needed. This is likely to cause serious issues in a distributed system such as Drill because there is no way to control what query goes to which Drillbit.

With all that said, I think there is a MUCH simpler way of implementing this plugin. In Drill 1.20 we added a file system extension for Dropbox. It reuses all the existing Drill file readers. Basically, you'd have to implement methods that get an InputStream of the file. You can also write by doing the same thing. It should be fairly straightforward.