Plugin FilePermission

[KennFalcon]

I learned that Elasticsearch does not allow plugins to have write and delete file permissions. As a result, the analysis plugin I developed cannot use the hot update dictionary. The old dictionary can not delete and the new can not write. Can the plugin have read and write permissions in the plugin directory?

[elasticmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[stu-elastic]

Hi @KennFalcon, thanks for the issue.

Why does the custom plugin need to hot update the dictionary? If you could elaborate your use-case, that'd be helpful. Is there another strategy without read/delete permission that would achieve the same ends?

To set expectations, we won't have time to address this issue for a few months. However, in the meantime, providing the use-case will help us with context.

[KennFalcon]

My plugin is Chinese word segmentation plugin. Before 7.11.0, I created the new dictionary which had new Chinese word in my plugin's dictionary, then read it regularly. But if the dictionary had a great many words, the process would take a lot of time. So my plugin read the dictionary which had part of speech and build a binary file which adopt double array trie on the first load. When the elasticsearch restarted, the plugin would read the binary file and didn't need to read the original dictionary file again. This loading procedure was fast, and the analysis procedure used the double array trie directly. If the new dictionary added to the plugin directory, the binary file would deleted and rebuild.

[stu-elastic]

It would take us a lot of work to add write permissions for plugins safely. Can you generate the trie from the dictionary beforehand and load them both?

[KennFalcon]

The main reason is that the dictionary may change frequently, so I used the hot update method. Can only give the plugin write permission in its own plugin directory?

[rjernst]

@KennFalcon Granting write permissions for the plugin installation directory is not something we plan to ever allow. The plugin directory is code, just like the lib and modules directories.

I have a couple suggestions for how you might go about updating your data. The first is to store the dictionary in your plugin's config directory. While this does not have write permissions at runtime, you could update the file outside of Elasticsearch, and watch for changes to the file within your plugin. This still requires pushing your dictionary to all nodes though. That brings up the second idea, which is to store your dictionary as a document (say a base64 encoded blob) in an ancillary index. You can periodically do a read of the special document, and thus all your nodes would get the update. The downside is you would not want the polling frequency to be too fast, so it would probably result in a slower update than pushing to every node's filesystem yourself.

Given that plugins lacking write permission to their own installation directory is by design, and we do not plan to change that, I hope you don't mind that I close this issue.