Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

event handling in the storage users service is fragmenting #10327

Open
butonic opened this issue Oct 17, 2024 · 1 comment
Open

event handling in the storage users service is fragmenting #10327

butonic opened this issue Oct 17, 2024 · 1 comment
Labels

Comments

@butonic
Copy link
Member

butonic commented Oct 17, 2024

When trying to understand zero byte file upload issues in posixfs @aduffeck and I noticed that the places that emit events have started to fragment:

  • the storage provider is hardcoded to have async uploads = false, which prevents it from emitting any event when a zero byte file is uploaded
  • the data provider emits postprocessing related events for uploads
  • the eventsmiddleware interceptor emits events for metadata operations, but it has no way of emitting the path when it receives ids, something we need to properly show the end user the source and target path in move activities
  • @2403905 started moving some events into the storageprovider to emit events that include the path

The storage driver currently usese a config produced by OcisNoEvents(cfg) whereas the dataprovider config is produced by Ocis(cfg).
Ocis(cfg) has

		"asyncfileuploads":            cfg.Drivers.OCIS.AsyncUploads,

OcisNoEvents(cfg) omits that in order to force the finalization of zero byte file uploads. This is all very confusing and hard to reason about.

IMO we should move the eventsmiddleware into decomposedfs and evolve decomposedfs to have middlewares for resource lookup, permissioncheck and events. it should replace the reva Storage interface and become a framework for building storage drivers because it implements so many necessities we implicitly rely on.

@aduffeck
Copy link
Contributor

👍

Regarding the decomposedfs middleware there's now https://github.com/cs3org/reva/blob/edge/pkg/storage/utils/middleware/middleware.go which was introduced to enable the posixfs gid switching: https://github.com/cs3org/reva/blob/edge/pkg/storage/fs/posix/posix.go#L142-L175
I hope this could serve as a starting point for the other things you mentioned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Qualification
Development

No branches or pull requests

2 participants