Friday, 2024/08/02
3:00 pm ET
Terrell Russell, Kory Draughn, Alan King, Justin James, Tony Edgin (CyVerse), David Cerf (GRAU DATA)
DISCUSSION
- iRODS S3 API
- 'Efficient store-and-forward' is 'working', but not yet merged
- Storing parts information in shared table in memory
- If we know all the preceding parts, we have enough information to stream directly to iRODS
- If not, we write part files locally at the API (like before)
- Approximately 30% faster than 'store-and-forward'
- 12s instead of 18s for 600MB
- With no network, all local, no distance
- irods/irods_client_s3_api#115
- Moved 15 issues to 0.4.0 milestone
- Tony wants to attach a bucket to a logical_path
- irods/irods_resource_plugin_s3#2196
- We're still thinking a new ARCHIVE_NAMING_POLICY
- with a new context string defining the 'left hand side' to 'chop' and 'enforce'
- If a new submission comes in for this particular resource, but is 'outside the chroot'
- we'd expect to see a 'bad path' or 'chroot requirement not met' or 'logical and physical are out of alignment'
- assumptions not met, therefore, no data is saved
- Someone should test this
- Next Meeting
- September 2024