Azure Storage blob versioning

[MariuszBartnik]

I have a Azure Storage account with blob versioning functionality enabled. I want to upsert file, by providing the same path for blob as the one, that already exists on the storage and create a new version of this file in this way. I was doing this without any problems on my previous setup, which was an Express app with multer streaming files to Azure, but now I want to migrate to using tusd instead.

I have noticed an issue with this approach though. When I try to download the updated file I am still getting the original one, instead of a new one, even if the upload was a success. I've notice that the issue is connected to this piece of code in the getOffset function in azureservice.go file.

getBlock, err := blockBlob.Blob.GetBlockList(ctx, azblob.BlockListAll, azblob.LeaseAccessConditions{})

I might be missing something, so I wanted to ask if we really need to get the block list for azblob.BlockListAll instead of azblob.BlockListUncommitted? With this setup function is returning blocks for both files - updated and original.

[Acconut]

The code of the azurestore has slightly changed since we migrated to a newer Azure SDK in #1205, but it seems you are referring to these lines:

tusd/pkg/azurestore/azureservice.go

Lines 196 to 210 in a50bc42

	resp, err := blockBlob.BlobClient.GetBlockList(ctx, blockblob.BlockListTypeAll, nil)
	if err != nil {
	return 0, checkForNotFoundError(err)
	}
	// Need committed blocks to be added to offset to know how big the file really is
	for _, block := range resp.CommittedBlocks {
	offset += *block.Size
	indexes = append(indexes, blockIDBase64ToInt(block.Name))
	}
	// Need to get the uncommitted blocks so that we can commit them
	for _, block := range resp.UncommittedBlocks {
	offset += *block.Size
	indexes = append(indexes, blockIDBase64ToInt(block.Name))
	}

The code after GetBlockList does take committed block explicitly into consideration, which seems to be the reason why BlockListAll is used here. However, my knowledge of Azure Blob Storage is unfortunately not deep enough to tell whether we can get rid this somehow. The best way to know is to just try it out. Feel free to give it a shot and let us know if it works. If it does, I am open to changing the logic to only consider uncommitted blocks.

Maybe @omBratteng can shed some light on this as well. As far as I know, he doesn't work with tusd anymore but he wrote the initial version of azurestore and might remember some details why BlockListAll is used.

[omBratteng]

I don't exactly remember.

@MariuszBartnik how are you downloading the blob?