Archival footage with rclone

[rourke750]

So I wanted to maybe start working on a new feature where instead of deleting content, it could upload it using rclone, keep the record in the db, and then when you wanted to view it it would download the stream and play it back to you.

I was thinking I could work on implementing this and then go through a normal pull request process but I wanted to see if there would be interest in this/ make sure it wasn't already being worked on.

[NickM-27]

Upload it to where? I don't think it would make sense to be hardcoded to a specific place. This has already been discussed #3673

[rourke750]

You would give it a binary and a command with template variables to run. If you haven't seen rclone before https://rclone.org/ but pretty much you'd be able to mount your config file and it would then handle all the upload/download for you.
But the really big change would be the db stuff where you'd still be able to use the UI and events without having those deleted. So you'd still be able to navigate the old content but it would just be slower

[rourke750]

Ah looks like in the other thread they mainly discuss the same things I want to do.
So my next question is if I did all the work coding, testing, etc would this be something welcomed?

[NickM-27]

Right, I'd be curious what @blakeblackshear thinks but I'd say my opinion is that a more general implementation would be preferable. Something that has a secondary "archive" volume that can be configured, It would be mounted in docker and that way it can be cloud or another local drive and frigate does not need to know / care.

[rourke750]

So the other thread was talking about a cool idea where you could mount data from fast and slow storage and have them merged.
But the way I'd want to do this is you can mount any binary you want and frigate will call that executable and you'd have a command for write vs read. So you could use rsync, rclone, scp etc

[NickM-27]

That seems much more involved, like something only experienced users would be able to do. It would also mean the db handling would have to be done manually vs the former approach where I think it would be easier to have frigate handle the db side automatically.

[rourke750]

For the db it could just be a single column field that says if it's been archived or not.
For the previous way you'd still have to write a script that moves the data over that you'd want archived.

[NickM-27]

I disagree, frigate could certainly handle the moving of the files including updating the location of the file in the DB

All the user would have to do is mount the volume and enable archival storage / configure after how many days the recordings are moved in the config and frigate would handle the rest.

[rourke750]

Sorry, I think I'm getting confused. So are you saying it would be fine if frigate handled moving the content, but it would be two docker volume mounts and frigate would move from dir a to dir b and then update the db itself. Then it's up to the user to mount specify the directories mount path and they could do what ever they wanted there

[NickM-27]

So are you saying it would be fine if frigate handled moving the content, but it would be two docker volume mounts and frigate would move from dir a to dir b and then update the db itself.

I'm saying I believe that would be much preferable. The default storage is /media/recordings and a secondary /media/archive-recordings could be used (just as an example)

Then it's up to the user to mount specify the directories mount path and they could do what ever they wanted there

Yes, exactly. Frigate doesn't know and doesn't care what kind of storage it is. All it knows is the main and secondary storage locations inside the container.

[rourke750]

Okay, I think for my use case I could make that work. So would this be wanted/could I work on this?

[NickM-27]

Given that the issue I linked is pinned I believe the answer is yes but let's wait for Blake to give his input.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.