Separate continuous recording from clips

[jvallery]

Describe what you are trying to accomplish and why in non technical terms
I would like to be able to store continuous recordings on a different path than motion/object clips to externally define different retention polices and physical locations.

Describe the solution you'd like
An independent folder hierarchy for continuous recording vs clips.

Using a FUSE client, clips are ideally written to cloud storage. Given the volume of continuous recording, it's not practical to store the full recordings in the cloud.

Describe alternatives you've considered
No alternative solution is obvious given the way files are named.

Additional context
None

[NickM-27]

Frigate uses the exact same recording segments to supply both the continuous and event based recordings. You can already define separate retention periods in the config and they will be deleted when the relevant retention config no longer applies.

[NickM-27]

additionally, storing the segments directly to the cloud is not really a good solution. If you want cloud storage you would be better off with a solution that exports the event clips and uploads them separately as a single file

[NickM-27]

#3673 also seems like it could be relevant

[jvallery]

additionally, storing the segments directly to the cloud is not really a good solution. If you want cloud storage you would be better off with a solution that exports the event clips and uploads them separately as a single file

@NickM-27 - The way this works with FUSE is there is a local write cache. Once the file handle closes, it uploads the file to the cloud and purges the local write cache. It actually works amazingly well.

[NickM-27]

The part that would not work well is viewing. Viewing each segment is much more complicated without Frigate, so if a video needed to be viewed in the event that the Frigate server was damaged it would be much more difficult vs viewing one contiguous event file

[jvallery]

The part that would not work well is viewing. Viewing each segment is much more complicated without Frigate, so if a video needed to be viewed in the event that the Frigate server was damaged it would be much more difficult vs viewing one contiguous event file

This also works great. There is pre-fetching in Blobfuse. I've had no issues reading/writing video directly to Azure from a deployment at my house. The big issue is just the Azure bill associated with continuous recording!

[NickM-27]

What software are you using to view the recordings?

[jvallery]

Frigate! Also home assistant.

[NickM-27]

Frigate! Also home assistant.

right, so you are missing my point entirely then. If a circumstance occurred where Frigate was lost; say for example a thief broke in and destroyed the Frigate serve. Viewing those recordings would be very painful, either watching small segments at a time. Using Frigate at all would require setting Frigate up again and having an update to date (down to the second) database to be used.

Meanwhile, backing up full clips makes it much easier to watch in this situation

[jvallery]

Frigate! Also home assistant.

right, so you are missing my point entirely then. If a circumstance occurred where Frigate was lost; say for example a thief broke in and destroyed the Frigate serve. Viewing those recordings would be very painful, either watching small segments at a time. Using Frigate at all would require setting Frigate up again and having an update to date (down to the second) database to be used.

Meanwhile, backing up full clips makes it much easier to watch in this situation

I have a daily backup that synchronizes the db files along with all the config files up to the cloud as well. If my server was destroyed or stolen, I could easily recreate it.

[jvallery]

Frigate! Also home assistant.

right, so you are missing my point entirely then. If a circumstance occurred where Frigate was lost; say for example a thief broke in and destroyed the Frigate serve. Viewing those recordings would be very painful, either watching small segments at a time. Using Frigate at all would require setting Frigate up again and having an update to date (down to the second) database to be used.

Meanwhile, backing up full clips makes it much easier to watch in this situation

I have a daily backup that synchronizes the db files along with all the config files up to the cloud as well. If my server was destroyed or stolen, I could easily recreate.

[jvallery]

Also, let's clarify "clips". In my case, I just want the moments where the object was in frame (minus a configured offset to account for detection latency) going to the cloud. I want continuous recording going to my local NAS.

[jvallery]

Also, let's clarify "clips". In my case, I just want the moments where the object was in frame going to the cloud. I want continuous recording going to my local NAS.

The experience I have with continuous recording disabled, and only object-based detection is working great in my current configuration. I'd also like to retain 24/7 just so I can review if for some reason object detection had a false negative. Today, I'm not storing this video.

[jvallery]

Also, let's clarify "clips". In my case, I just want the moments where the object was in frame going to the cloud. I want continuous recording going to my local NAS.

The experience I have with continuous recording disabled, and only object-based detection is working great in my current configuration. I'd like to also retain 24/7 just so I can review if for some reason object detection had a false negative.

[NickM-27]

Right, but frigate stores them as a single file. So what you're suggesting would require duplicating segments and managing those separately, increasing the amount of storage used for recordings as well as making recording management more complicated.

[NickM-27]

And to be clear, I'm presenting the problems with the approach in general not necessarily in your specific case. To do it the way you're doing it requires a lot of external factors like DB backups.

To solve the need you're specifying, I think there are better solutions than duplicating storage that frigate can support natively and would be easier for users to configure without requiring external configuration

[jvallery]

I'm not sure how this would work unless Frigate truncated the videos along the detection boundaries. I'm not going to post process the continuous recording, identify the objects and extract the clip, and upload it. I don't want to upload all the video due to cost. I want to keep the moments with object detection indefinitely.

What solution do you suggest?

[jvallery]

I'm not sure how this would work unless Frigate truncated the videos along the detection boundariess. I'm not going to post process the continuous recording, identify the objects and extract the clip, and upload it. I don't want to upload all the video due to cost. I want to keep the moments with object detection indefinitely.

What solution do you suggest?

[NickM-27]

Right, and this exporting of review items / events is already supported in the UI and via API. In the long term I can envision frigate has a "cloud backup" feature where there is a separate feature that can be enabled and a separate volume that can be mounted. Whenever a review item ends it is exported and put into that backup volume

[jvallery]

That would work. I do recommend you investigate direct object storage support via the S3/Blob APIs. That would simplify things. Buffer into memory, write the object async. Lots of NVR solutions do this in the commercial space.

[jvallery]

That would work. I do recommend you look into direct object storage support via the S3/Blob APIs. That would simplify things. Buffer into memory, write the object async. Lots of NVR solutions do this in the commercial space. Lots of NAS devices support Object APIs natively.