Recordings visible in the WebUI, but not present in the configured directory #837
Comments
|
I think I might know what I did wrong, so it seems in the dev version the configuration has changed and the In docker compose I now changed my And in Viseron I added: Will need to make a few tests where something gets detected and recorded to see if the recordings finally end up where they should (for example if they will get moved after a shutdown). |
|
Retested with the new storage settings, something is still off, the log shows: Note the log entry: Now looking inside of the container ( Now, gracefully stopping the container, hoping that Nothing in the logs indicates it was going to move the recording anywhere? The recordings directory on the host shows the correct subdirectory structure, but has not Is this a bug or what am I missing? |
|
Sorry for the confusion! The In v3, the cameras recordings are stored as short If you are in need of having the events as mp4 files, just like v2 did, you can set the The logline and the folder creation is missleading since the file is never created, will fix that right away |
|
Thank you for getting back to me! Yes indeed I saw the segments on ramdisk, but assumed they were something internal, especially seeing in the logs how they get regularly deleted. I was indeed looking for one complete clip for the whole event, expecting it to end up in I now found it in the sources and have a question regarding this remark:
Does that mean, that after the event fragments get concatenated into one event mp4, they will not get cleaned up? If so, why? My naive assumption would be, that it is not necessary to keep the same data twice or is there a technical limitation? Thanks again for helping out, I'll test it shortly. My main issue at this point is the terrible detection quality of the Coral TPU regardless of the model used, but that is not a Viseron problem :) |
|
Did a short test with the suggested setting and the complete My assumption was, that Does this work differently or is my configuration not correct? |
Correct, the fragments are not cleaned up since they are what Viseron uses mainly. The Viseron UI does not utilize the MP4 recordings at all, it only displays HLS playlists using the m4s files. |
That should be working, let me investigate. Just a note on your configuration, the second tier pointing to |
OK, floating some ideas:
Yes, thanks for the hint, I noticed 😅 |
|
I have more logs and a somewhat strange behaviour, I should probably update my dev image to make sure I am running the latest code. So, after making some changes to the configuration (enabled more detection labels) I hit the restart button in the UI, which resulted in the following backtrace: the startup after that looked fine, so I assume all is good, but when I returned to check the events I saw that nothing new has been recorded, although I was outside to test detection. I checked the logs and saw the following errors: Not sure what happened there and why the files were missing. I did not restart the whole container, I only restarted Viseron via the UI. The encoding error is also quite strange, I do not use anything except UTF-8. |
The segments are stored for as long as they are wanted. I do see the need for storing the MP4 files, and you are not the first to have similar requirements. storage:
recorder:
tiers:
- path: /config/tier1
continuous:
max_size:
mb: 50
events:
max_size:
mb: 100
event_clips:
max_size:
mb: 1 # Setting a low value would essentially mean they get moved instantly
move_on_shutdown: true
- path: /config/tier2 # Store only event clips in this tier
event_clips:
max_size:
gb: 100
- path: /config/tier3
continuous:
max_size:
mb: 50
events:
max_size:
mb: 100I will have a look at how complicated that would be. v3 has been in beta for ages now and i would like to get it released soon and i can work on polishing it further later on. |
Yes i noticed that as well,
The missing files most likely a side effect of the unclean shutdown.
Indeed, never seen that before. Can you enable debug logging and see if the error keeps appearing? logger:
default_level: debug |
I guess it is :) So, my understanding was that an "event" is something that triggers a recording when it occurs, so essentially in the end you get a recording of the event. Now, seeing your config proposal I think I did not get that right, it seems there are "events" and "event clips" which is not the same? Ah, I think I know what you mean, an "event clip" is the complete mp4? I am not sure if these names are not too confusing... From the user perspective I consider the gazillion of segments something internal to how the application works, focus of those is to show up and be playable in the UI. They should get rotated according to the configured rules to ensure the system does not run out of space, but otherwise I do not think that anyone or anything except for the application itself is doing anything with those files? The complete mp4 files, as you called them "event clips" are actually what a user would want to grab and copy out of the Viseron storage and save them permanently, the assumption here is, that these files are needed outside of Viseron. Let me float yet another idea :) So, as long as I am within Viseron (i.e. using the UI) I do not really care what format the files are in, the UI plays everything nicely and I can look through the events to check what happened. The format only becomes interesting as soon as I see something caught on camera, that I need to store for myself permanently. This is actually the use case for having an mp4 clip of the event in the first place. So, perhaps a more intuitive way would be to offer some sort of an export button in the UI, allowing to convert selected events to "event clips" and have them saved in a dedicated storage tier. This way there is no automatic duplication as there is no real need for bulk-saving the mp4 clips - they are only needed when the user saw an event that needs to be saved permanently and since reviewing the events is anyway a manual step, hitting a button in the UI to generate an mp4 clip should be totally fine from a user's perspective. What do you think?
I totally get that, as a developer I also do not like when something does not come to an end and just drags on and on, better to get out a stable release with lesser features and then follow up by smaller releases, than polishing a scope-creeped beta release for years :>
Unfortunately I was not able to reproduce it, i.e. the exceptions during the shutdown are there, but the exception at start did not occur anymore. Perhaps some UTF-8 characters got truncated when saving due to the shutdown exceptions and produced illegal UTF-8, which in turn caused an exception on startup? Just guessing.. |
I think placing the segments in the
Yes that is correct.
Yes having a way to selectively export seems to be a good resolution. |
Hmm, as a user who is new to Viseron, "recordings" was the name where I expected to find the actual coherent mp4 recordings. One has to actually dive a bit deeper to understand the difference between "recordings", "clips" and "segments", actually if short segments are stored in "recordings" and if we also conclude, that those short segments are something internal to Viseron, then why isn't the directory where they are stored called accordingly? If you have to keep the names as they are now, then indeed having My later argument was, that if we can get the export feature, then having
Oh it's on the list? I did not see it mentioned anywhere in #619 which is already merged?
Awesome! If there is anything to test, please ping me. I am currently setting things up and waiting for the remaining cameras to arrive, so doing a lot of tests and experiments at the moment. |
Yeah a lot of people seem to have the same expectations as you have.
Sorry my bad, its this PR: #716 |
Indeed, I'd be interested to read and understand how you see these things, because the separation of "Events" and "Recordings", where both are playable videos of the same occurrence is not quite clear to me 😅
No problem, as a user I'm happy that you are interested in feedback and are considering to tune a thing or two. I've been running an open source project too back in the day, so I know how much work it actually is :)
Ah, this must be the point "Allow export of timespan"? |
|
I am still battling the "clips" issue, so my config has: So my expectation was that it would now create those I see the event in the UI, but there are no mp4 files corresponding to this event. A few mp4 files with earlier timestamps have been created, but last I see is from 14:00 while I am looking for an event after 15:08 I assume it is available in the segments format since I can play it in the UI, but dozens of files named as Also, when I go to "Recordings" in the UI, I only see one video from today, although I have dozens of events which I expected to end up as a "recording", I guess this brings me back to the earlier question on what the difference between "Events" and "Recordings" is and what it is good for? I do not see any errors in the log, so I really have no clue why no mp4 "clips" have been created for these events, but I'd really like to download them. Any hints on how to figure out what segments belong to which event, without having to go through everything manually? |
Yes my plan is to allow export of individual events or by specifying a time range. |
|
Could you paste your config? I tried (with your filename_pattern just to be sure) and i get clips created properly. abc@458814191c5f:/workspaces/viseron/config/tier3/recordings/viseron_vscode_camera/2024-11-25$ ls -al
total 5576
drwxr-xr-x 2 abc abc 4096 Nov 25 22:05 .
drwxr-xr-x 9 abc abc 4096 Nov 25 21:55 ..
-rw-r--r-- 1 abc abc 1977969 Nov 25 21:55 21:53:44.mp4
-rw-r--r-- 1 abc abc 1887373 Nov 25 22:01 22:01:04.mp4
-rw-r--r-- 1 abc abc 1831587 Nov 25 22:05 east_2024-11-25-22:05:04.mp4Note that they are stored in UTC so depending on your timezone they might not appear where you think they are. |
OK, sorry, no need to check the config - this was exactly it, the recordings were made, but time was off, and since I did not see any recordings for after 3pm. I thought that they were missing. Which brings me to another interesting question: how can I, as a user, map the time I see in the UI to the timestamp of the filename (of course knowing this nuance one could "manually" convert the desired time to UTC and figure out what the clip name might be)? I would expect them to be the same, since I am interested in a recording of a particular event and the time when it happened is important. Is the issue with my pattern From a software engineering point of view I understand, that using UTC eliminates all sorts of problems (especially the daylight saving time jump horrors), but when exporting the clip I would expect it to be saved in the same time format that is being displayed in the UI, otherwise it's just confusing... Would you be open to the thought of tuning this (or perhaps adding an option for this) at some point? |
|
I agree, it should definitely be in the servers timezone. Will fix for the next beta release |
|
Just released It contains some fixes for the |
|
Cool, thank you for the update, will pull it right away. Going back to the question of "Events" vs "Recordings" - I was hoping you could explain the idea between these two? I understand, that for 24/7 recordings it becomes unrelated to events as it's just a continuous recording no matter what, but I do not think that I have that enabled and I'm still having a hard time figuring out what the difference between an "event" recording and a "recording" recording is? EDIT: need to revisit the statement about missing recordings, may have been a user error :) |
|
I noticed one more issue which I have been following previously and which is still there in the latest dev version: after midnight I check the events, naturally it shows only the ones from the current (new) day. I go to the calendar icon, so that I can select the previous date (i.e. before midnight), but I can't click the previous date - I guess, because it thinks, that there are no events there. I know however, that there have been various events during the day and they have been visible before midnight. Has anyone noticed this? @roflcoopter please let me know if I should rather open a new issue about this particular topic. EDIT: today is the first time that I saw this work - not sure why it did not work previously, but today I was able to select the previous day after midnight. |
|
Tested the feature which should move the recordings after shutdown from ramdisk to the next tier. According to the log this seems to work now: |
You mean when I want to test the dev version? I thought the CI is building those anyway and a docker pull of the |
|
Oh crap i replied in the wrong issue! |
If you experience that issue again, could you take a screenshot and paste it here? |
I am working on some documentation on this. Would be great if i could get your feedback on that when it is done. First i would like to explain how i ended up where we are now. Viseron v2In v2 there are only recordings that are triggered by the Viseron v3But now in v3 we also have the continuous recordings, so now there are two types of recordings. The Events page also displays other events which does not trigger recordings, like face recognition for instance. Going forwardThese are the changes that i see are needed to get everything to make more sense:
What i cant decide is what to do with the Recordings page and the Events page. Anything else that you think should be considered? |
OK, I'll be monitoring this, so far there is for sure something fishy going on with events. I might have had an unclean shutdown now and then, there I understand that something could get lost, but when I It may be related to my ramdisk setup, but I need to specifically sit down and test some scenarios to make sure I understand what is happening, before I open an issue on this.
Both types are still recordings though, so I do not necessarily see any contradiction, i.e. as a user I would expect both of them to be just "recordings".
About
OK, this was the main point that was missing in order to understand the actual difference between "Recordings" and "Events", so, basically not each event has a corresponding recording? Or is there still always some sort of video associated with the event (for instance, recording of the face that got recognized)? Without knowing this subtle difference, the "Recoridngs" page seemed like a duplication of the "Events" page, just in a different layout. Although, if we ignore v2 and the argument that people got used to "Recordings" - one actually does not need both. I guess this is why it was so confusing to a new user like me. And since in the end each recording shows up in the "Events" view anyway, I usually do not bother to look at the "Recordings" page at all. Or am I still misunderstanding things and in reality there is a saved video/segments of an "Event" and then another saved video/segments of the same event, but saved as a "Recording"?
Please confirm, so apart from the 24/7 option, there are no recordings that happen "on their own", a recording of video data is always triggered by an event, correct? If this is the case, then
This would be good, we have a confusing mix of "segments", "events", "clips", "recordings" and almost all of them result in video data saved on disk in one form or another :)
In the sense that segment files will go to
As mentioned earlier, I would simplify things, does unconditionally storing the same recording in an
My personal opinion: drop the "Recordings" page completely and rather allow an additional filter in "Events" to show only show events which resulted in stored video data (i.e. filter on events which resulted in something being recorded on disk). My question to the above mentioned users would be: do they use the "Events" page, or do they just continue using "Recordings" coming from v2 and basically ignore "Events"? The "Events" page provides a bit more info, but in reality "Recordings" is now just a duplication of a subset of what is anyway available in "Events", perhaps with a different layout. For new users who start right with v3 these two pages are just confusing, because one starts to interpret, that they show different data/different videos, which is not the case.
Hmm, especially for starters it'd be good to keep things simple in a "less is more" like approach and make sure the basic surveillance use case is rock solid. What basic functionality does someone expect from a surveillance software?
Advanced:
The above would be my personal surveillance software minimum requirement list. I think Viseron covers a lot of it already, but I have the feeling that you got a bit entangled with "clips", "segments", "events", "recordings" during the migration from v2 to v3 and this entanglement is also leaking into configuration a bit, which makes things that should be simple somewhat confusing :) Once the basic stuff is covered, there are of course tons of nice things that could be tuned, improved and added, but in my view these absolute simple and basic must haves is the essence of it all. |
|
I re-downloaded the dev docker image (thought maybe something got corrupt as I once had a power outage while pulling), but it had no effect, with the latest image I reliably get a crash when trying to access the web interface. What exactly has changed in the last |
|
Your crash indicates that there is a problem with dlib, probably some CPU instruction that is unsupported like Do you have |
|
Well, I did not change anything on that machine between the previous and the last I tried to go back, but did not know how, I am not too good with docker, but I did not find a way to return to the previous
This CPU does not support AVX2 nor AVX, again, wasn't a problem with the previous docker image version...
Intel(R) Celeron(R) N5105 @ 2.00GHz
I have no idea what you are talking about :) I had no clue that something extra needed to be configured? I just did a
I renamed it from |
|
I cant find anything obvious in the most recent changes, or the code base as a whole. I am not familiar with Fedora but on Ubuntu i can get the crash logs by mounting /var/crash into the container. |
|
Well, if it's the docker container which runs Ubuntu, which is supposed to dump the logs to Do you know how I could get to the dev version prior to the last update? Is it tagged in docker somehow? |
|
Help me out please, what do I need to run to start Viseron within the container? I thought it would be However, I am somehow not able to figure out what or where the Viseron script is and how to manually start the backend from an interactive shell inside the docker image? I'd really like to solve this issue, because for over a week I am basically not able to use Viseron anymore due to the crash, which is happening each time I try to access the web UI. |
From my understanding the segfaults are handled by the host and not Docker but i could be wrong here. The previous dev build is sadly no longer available, it is overwritten by the current one. |
You can start the container like this to stop the init scripts from running: And then to run Viseron you cd to Edit: You might end up with other issues with that approach tho, let me run some tests and see if we can get it working |
|
Okay so the only way i could think of to run all init scripts except Viseron is to do this:
Now you can exec into it again and run I guess this is what you were planning to do? https://stackoverflow.com/a/10035594 |
Thank you for the instructions, this worked as expected.
It had some issues with ffmpeg when launched like that from within gdb, but this did not matter for this particular test case - when I tried to open the webpage, python crashed as expected.
Pretty much, yes, the backtrace is huge, so I am not pasting all of it, but I traced it up to where the call is being made in Viseron: It happens when importing the Or, if we mimic what Viseron is importing: Now... I do not use face recognition, so I would prefer not to end up debugging the face recognition module which is probably compiled with optimizations not available on this particular CPU. What I hacked for now just for testing - I removed I do not know if it's easily doable on your side, but if only the components that are enabled in the config would be imported - that would solve the problem. Or perhaps you have better ideas? |
|
Nice work! Components are only imported when they are used, but the problem here is the new API endpoint for When the webserver gets a request this import chain starts:
When looking for this i just searched for Gonna see if i can redesign how the API imports its handlers so that it only imports the endpoints being hit. |
|
Many low end systems that can be used as NVR (e.g. Celeron and Pentium before 2020!) do not support AVX instructions. Building DLIB in a container with appropriate optimizations leads to compatibility problems. |
Agreed, a lot of tools are built for Viseron tho so need to work out how to disable AVX but most tools auto discovers it and enables the instructions. |
|
AVX can be disabled on build:
|
Yes but need to find appropriate flags for Would be nice if i could just globally disable AVX but i dont know if that is even possible. Edit: On the other hand other tools seem to already work, i guess by check CPU instructions at runtime. |
|
Pushed a fix and builds are now running, you can follow the progress here: https://dev.azure.com/jespernilsson93/Viseron%20Pipelines/_build/results?buildId=748&view=results |
Instruction sets can be disabled for virtual machine on hypervisor side. I think it can't be done on Azure or similiar services. |
|
--disable-avx for ffmpeg, but not needed in this case: avx support detected in runtime. |
Latest pull works for me again, thank you for fixing this issue so quickly! |
|
The export/download feature is now available in Have not had the time to update the docs with screenshots yet tho |
|
Awesome, I did not expect that this feature would be available so quickly, really nice, thank you!
Hmm, how can I download a specific event? When I select and play an event, I only have the new download icon on the bottom right, which offers a timeframe, but I don't see where I could download the current event clip which I selected in the list? Is this not yet in |
|
You hover over the event icon to view each individual event and there you have the download button. There is no download button inside the player |
|
Hmm, I don't see that, I only have the "global" download button, but my individual hover looks quite different |
|
So you dont have any text or anything below the thumbnails? |
Nope, it only shows the image as in my screenshot, I do not have the view as shown in yours, puzzling...
I tried shift-reload which I think does the same, also tried from a new private window - always looks the same, as in my screenshot. I am using Firefox 133.0.3 (64-bit) on Linux. Ooh now I see :) I retested in Chromium and there I noticed, that I have a scrollbar on the right side and if I scroll down - then I do see the text you mentioned and the download button. So, this is Chromium:
Now that I actually know, that there is a scrollbar, I tried scrolling in Firefox and indeed it shows the text and the button too, but the scrollbar itself is not visible. Question is - why it so huge in my case, the thumbnails are much smaller on your screenshot and fit nicely without the need to scroll, but well, web stuff is a nightmare 😅 |
|
Now that I know that there is a scrollbar and that recordings can be downloaded (would be nice to get a fix for the UI though), I did some further testing. I noticed that even though I can play the entire event clip in the UI, it only allows me to download small chunks of 12 seconds, which is somewhat confusing. I thought the idea was to be able to download the entire event clip as a whole? Am I missing some configuration option? |
Hi,
I am not quite sure if this is similar to #797 but I simply can't find my recordings, although they are shown in the web interface.
I am using the dev version, the relevant docker compose config:
In my viseron config (I know it's default) I specified the recordings path anyway:
On the host the directories
recordings/camera_1/2024-11-18/get created, but there are not files there.I do not see any errors in the logs:
However, the file:
/recordings/camera_1/2024-11-18/east_2024-11-18-14:08:47.mp4is nowhere to be found. I tried attaching to a shell inside the docker container and the recordings directory is there as empty as on the host:I tried searching for the file in the container root, also in the host system root and I do not see it anywhere. Yet
http://nvr:8888/#/recordings/camera_1/2024-11-18shows quite a number of files which can be played back in the browser.So... where are they located? :)
The text was updated successfully, but these errors were encountered: