Tuesday, October 1, 2019

A quick and simple video show how my system is setup to return from triggered events with one voice command.

Warning: Alexa voice control in audio may trip your Echo so you might want to mute your Echo's mic.

The video starts with the UPS box high alert (high switches main video while low only highlights camera group on console) triggered triggered so the south cam console (main and console above main on left) has been swapped to the main screen.
The central cam server (far right) has been manually triggered to show the shop deck.
And the internal cams server (far left) has been triggered to show the living room cams.
Once I give the command "turn on video" you can see the consoles all go back to showing all cams and the main screen goes back to showing video from my workstation (the current site wide stream). If the main video had been a playable source it would have also sent the appropriate play command to the device to restart the video.

Note the internal cams server (far left) gets triggered to show the barn just before it receives the command to return to showing all cams.

How it is done:

  • To do this Alexa "turns on" a virtual "video" Homeseer device. (This is also triggered after no new event has been seen for a few minutes.)
  • That device triggers a Homeseer event
  • Homeseer announces the "video on" event is running
  • The event signals each Blue Iris console to to show the "all" group of cameras.
  • It then sends the Harmony pointed at the AV rack to switch the camera switch back to main (input 1) video.
  • It then "turns on" another virtual Homeseer "play" device.
  • It turns off the virtual "video" Homeseer device
  • That "play" device being on and the setting of the current Harmony activity triggers a an event that matches.
  • Homeseer announces which device it is sending play to.
  • That event tells Harmony to send the appropriate play command to the device to restart the video.
  • Lastly after a minute the "video on" event turns off the virtual Homeseer "play" device
Sounds a lot more complicated that it is. The toughest part is sending the highlight commands to Blue Iris but I have a script to do that for you.

Friday, September 6, 2019

Quick note on recovering video files from a corrupted SD card

What happened

Originally written Oct 17 2018:
If you are like most you probably move files from a SD card to a hard disk for processing instead of copying first and then deleting. Generally that saves time but then ever so often something happens like it did to me last weekend when between files one and two I received the error the SD could not be read. Seems the delete of the first file took out the partition somehow. It is still unclear if it was a SD card issue or a glitch on the PC / card reader side. Though I'm leaning toward the card. On principal (gut feelings) if nothing else.

What did not work

My first thought was to run SpinRite on it to try and sort any dead sectors but in this case SpinRite just hung discovering devices. I first assumed this was the corrupted partition causing an issue but even after formatting the SD card, SpinRite still hung discovering devices. Further research points to this being a known issue with some USB devices.

A little Googling will tell you the best way to sort this is to run PhotoRec which will scan the card and reconstruct the files for you. Note the source is on Github but get from the CGSecurity site. But this seems not to actually work or at least not for me. (What did comes later.) I was trying to retrieve files from a Canon G30 camera in this case which might have made a diff but I doubt it. It records in the usual 4 GB mp4 format files mike a lot of cameras do.

I tired various combos of file types to scan for but the best I could get was a mix of mp4s and movs, none of which were playable.  Then I found a post that said there was more info in the TestDisk Documentation.  (PhotoRec is part of the TestDisk package.) On page 45 it says to

type file2_ftyp.mov file1_mdat.mov > test.mov

But I did not have any _ftyp.mov files in my recovered files. I tried using some mp4 and _mdat.mov files that looked like they should match but no luck. Note this insanely slow. After I was able to recover the files I revisited this to try and reverse engineer. After having it recover anything looking like a file (default file type options) I looked for what might match one of the short files that was 3,833,076,414 bytes. 

The nearest matching mov files were 3,731,881,984 a diff of 101,194,430 bytes and 3,832,158,910 bytes a diff of 917,504 bytes. The mp4 files created by the tool are 917,504 bytes. So the second one seems right but does not play and thinks it is the same 16 minutes long as the recovered mp4. This got me thinking digging deeper into the properties of those mp4. None of them matched the run time or data rate of the recovered file. A couple had no info though.

So then I thought what about taking the mov file size, add 917,504 and then find the matching sized recovered file. But that did not work either because adding that to any of the full sized movs makes them larger than the mp4s they should become. Finally I made a sheet to cross ref all the data to make one last try of getting at least one of the recovered from the pieces in case I need to some day. That seems to say this should work
type f24968960.mp4 f73957376_mdat.mov > 3.mp4
And it seems to!  So basically if you end up going this way what you need to do is try all 12  mp4 with each of 14 mov files. That is 168 combos. You might be able to cut that down a bit with some guessing. Here one of the movs was obviously too small and I was able to guess at a couple by looking at mp4 run times vs mov sizes. (See the sheet.) Then by trying all possible combos I was able to get 4 more. If you have the space (about 750 GB to recover this 43 GB of files) and can run while doing something else, like sleeping, your quickest option is to create a matrix like I did in the sheet, copy those commands to a bat file and let it run. Then once it is done you can quickly try and open each one to see if it works.

Bottom line though I was still only able to recover 8 of the 12 files which is not very good. Note too I did "find everything" runs on Linux and Windows and got diff results! (See sheet.) The run using the Windows GUI recovered 2 each less mov and mp4 files. 2 of which matched up in the Linux run.

Note the instructions tell you to upload a file to their site if you can't get it to recover your files correctly and it will try and give you hints to tweak the scan. Problem is the max is 10 MB which in this case is less than 2 seconds of video. Not encouraging plus meant I needed to record something short onto another card to even have a sample. I tried it anyway with an 6.53 MB (< 1 second) file and it said was still too large.

Trying to rebuild partition instead

After I gave up on recovering the files the first time I started thinking maybe I could recreate the partition and maybe I could then recover the files with some other tools. Starting on page 31 of the TestDisk Documentation it gives you some basic instructions. At first I cloned the SD card to an image file and tried options to recreate the partition which did not seem to work. Then I noticed I could list the files and things fell into place. Here are the steps that worked for me.

Select the image or disk to work on Note can give file or device path on the command line as I did here in which case only that one will show.

You probably want to select Intel here. If the partition is toast it come up with None highlighted.

Select Analyse

Which gets you this screen but in this case is unhelpful so select Quick Search

Now it gives you its best guess at what the partition was. Note the type names seem a bit fluid. Just try what it gives you first. Type P, if you do not get a list of files on the next screen backup and use T on this screen to try something else.

You should now see the top folder on the card. Select your way down the tree till you get to the files you want.

Type h to hide old deleted files if they are not ones you want.

Select each file by typing : (the colon key) on while it is highlighted

Once you have all the ones in this folder you want selected type C. Note that is a capital C. Else where case does not matter but on this screen it does.

Select your way to the folder you want the files to be copied to on your hard drive and type C. Note the path of the file on the SD will be recreated under this folder. For instance with the above, I select E:\out so the files will end up in E:\out\ DCIM\124_1005.

Unless you want files from other folders on this SD, that is it. Just Cntrl-C to exit or Q your way back through all the menus.

Noticing this never was published. Cleaned up a bit and doing so now though obviously almost a year later if some bit it missing I would not know. Will update if needed.

Wednesday, August 28, 2019

Wyze RTSP updates

As mentioned in Should you get a Wyze cam? Wyze has released updates to the RTSP firmware versions.

"Update 8/10: Went to add a Wyze V2 out by the gate now that I have power and found they updated the RTSP firmware so though it might now do person notifications. It has disappointing video compared to the 720p Foscam I hoped to replace. And the sensors seem iffy but I'll leave it a bit to see how well the person notifications work."

It even looks like there is now a beta available of the next release. (See below.) So from now on I'll have to note which version the cam I'm talking about is running. The above was in reference to firmware demo_V2_RTSP_4.28.4.41 on a cam I was planning on using at the gate. TODO: add link to Wyze / Foscam compare and how sensors stopped working on 87 in heat.

First off despite the option to turn on person detection showing up for just the latest of my Wyze V2s when I added it and only for that cam for some reason, person detection is still not supported in RTSP firmware. I confirmed it did not work with the demo_V2_RTSP_4.28.4.41 firmware.

I started this post as an update to the above and on the Pan RTSP firmware. See related Pan video. So I went to my downloads folder to get the version number for the Pan firmware and noticed the last version I downloaded on 8/25 appeared to have an older version number (demo_Pan_rtsp_4.20.3.48) than the one I had downloaded before on 8/10 (demo_Pan_rtsp_4.29.4.41). Interestingly when I went to my tablet to confirm which I had installed on the camera the app would not even finish loading. Even after a repower. I had to install the Wyze app on my phone to go check that it was indeed demo_Pan_rtsp_4.20.3.48. The reason seems to be conflicting info. In this post the later firmware is listed both as beta and the current version. demo_Pan_rtsp_4.29.4.41 appears to be the same 5/10/2019 update form the pan beta zip. Note they have started making links to the bin files now instead of the zips and you can not see the folder listing online so file compile dates are now hidden. So it looks like I still need to update my Pan cam. Note with RTSP you can not update the camera from the app. You have to do it manually via the SD card.

Sunday, August 25, 2019

My current setup

Currently 53 (1 added since last pic) cams on 4 Blue Iris servers linked into my home automation so it can highlight cams groups on consoles based on various sensors being triggered. It can also switch the distributed video to show a console automatically depending if the alert and alarms levels are high enough. As in a someone opening the parcel box at the gate will always trigger the cams pointed that way and switch the maina video to that console. Motion at the gate though only highlights the cams on the console those cams are on but only switch the main video is the alarm is armed.

I'm always upgrading or adding but here is the latest picture of my setup. (See older pic below.)

Note the 70 inch display shows the site wide distributed video stream and can be used as a monitor for video editing or any of the security camera consoles. The camera consoles will highlight (just show) groups of cameras when triggered by Homeseer based on triggered motion or contact sensors. Depending on which sensors and the current alarm settings the main video stream may be switched to that camera console as well. Triggered sensor also give verbal notifications. So for instance if motion is sensed at the gate with a glance I can see if I'm getting a delivery or a visit from wild live or a neighbor's escaped livestock from any where there is a TV. I can return the setup to the previous state with one voice command.

A quick and simple video show how my system is setup to return from triggered events with one voice command.

Here is an example of a guy looking for stuff to steal. Since I live in a neighborhood of mostly 5 to 10 acre lots seeing anyone is pretty rare. Predators and loose livestock are a much larger issue. So as I was in the middle of conference call I ignored the first alert but when a second came I could easily make out the guy trying to pull the motion sensor off the tree. Unfortunately he took off before I got down there but he did not get anything and if not for this setup I might not have known he was even about much less had video from 5 cams and good screen grabs of his face for the neighbors and cops.

Future Goals:

Looking at adding a DeepStack server when I get the time. Or something similar. I want something I can train not just tell people from non people like the current Blue Iris plugin. The LightHouse cam used to be able to recognise your pets so this seems reasonable. So I want to work toward telling critter types apart at some point. If I can get it to tell my dogs apart even better. The bare minimum I want to get to is alerts telling cat vs dog/coyote vs racoon/opossum/skunk/armadillo vs pig/cow/horse/deer. Yes I've actually had all those and more in my driveway. There is even been a mountain lion spotted in my neighborhood few believed in till someone caught it on a security cam in a nearby neighborhood.


This is how it happens. Your setup just grows and grows.

August 2019

Added 4 Blue Iris server and dedicated monitor and increased main monitor to 40 inches
May 2017
Added 11th screen (not counting the unused laptop one.) 3 dedicated to cams. Note too indicator lights added behind cam monitors.

May 2016

10 screens, 2 dedicated to cams.
May 2015
Bookshelves added.

August 2013

Mid digital conversion. you will notice 29 low rez cmas on the Blue Iris monitor. Some are new IP cams and some are being pulled from Geovision servers,

May 2013

2 screens switched to larger portrait mode ones.

January 2012

Using 2 Geovision (analog cam servers) here with another down at the shop. Starting to standardize on 1080p monitors too.

December 2011

Just the one 16 cam Geovision server at the house here. Note how dark some of the cams are.

November 2010

5th added. Note upper right monitor was switchable between TV and computer monitor so I could watch digital sources on it.

October 2008

4th screen added for seconde Geovision server.

October 2008

Soon after I moved. 3 screens but no dedicated monitor for cams and even the big screen was analog.

October 2008

Desk setup with 2 monitors.

July 2008

Moving in

June 2007

At the old house

Tuesday, August 20, 2019

Viewing the night sky in color with an Amcrest IP4M-1026

So called Starlight cams seems to be becoming a thing so I thought I'd see what is possible with a camera I already had. You can look at my video Night sky Aug 10th 2019 and Night sky Aug 9th 2019 to see what I'm talking about here.

With a few simple tests the Amcrest IP4M-1026 seemed to have the best low light video and the widest view. Mainly I wanted to try and capture some of the neighborhood's fireworks displays.  My first test with pretty much default settings came out OK with lightning.  It seemed to me it switched to night view earlier than it needed so I played with the options to get the video linked above.

I was surprised how much I could make out by just moonlight. It basically looked like what I would see looking out the window.

Below are the setting I used. You might be able to tweak this a bit further with experimenting but I'm happy with the results. Though the fireworks capture was pretty much a bust as they were a lot more subdued this year and the few that did make it into were mostly blocked by the trees. Thinking now I'll add another cam to close to double the 118 degree view. Note I also did a test with a GW5747MIC 5MP Optional TWO-Way Audio PoE IP Camera 1.9mm 160° Wide Angle Night Vision Sony Starvis HD 1920P Security Mini Dome 5 Megapixel Built-In Microphone and Micro SD slot, Audio in/out Recording but the view was not only no wider but not as good in low light.

Added video without the Moon to show when it is too dark to even see out you can still see the horizon glow of Austin, the planets, at least 1 star, the bard and trees. Here are compare pics from my cameras that catch bits of the same view all from the same time, 8/21/2019 at 10PM local despite what some of the on screen timestamps say.

Screen grab from the posted video
Snapshot directly from Blue Iris
Snap shot from Reolink RLC-411WS (5MP Version) of west end of barn in sky pic
Snapshot from RLC-511W across south deck. Note plane from video and above Amcrest snapshot is just above far horizon. 

Snapshot from Reolink RLC-511 (non WiFi) of mid drive. Ton of IR down here so any bits not lit are hard to see.
From Hikvision DS-2CD2035-I overlooking the pool area (not currently up). Note when the moon is up you can usually see the horizon in this view.
Just in case you think the Amcrest is optimized for night, here is a snapshot from high noon.
Amcrest IP4M-1026 snapshot of sk with sun skirting top of frame.

Sunday, August 11, 2019

Doing the math on pixels per inch at a distance for your cam

Seeing a lot of posts where people are talking up 1080p cams with wide angle. Let's do the math on that. There is a simple calculator online to get the width of the area being viewed by a cam with a given view angle. For example let's take my old work hours the Foscam 9800 series that is 720p and 70 deg view angle.

So at 20 feet the view width is 24.4 feet or 293 inches. 720p resolution is actually 1,280 pixels so when we divide the pixels by the view width you get  4.3686 pixels per inch. So a face, about 6 inches wide will be 26 pixels across at 20 feet with this cam. 40 is considered to be the min. So even with the best lens focused for 20 feet this is still not going to cut it.

For convenience I worked up the charts below from my compare sheet. Note these charts assume the camera is focused for the target distance and not the upscaled resolution some cams advertise. Despite claims, outdoor fixed focus cams seem to be focused at around 20 to 30 feet. You might get a "decent 2X screen grab" of targets within +- 10 feet of that. Of course for a large range of  depth you really should look at a camera with autofocus like the Reolink RLC-411WS I've tried. Though it has had the occasional glitch as well. For long distance something adjustable like the Microseven 6-22mm 3MP Manual Zoom Varifocal Len HD 1080P.

Not those are best case numbers under perfect conditions. While 7 pixels per inch is bare min 14 is the more accepted min. If I change the wanted pixels per inch to 14 the last chart becomes

Note 12 degrees is a fairly extreme telephoto lens in the range of 22mm
70 is standard though up to 90
between 90 and 130 is often called wide angle
180 to 360 are often called 360 view cameras

For a more indepth info look here.

For a visual compare here I am about 33 feet from the cameras.
First an old Foscam FI9804PS 720p and 70 degree view angle

Noew compare that with a Reolink RLC-511-5MP 5MP with auto focus but zoommed to widest view of about 90 degrees.

We can obviously see a lot more  and it looks clearer. Just for comparison here is the view from a 1080p  M7B77-WPSV1 at max zoom to a view of about 12 degrees located about 190 feet way.

The main diff appears to be the angle .

Now crop the images down to just me and the cart and make them the same size for a recognition compare. Note the sizing is done by the browser. You might interpret a bit more with a smart resize.

The Foscam image cropped
The Reolink image cropped

Lastly the Microseven image cropped.
So a 1080p cam with a 12 degree angle and focused for distance yet almost 6 times the distance away would be better for pulling a plate at the gate than the 720p with 70 degree view. Obviously the 5MP at the gate wins though probably by not as much as you would have thought. I should note here too the Foscam was only doing about 1 fps till I ran power out there so I hook up and access point via Ethernet Over Power. I did that so I could stick a Wyzecam V2 out there but that was a bust. Given its 110 degree view you get a shot like this.

First here is the Foscam from the Wyze compare test. I'm a bit further out in this one about 60 feet from the cams. You can see the zoomed Reolink image behind it. Note images were resized to match width of widest image (the Roelink).

Here is the save shot form the Wyzecam V2

And the uncovered Reolink zoomed to its max of 31 degrees horizontal.
Note the glitch in the pic turned out to be a config Blue Iris issue. Inspect sets it to be a generic RTSP and you want to set it as a Reolink.
And lastly a similar shot with the Reolink unzoomed.

In case you are wondering here are some shot with me much closer.
Foscam shot of me at the parcel box
Wyzecam V2 shot of me at the parcel box

One last distance compare back down the drive about 85 feet from the cams. Between the Microseven still zoomed the max.

And an Amcrest 4K IP8M-2496EB with about 112 degree view angle.

For those that think you can just zoom in, here is what you see if you zoom in to match the Microseven.

But there is one more factor to consider, focus. Autofocus can add a lot though when it gets it wrong it can make your cam useless too. Here is a close up shot with the Foscam where I'm just 7 feet away. The cart maybe double that.
Note the plate looks good but the logo on my cap is blurry. I'm too close for the range this cam is set too.
Here is the same shot from the Reolink zoomed out to the max.
Notice both my face and the plate are in focus because the camera has adjusted.
The down side, something flying too close to the lens can leave you looking like this.