Blue Iris with DeepStack Notes

Been keeping notes of what I've see since DeepStack has been integrated with Blue Iris. Moving here for others to see. Hopefully to help them when encountering similar issues.

Iris6: Updates so far: timestamp is when the version was backed up / replaced, not installed. 

C:\Program Files\Blue Iris 5\BlueIris_*.exe

-rwxr-x---+ 1 Administrators avata 17048024 Apr 23  2020 BlueIris_52601.exe

-rwxr-x---+ 1 Administrators avata 17048536 Apr 26  2020 BlueIris_52603.exe

-rwxr-x---+ 1 Administrators avata 17049048 May 16  2020 BlueIris_52712.exe

-rwxr-x---+ 1 Administrators avata 17873880 Apr  5 18:47 BlueIris_54102.exe

-rwxr-x---+ 1 Administrators avata 17875416 Apr  6 10:54 BlueIris_54200.exe

-rwxr-x---+ 1 Administrators avata 17877464 Apr  8 13:40 BlueIris_54202.exe

-rwxr-x---+ 1 Administrators avata 17885144 Apr 14 17:10 BlueIris_54305.exe

-rwxr-x---+ 1 Administrators avata 17975256 Apr 17 16:09 BlueIris_54308.exe

-rwxr-x---+ 1 Administrators avata 17975256 Apr 20 17:40 BlueIris_54309.exe

-rwxr-x---+ 1 Administrators avata 17976280 Apr 22 19:00 BlueIris_54311.exe

-rwxr-x---+ 1 Administrators avata 17976280 Apr 25 14:54 BlueIris_54312.exe

-rwxr-x---+ 1 Administrators avata 17979864 May  6 22:02 BlueIris_54407.exe

-rwxr-x---+ 1 Administrators avata 17982424 May 13 13:57 BlueIris_54503.exe

-rwxr-x---+ 1 Administrators avata 17985496 May 19 16:21 BlueIris_54602.exe

-rwxr-x---+ 1 Administrators avata 17986008 May 20 14:05 BlueIris_54603.exe

-rwxr-x---+ 1 Administrators avata 17995736 Jun  2 17:58 BlueIris_54707.exe

-rwxr-x---+ 1 Administrators avata 17995736 Jun  4 17:04 BlueIris_54708.exe

-rwxr-x---+ 1 Administrators avata 17996760 Jun  8 16:46 BlueIris_54709.exe

-rwxr-x---+ 1 Administrators avata 17997784 Jun 11 17:04 BlueIris_54711.exe

-rwxr-x---+ 1 Administrators avata 18009048 Jun 17 13:03 BlueIris_54801.exe

-rwxr-x---+ 1 Administrators avata 18323416 Jun 19 11:07 BlueIris_54802.exe

-rwxr-x---+ 1 Administrators avata 18327000 Jun 26 11:09 BlueIris_54901.exe

-rwxr-x---+ 1 Administrators avata 18329048 Jun 28 19:03 BlueIris_54902.exe

-rwxr-x---+ 1 Administrators avata 18352088 Jul  8 17:25 BlueIris_54906.exe

-rwxr-x---+ 1 Administrators avata 18358744 Jul 12 16:41 BlueIris_54907.exe

Initial gotchas from testing with single indoor cam

Hell Yeah! Direct Deepstack Integration - 5.4.0 - March 31, 2021

Testing with Amcrest 4k in living room, Blue Iris 5.4.2.2 and DeepStack CPU version for Windows. Doing basic test of walking the same path to see how various options affected detection.

Alerts must be enabled and set to triggered for AI to work even though no actions might exist.

Turning on any "advanced objection" (for instance to only check motion in zones) will limit to to ~30 secs min between AI calls.

Dropping the break to .1 secs still only got down to one pic per 1 sec make and only one AI recog/10 secs (even with make of .5 and breal of .1 only got 23 pics in ~30 seconds of action with only 2 of those AI rec pics. Though on one run I got 4 AI rec pics of 24 in 40 seconds of action.

Setting min time between alerts got me 4 of 23 in 37 secs of action.

Adding also on retriggers got 3 of 23 in 42 secs of action.

Switch from edge vector to simple algorithm and got 4 of 11 in 31 secs (Note 3 of the 4 were in a 9 sec window)

Switch back to edge vector and added hi-def and got 2 of 24 in 41 secs

Increased min obj size and contrast to 608/50 from 550/25 which got me 0 of 8 in 26 secs

Turned off hi-def and got 1 of 8 in 27 secs

Reduce contrast back to 25 got 2 of 18 in 33 secs

set min obj size to 401

Set to send 30 frames to AI per alert and set make to 30 sec. Got 0 of 1 frame IDed

Set break to 3 secs and got 3 of 3 frames IDed

Set break to 0.5 and got 6 of 15 frames IDed

4/12/2021 

On version 5.4.3.5

Setup on most of the live outdoor cams on Iris6.

4/17/2021 @ 16:54 Iris6

Changed DeepStack from Medium to High

Upgrade to 5.4.3.8

4/20/2021 20:08

Upgrade to 5.4.3.9

4/23/2021 19:32

Upgrade 5.4.3.11

4/25/2021

Upgrade 5.4.3.12

Seeing some issues with person detection.

4/27/2021 11:11

Adding email and SMS alerts using sites gmail. Variables available are:

&ALERT_DB The DB record locator for the most recent alert image for the camera.

&ALERT_PATH The path to the most recent alert image on the camera. Note that alert images are not saved to disc by default;

this is controlled with a setting on the Trigger page in camera settings.

&CAM Camera’s short name

&FILE The path to the currently recording clip on the camera

&MOTION_RECT The coordinates of a rectangle enclosing the motion [left,top,right,bottom]; values are normalized 0-1.

&NAME Camera’s long name

&PLATE License plate captured with ALPR if configured

&PRESET The most recently used PTZ preset on the camera

&PROFILE The active profile number

&SERVER The system name as set on the Settings/About page

&TYPE The source of the camera trigger, for example MOTION_A, EXTERNAL, or DIO. The letters following MOTION refer to the motion zones. Most useful for knowing if motion or external sensor trigger

&WAN The current system WAN address as shown on the Settings/Web server page

%x, %X, etc. Date and time. See the table below for a full list of time formatting macros.

&MEMO info like object type and confidence %

Email template

Blue Iris alert &MEMO on &CAM

Motion detected on camera: &CAM 

Name: &NAME

Alert Type: &TYPE

Area: &MOTION_RECT

Profile: &PROFILE

Preset: &PRESET

Memo: &MEMO

plate: &PLATE

Select first 2 attaches

SMS template

Blue Iris alert &MEMO

Motion detected on camera: &NAME

&TYPE

No attachments

Facial recognition seems iffy at best.

4/31/20201 

Upgrade 5.4.4.2

Added script and alert actions to copy confirmed alerts into folders by object found.

Note a space in the comma delimited list of object types will stop the AI from processing!

5/2/2021

Add actions to copy alert files to folders based on object types detected.

Pics are in folders by type AI detected, not what is actually in the shot.

Known types

bird, cat, dog, horse, sheep, cow

lumped into other folder: elephant, bear, zebra, giraffe, person

See robmarkcole/HASS-Deepstack-object.

Bumped min confidence to 60% on DAH412

5/3/2021

Upgrade 5.4.4.2

noticed mouse was not in "animal" list but is in object type list so added things to look for on DAH412

Set min confidence to 60% on DAH628 to cut down on seeing a tree as a person

Added save flagged actions to AM476

AI set to detect 

mouse,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,person,car,truck,bus,bicycle,motorcycle

copyPublic.bat script is

echo %* >> F:\BlueIris\cpLog.out

copy E:\BlueIris\Alerts\%2 C:\_dea\odrive\GD.video\cams\flagged\%1 2>&1 >> F:\BlueIris\cpLog.out

Note first line is just for a debug log

Action looks like 

 

Note this action on fires if the AI detects at least one of the Required AI objects. In this example elephant,bear,zebra,giraffe,person

other is the folder here and &ALERT_PATH is the filename of the marked up alert.

5/4/2021 

made aliases for scripts so could tell in actions windows which was which and moved the bat and log files to the flagged folder.

Added scripts and folders for bear and person.

Also added &MEMO to all the actions so will be in log.

C:\_dea\odrive\GD.video\cams\flagged\copyPerson.bat

person &ALERT_PATH &MEMO

 

5/7/2021 10:16

Upgrade to 5.4.4.7

5/8/2021 14:39

Noticed a typo in copyPublic.bat script made on 5/4 broke all the saving of flagged alerts fail. Fixed it.

Also changed repost when triggered from every 10 sec to every 60 to try and stop Odrive warnings about long wait times.

5/9/2021 13:10

Removing "truck,bus,bicycle,motorcycle,car" from DAH616 AND IPcam124 now detection set to

mouse,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,person:80

DAH628 changed to mouse,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,person:70

Replacing dead DAH626 with DAH345 set to 

mouse,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,person @ 60%

5/14/2021 System hung ~4:42 AM

rebooted and upgraded to 5.4.5.3 (note 5.4.4.8 is now listed as stable)

6/3/2021 

upgrade to 5.4.7.7 to get custom models support. (note almost no detected objects after)

,licence-plate,logo

Note: Turned out you can only have one custom model.

6/5/2021

upgrade 5.4.7.8 to try and sort almost nothing being detected. Helped but still detecting less than before. Seem to be getting fewer triggers causing fewer frames sent to AI.

6/8/2021

Increase extra frames to 30

upgrade 5.4.7.9 to try and sort almost nothing being detected. 

6/9/2021

turned off custom models which helped on cams set to 10 sec end triggers so set all cams to 10 secs.

It appears the second model is causing the issues. Left just the logo model in the custom folder and things seem to be back to normal.

Also turning off ignore static objects as that seems to be causing issues.

6/15/2021 11:33

switch custom model from logos to plates and upgrade to 5.4.7.11

6/18/2021

update to 5.4.8.1. Adds DeepStack stats dialog.

6/22/2021 9AM

update to 5.4.8.2 (note said 6/17/2021 though not there 6/18)

Finally got a licence-plate hit.

10:10 AM

Added plates to saved alerts.

licence-plate

C:\_dea\odrive\GD.video\cams\flagged\copyPlate.bat

plate &ALERT_PATH &MEMO

6/26/2021 14:34

update to 5.4.9.1

6/29/2021 15:30

Swapped logo custom model for plate one since all it tells you is there is a readable plate in the frame and often mistakes signs for plates.

18:09

update Iris6 to 5.4.9.2

Set up a flag archive script for logo objects

amazon,audi,bmw,chevrolet,chevrolet_text,fedex,ford,homedepot,homedepot_text,honda,honda_text,hyundai,hyundai_text,infiniti,infiniti_text,kia,lexus,mercedesbenz,mitsubishi,nike,nissan,porsche,renault,subaru,toyota,ups,volvo,walmart

Cloned cams from Iris4 (BI4) to Iris7 with BI5. Massive CPU reduction so installed and setup DeepStack 5.3.9.2 Working well.

7/10/2021

update Iris6 and Iris7 to 5.4.9.6

Iris6:lmost no alerts much less IDed. Cancelled alerts say DeepStack -1. Tried rebooting and such.

Iris7: still working as before.

7/12/2021 17:00

update Iris6 to 5.4.9.7

Iris6: Still almost nothing getting IDed. Running a clip of me returning from the mailbox registered my face and nothing else. I've unchecked custom models, "default objects and face recog but that did not help. Cancelled alerts say DeepStack -1. Tried rebooting and such.

Iris7: still working as before.

20:07 

reverted to 5.4.8.2

Started see alerts almost immediately. Ran the clip through analyse again and is seeing multiple times.

7/14/2021 11:47

Without changing anything IDs on Iris7 seem to have returned to normal.

8/1/2021

Update Iris5 to BI5 so now the 4 outdoor and 1 indoor cam servers are all on BI5.

Iris6 (south cams) has DeepStack on the same PC. Iris3 (central cams), Iris5 (north cams) and Iris7 (indoor cams) are pointed at docker instance of DeepStack on separate CentOS 7 server.

Created generic shell script to use with Cygwin to sort flagged alerts into folders by the first thing found.

##

## Usage: copySort.sh &ALERT_PATH &MEMO

## where &ALERT_PATH is like DAH412.20210731_143912.4034343.3.jpg

## and &MEMO is like person:93%,dog:81

##

set -x

export FLAGGED=/cygdrive/e/odrive/GD.video/cams/flagged

export ALERTS=/cygdrive/e/BlueIris/Alerts

echo "$*" >> /cygdrive/c/DeepStack/cpLog.out

obj=`echo $2 | cut -f1 -d':'`

mkdir ${FLAGGED}/${obj}

cp ${ALERTS}/$1 ${FLAGGED}/${obj} 2>&1 >> /cygdrive/c/DeepStack/cpLog.out

Set script trigger to

C:\cygwin\bin\sh.exe -x C:\DeepStack\copySort.sh &ALERT_PATH &MEMO

Set objects on the Iris5 cams to mouse,bird,cat,dog,horse,sheep,cow,elephant,bear,zebra,giraffe,person @ 60% and 2 spare frames @ 1 fps. And truck,bus,bicycle,motorcycle,car to the cams looking at the driveway.

Release notes from the help file:

5.4.9 - June 25, 2021

Specify a maximum trigger time [5.4.9.12].

Playback offsets for new alerts [5.4.9.11] will begin at the motion-leading position rather

than the beginning of the entire pre-trigger buffer.

Hold Control as you double-click to open an alert in order to position directly and then

pause at the AI-confirmed frame [5.4.9.11].

Specify zones used for DeepStack [5.4.9.9]

Specify known plates for Plate Recognizer for alert rules; “myplates” and “notmyplates”

objects may be applied to individual alert actions [5.4.9.9]

Enhanced synchronization between alert images and alert processing. Alert images are not

created until timer conditions are met on the Alert tab in camera settings. Processing of “re-

triggers” is improved following a cancelled alert.

Improved handing of DeepStack static objects and objects outside of detected motion areas.

On the DeepStack status page you may now choose to show/hide Blue Iris detected motion

rectangles, helpful for diagnosing objects which do not overlap these areas. Also you may

now drag & drop saved alert analysis “.dat” files into this window.

The “run a program or script” action is now “run a program or write to a file.” This will be

handy for external logging.

5.4.8 - June 16, 2021

RTMPS is now supported for Flash live video streams to Facebook, etc.

A DeepStack status page has been added. This allows you to inspect DeepStack results and

performance following an alert confirmation or when using the testing & tuning options in

the clip viewer.

You may now specify 2 objects together with “and” logic on the “to confirm” and “to cancel”

fields for DeepStack alert confirmation by combining them with a +. For example,

car+licenseplate.

An option to add motion overlays when re-encoding has been added to the Convert/Export

clip function.

5.4.7 - May 25, 2021

Support for DeepStack custom model files; specify the path on Settings/AI. Use a custom

folder name, not sharedfiles as this will conflict with DeepStack’s own models. [5.4.7.7]

select custom models per camera, per profile.

Support for Plate Recognizer’s Make/Model/Color analysis for cars.

A new option to only use Plate Recognizer for alerts where either DeepStack or Sentry has

confirmed a vehicle alert.

5.4.6 - May 17, 2021

As requested by several customers, a continuous + triggered recording mode is now offered

separately from the continuous + alerts mode. This mode ensures that the main stream is

also recorded for cancelled alerts as well as confirmed alerts.

Replacing the Analyze image with DeepStack option on the viewer’s right-click menu is a

Testing & tuning menu with an Analyze with DeepStack option. This option pushes

BVR video frames through DeepStack as quickly as possible (may not always update in real-

time unless you pause and step).

With dual-streaming and direct to disc enabled, main stream frames that are sent through

DeepStack when the camera is triggered are flagged in the BVR file. As these are played in

the viewer with the Analyze with DeepStack option enabled, the image border will be

shown in blue, allowing you to identify precisely which frames were used for the alert

confirmation. A catch-22 arises however if you are using the continuous + alerts recording

mode, as these frames are never actually recorded—use continuous or continuous + triggered if

you want to take advantage of this tuning feature.

When analyzing multiple frames with DeepStack against both “to confirm” and “to cancel”

object lists, an effort is made to choose the “best” confirmed image according to higher

confidences and the presence of (more) faces. By default, the alert will continue with a

single found “to confirm” object, but you can force continued analysis by placing any object

label in the “to cancel” box (even one that will never be found).

The “to confirm” and “to cancel” boxes may now contain labels ending with the * wildcard.

This is handy if you have multiple faces for a single person—you might use chris* for

example to match defined faces for chris_1, chris_2, chris_side, etc.

A new right-click option in the Alerts list allows you to manually cancel and confirm alerts.

5.4.5 - May 10, 2021

(5.4.5.3) The pre-trigger image is now used as the first image sent to AI for alert confirmation.

This is an image captured at first motion rather than at the moment of trigger, so it may be

earlier in time by as much as the motion sensor’s make time.

When recording continuously (including the new continuous + alerts mode), physical

recording is delayed by the duration of the pre-trigger buffer. This allows the opportunity to

record the main stream frames in the pre-trigger buffer as required. In addition, this means

that if you manually record or start/stop using the traffic signal icon, the pre-trigger buffer

will now also be recorded.

By default, the main stream is decoded when a camera is in full-screen or “solo” in the

window. A new option on the camera right-click menu allows you to disable this and force

the continued use of the sub-stream in these cases.

5.4.4 - April 27, 2021

(5.4.4.3) face recognition must now substantially overlap a “person” object detection. Too

many false-positive “unknown” faces result otherwise. A new macro &JSON may be used

for access to raw DeepStack object detection data during an alert.

When the continuous+alerts record mode is used with a dual-streaming camera along with

direct-to-disc, the result is a BVR file which will contain the sub-stream continuously

recorded, but the main-stream only recorded when the camera is in a triggered state.

During main-stream playback, the sub-stream will be upsampled whenever the main-stream

is not available.

The DeepStack detect/ignore static objects option will now force a re-analysis of the image each

10 minutes when not triggered in order to refresh the static objects collection.

5.4.3 - April 9, 2021

(5.4.3.5) Minimum confidence may be overridden by appending a : and the number to any

label placed in the “to confirm” or “to cancel” boxes on Trigger/AI as well as on each alert

action by, for example person:80,car:70.

DeepStack objects recognized only when substantially overlapping areas where Blue Iris is

also tracking objects in order to reduce the detection of static objects.

Require AI objects or use them to skip individual alert actions.

Motion detection updates to produce fewer/larger object rectangles with more effective

edge-vector tracking.

DeepStack “mode” setting. Processing time added to object detection log entries.

5.4.2 - April 6, 2021

A new action set “immediate actions” has been added to the Trigger tab in camera settings.

This action set runs before alert AI confirmation.

You may now specify up to 999 images to be sent through DeepStack for alert confirmation

(or until the trigger ends).

By popular demand, you may now delay recording or snapshots until alerted rather than

starting immediately upon a trigger. This allows the AI time to analyze the trigger before

recording begins.

With a new option on Settings/AI you may now select to save unknown faces to a specific

folder for later review or import.

At startup and after PTZ movement, DeepStack will now analyze the scene for initial objects

in order to prevent alerts on static objects.

The marked-up alert image is now re-saved to the database so that it will appear on the

alerts list.

With a right-click option in the Viewer, you may run a snapshot through DeepStack for

analysis. This will eventually be expanded to include BVR content.

5.4.1 - April 3, 2021

Support for DeepStack facial recognition. Maintain the face list from the AI page in

Settings.

Alerts may now be cancelled with specific DeepStack labels. It’s now possible to Alert only

when “unknown” faces appear or only when there is NOT a dog present, etc.

5.4.0 - March 31, 2021

Support for DeepStack integration (without the separate AI Tools package) via new Settings

AI page. DeepStack may be used for object classification to confirm alerts containing

persons, vehicles, pets, and more.

Plate Recognizer ALPR configuration has been moved to the new Settings AI page. You will

need to reconfigure this integration, now done globally.

Support for legacy SeaIO devices from SeaLevel.

Test button added and UI updates to the camera Schedule/Events page.

You may now select to use a specific global schedule on the camera Schedule page.

“Use defaults” button added to the video encoding configuration pages.

Updated Apple push notification service which will expire February 28, 2021; use 5.3.9.10

or newer for uninterrupted push notifications from the iOS app. The new expiration is