Intelligent Stereo Vision
- Intelligent analysis of 3D environments using custom-built stereo vision cameras and an AI component hosted within the camera.
- Predefined rules are available to identify scenarios such as unattended luggage and unattended children.
- Ability to measure objects live in 3 dimensions.
- The camera is built using Intel’s Realsense and NVidia’s Jetson component which has a built-in GPU.
- On-device machine learning capabilities enable the solution to dodge network challenges.
- Involves a dynamic rule engine where rules can be written for each scenario that needs to be tracked and the camera shall automatically identify such scenarios.
- Python 3 based application to read camera stream using Realsense SDK.
- Preprocessing of frames performed using OpenCV and Leptonica based image processing.
- The rule engine was developed using Lex and Yacc. Rules were parsed and converted into a state machine.
- Rule evaluation was performed while streaming through the video frames and making state transitions.
- Yolo-V3 based image classification was developed and the model was extended for newer entity types.
- 3D measurements were performed using a homegrown algorithm written in python. This involved StereoDNN neural network based modeling and prediction.
- Collision Detection was performed using a collider based overlap methodology.
- Temporality was maintained in order to be evaluated in the rules.
- An Android application was developed to control and deploy the rules onto the camera.
- Python 3
- Jetson SDK
- Intel Realsense SDK