Forum Discussion

pdecarlo's avatar
pdecarlo
Icon for Microsoft rankMicrosoft
Oct 01, 2020

Ask the IoT Expert: Computer Vision based AI Workloads at the Edge

Recent advances in artificial intelligence allow computers the ability to identify objects within image data and video streams in near real-time.  This concept, referred to as “Computer Vision” is now more accessible than ever before, with the advent of services like CustomVision.AI that can allow you to train custom object detection models by providing sample images of the objects of interest. 

As computer power is increasing, it is unlocking the ability run accelerated computer vision workloads on small form-factor IoT devices.   This opens the door to a wave of AI powered solutions which can perform vision related tasks without the need for heavy computational resources located off-site.  This paradigm is often referred to as the Artificial Intelligence of Things or AIOT. 

Applications of AIOT can provide benefit to practically any physical space (factories, smart cities, office space etc.) by allowing for solutions that can detect anomalies, alert on visual cues, and provide insight into the environment by using camera feeds as a new means of sensor input operated on by Computer Vision algorithms.

My name is Paul DeCarlo, Principal Cloud Developer Advocate at Microsoft, with a focus on Internet of Things solutions and the application of Computer Vision concepts within Edge environments.  I have a passion for sharing how to design and develop AIOT solutions with broad audiences and often do so in the form of developer livestreams and the creation of online training materials.

Are you interested in learning more about how Computer Vision workloads might work for you or are you interested in exploring the field to understand what is currently possible?  I am here to answer any and all questions that you might have regarding the topic from all levels, whether you have a specific question on how to implement the appropriate computer vision algorithm to solve a problem for your line of business or if you are interested in where to get started from a beginner perspective. 

I will be monitoring this forum throughout the month of October and look forward to helping answer your questions as they arise. 

If you have any questions regarding Computer Vision based AI Workloads at the Edge, please leave them as comments in this very discussion and either myself or members of the community will be on hand to provide you with answers. To make this Ask the IoT Expert globally inclusive the Q&A will play out in this post and last for the whole month of October.

  • ubriggs615's avatar
    ubriggs615
    Copper Contributor
    Dec 08, 2020

    pdecarlo  I have been following you guidance and I really appreciate all the work you put into these documents. I have been having trouble figuring out however attaching USB cameras as a source. I believe I have the right brands but my goal was to use two USB cameras for a connected project. I might of missed where you discussed that in the past. If so I apologize. 

     

    • pdecarlo's avatar
      pdecarlo
      Icon for Microsoft rankMicrosoft
      Dec 08, 2020

      Essentially, you need to expose the /dev/video* entry to the container and from there it should be accessible from your IoT Edge workload.

      First list the available USB video devices like this:

      ls -ltrh /dev/video*

      Which should list your available devices, then for each device that you would like exposed to an IoT Edge Module, add an entry similar to the following for your devices in your deployment.template.json:

      "HostConfig": {
      "      Devices"      : [
      {
   "PathOnHost": "/dev/video0",
   "PathInContainer":"/dev/video0",
   "CgroupPermissions":"rwm"                      
},
      {
   "PathOnHost": "/dev/video1",
   "PathInContainer":"/dev/video1",
   "CgroupPermissions":"rwm"                      
}
      ]
      }


      We have an article that focuses on connecting CSI cameras that should assist in more specifics, while not related to USB cameras the underlying concepts are very much relevant:
      https://www.hackster.io/pjdecarlo/custom-object-detection-with-csi-ir-camera-on-nvidia-jetson-c6d315

      We also have a a recorded livestream that I believe covers the usage of USB cameras and how to modify an associated DeepStream configuration to use them @ https://www.youtube.com/watch?v=yZz-4uOx_Js

      Let me know if this helps, while not specific to your request it should give details on the bigger picture to help understand how to accommodate a variety of video input methods.

Resources