Computer Vision at the edge with Mycelial

In this video, we demo computer vision at the edge using Mycelial to store data locally and send it back to the cloud. We’re using Mycelial to get a computer vision model to the edge where we are then doing inference and data storage locally and using Mycelial to move data back to the cloud (Snowflake in this case). This means that the computer vision inferencing will continue to run even if the network connection is lost. We can then trust that Mycelial will synchronize the data to the cloud when network connection is restored. This has a couple crucial advantages over sending all your data back to the cloud for processing, which we discuss in the video.

‍

Transcript

"In this demo, I'm going to show you an edge device that's running a computer vision application where the inference is happening locally, at the edge, and it uses Mycelial to transmit the inferred data back to the cloud.

Ok, so what exactly does the demo do?

Well, I trained a machine learning algorithm, how to detect defects in washers.

So, in a moment here, I'm going to place some washers in front of the edge devices camera, and you'll see the application identify the washer as either valid or defective, and you'll also see a running count of each type of washer it detects.

Now, what's interesting about this application is that it operates as a local application, in other words, all the processing of images and the associated data happens locally.

Ok, so why is this application a local application, I mean, wouldn't it be better to use powerful servers in the cloud, instead of a less powerful edge device?

Well, there are a few reasons you might need to create an application like this as a local application, and the reasons are all related to network connectivity.

So, let me ask you this, generally speaking, do you think there's enough bandwidth to stream the video from a computer vision application to the cloud, to be processed?

Well, you might have enough bandwidth for a few devices, but when you’re talking about tens, hundreds or thousands of devices then you’re going to quickly run out of bandwidth, because a computer vision application generates a lot of data, mostly in the form of videos.

But even if you had enough bandwidth to support sending all your video streams to cloud servers, there's another issue that could be a problem for you, and that issue is latency.

Let me give you a quick example of the latency issue.

Imagine the computer vision application is making time-sensitive decisions, of some sort.

For example, consider computer vision that's used for the autopilot feature of a self-driving car. Do you think the latency of communicating with a cloud-based server would result in safe operation of the autopilot?

Well, what if an animal steps in front of the car, while the autopilot is engaged? Now, if we were using the cloud for machine learning processing, there would be some non-trivial amount of latency, and the autopilot might not be able to respond in time to avoid hitting the animal, not good!

Now, there's one other reason you probably shouldn't rely on the cloud for an application like this, and that's because if your network connection is interrupted, which is often the case at the edge, then your application stops functioning.

So, given these network-related concerns, it usually makes sense to write an application like this, as a local application. That way, the application continues to function, regardless of the network connection.

Ok, but odds are, you're still going to want a device like this, connected to a network, so that you can get important information off of it, and so you can update the application and the associated machine learning models, when appropriate.

Now, in this demo, we'd like to get the detection data, synchronized to a Snowflake data warehouse in the cloud, for further analysis.

So, you might be wondering, how does the application synchronize to external destinations, like Snowflake?

Well, let's take a quick peek at a simple overview of how this demo works.

So, as I said a moment ago, this application is largely functional, regardless of its network connection. In other words, Machine Learning inference happens locally, and the detection data is stored locally in an SQLite database.

Now, this device does have a network connection, for a couple of reasons.

First, it’s using Mycelial to manage the model delivery as well as coordinate data transfers. But I’ll explain more on this in a moment.

The other reason this device is connected to the network is so that the important information collected on this device can be sent by Mycelial directly to a destination of your choosing, such as Snowflake, or a Postgres Database or some other type of destination.

Now the question is, what exactly should be synchronized from the edge device to snowflake?

Well, you'll define exactly what data goes from the source to one or more destinations, and you do this declaratively with a slightly enhanced version of SQL.

So, the next question you might have is, how do you manage this device, or maybe a better question is, how do you manage a fleet of these types of devices?

Well, this is where the Mycelial server comes into play.

So, a moment ago I said that the Mycelial server is used to manage edge devices, and what I mean by that is that it's the controller that pushes synchronization plans to the edge device, which is running a lightweight Mycelial agent, that executes the data synchronization plan, and additionally, Mycelial can also push ML model updates down to the edge device as well.

Now, one of the nice things about this architecture is that it's resilient to poor network connections, which is quite common for edge devices.

So what this means is, Mycelial can continuously synchronize local changes from the edge device to the cloud, in near real-time, so long as the network is up, but if the device gets disconnected, local operations continue to function, and when the network connection is restored, then Mycelial will quickly pick up where it left off, and synchronize everything that was missed when the network was down.

Ok, I'm just about ready to show you the demo, but first let's take an initial look at Snowflake, and in particular, we're looking at this database, and you'll notice there isn't anything in the public schema, yet, but we'll come back and look at this again, in just a moment here.

So, I'll start placing both good and bad washers in front of the camera, and as you can see, the application identifies the washers as either good or bad, and you also see a running count up here, in the upper left corner of the window.

Now, back in Snowflake, I'll refresh the screen, and now we see that there are tables, and more specifically, there's a detections table, and if we click on the data preview tab, we see the washers and the defects, along with some additional metadata.

Next, I'll disconnect the network cable, and we'll watch the device, via this separate video stream, and notice that the application continues to function, but obviously, it's not able to synchronize the local changes with Snowflake, and just to show you this, I'll refresh the dataset, and we see that nothing has changed in Snowflake.

Now I'll reconnect the network cable, and within a few moments, the synchronization will restart, and if we refresh the data preview in Snowflake, we see that there's additional detection data showing, and in fact, Mycelial has fully synchronized the data between the edge device and Snowflake.

Hey, if you're interested in what we're working on here at Mycelial, I'd encourage you to join our mailing list, to learn more."

‍

‍