Flying with Artificial Intelligence
When I was a student at Syracuse University, I studied mechanical engineering because I wanted to learn about drones. Unfortunately, the curriculum did not dive too deep into that subject. Aside from learning the basics of aerodynamics, I didn’t learn much about the components used to assemble a drone so naturally, I bought one.
Once I was comfortable using it, I decided to test its capabilities and discovered “active tracking”. Active tracking is a mode on most DJI drones that allow the drone to follow a subject that is also moving. I decided to test it out. My roommate and I went to an open field and sprinted across it to see how accurate it would function and it did not disappoint.
So, How does it work?…
DJI assembles all their drones with visual sensors and GPS tracking technology. Firstly, the gyroscope sensor allows for stability when yaw, pitch, or roll are changed. When a drone is simply hovering, the gyroscope is what mitigates the angular changes by the wind. The second sensor is called the accelerometer. The difference with this sensor is that it collects information about linear movement. Its job is to constantly update the drone about what is directly below and in front of it, and how far in distance the object is. Now that we’ve covered how drones move to avoid obstacles, we enter the world of AI.
Object detection is a field within artificial intelligence that can understand images. This technology branches into facial recognition, object detection, image regeneration and autopilot features in the automobile industry. It does get a bit confusing to program such a thing from the foundation.
Luckily for beginners such as ourselves, we have a library in Python that allows us to use easy to integrate computer vision. So let do that!
from imageai.Detection import ObjectDetection
import os
execution_path = os.getcwd()
detector = ObjectDetection()
detector.setModelTypeAsRetinaNet()
detector.setModelPath( os.path.join(execution_path , "resnet50_coco_best_v2.0.1.h5"))
detector.loadModel()
detections = detector.detectObjectsFromImage(input_image=os.path.join(execution_path , "image.jpg"), output_image_path=os.path.join(execution_path , "imagenew.jpg"))
for eachObject in detections:
print(eachObject["name"] , " : " , eachObject["percentage_probability"] )Within Python we will need to access a library called ImageAI which we imported on the first line as well as a path for our files that is defined by a variable. The next set of lines are defining the object class, using RetinaNET on our photos, and parsing through the input images and output images path, iterating through all photos and displaying an object with a percentage of certainty.
As you can see from Figure 3, our code has updated the photo with a title for the object and the percentage of how certain it believes that classification to be. The bus is given the title “bus” with a 98% chance that the program is actually detecting a bus; whereas the person on the bike is given a 77% probability of certainty. Cool, right?
What does this mean for the future? Well it means that you’re always being watched by Big Brother. Realistically it’s an advance in technology that will be beneficial for unmanned aerial vehicles and any other camera based technology. It’s already being implemented in Tesla cars as a mean for the car to understand what is surrounding it. I can see this being used for search and rescue missions where cameras make it easy for people or animals to be found. It can also be daunting to think about perhaps maybe your facial features are being recorded in some database through public security cameras. Nonetheless, I’m not here to scare you so here are some photos that I’ve taken using my drone (found below).
If you enjoyed this content, please check out these articles I used that go further into detail about object detection and the hardware needed to mimic what we see in the industry!
