Disaster strikes. Earthquakes. Floods. Hurricanes. Explosions. Thousands of people need immediate help, but widespread destruction can mean they’re not found for weeks. How can workers assess such situations more quickly and start aid sooner?

Drones. Fleets of drones.

“For those scenarios, it is hard for people to get in. This kind of research is very, very beneficial for those kinds of applications,” says Assistant Professor of Computer Science Shaoen Wu.

He conducts his research on drones for the sole purpose of helping others. He proposes to create a cyber-physical system (CPS), which is an integration of computation, networking, and physical processes. His CPS would be called cooperative unmanned aerial vehicles enabled panoramic video surveillance. He recently received a National Science Foundation grant of nearly $238,000 to support this three-year project which ends in 2017.

“This CPS … generates mobile real-time video panorama using a fleet of cooperative drones equipped with cameras,” Wu says.

“Right now, we have the ability to fly a single drone, but the camera view is limited,” he says. “For large regions, especially after large disasters like floods or earthquakes, our hope is to fly a group of drones and actually see the whole situation. Those drones will collaborate and automatically control each other. If you use a single drone, you obtain a very limited, partial view. You have no global view of that region. So with this research, you will have that capability.”

Currently, drones send separate pictures to a home base where the images are stitched together into a panorama. The problem is that the visuals often overlap, and sending the redundant information wastes energy. If the drones could piece the visuals together immediately and send a completed panoramic view back, it would keep the battery-powered aerial vehicles in the sky longer—that is what Wu is working toward.

Stitching static cameras

Step by step, Wu is achieving his goal. So far, he’s been able to create a panorama from static cameras. His current challenge is how to create panoramas when the cameras become mobile. Instead of jumping straight from static to flying cameras, Wu is experimenting with cameras on top of carts.

“We plan to install at least four cameras and make a small cart, and we will use a remote control to move the cart, so it will be mobile.” As the cart moves, the cameras will automatically create panoramas, theoretically. Once the cameras on carts work, Wu will set to the sky.

Several students assist Wu with his research. “We have seven graduate students and three undergraduates working on this right now. We are very glad that we can have the financial support for multiple students. I always think involving students in research is very important, because it’s not only about the outputs for faculty members but also to enhance student experiences. They attain real life experience. My students are very excited!”

Small beginnings, big dreams

Wu is from a little community in China where he had minimal exposure to technology. “In China, I grew up in a small countryside village; we did not see any computers.”

When it came time for Wu to choose a college major, he took a chance and picked information technology. “I had no idea what major to choose. My high school did not have a computer. I had never seen a computer.

“I can imagine back to those moments when I was wondering what career I will have. When I look back, I feel that God led me this way. I feel it’s just amazing and unexpected .… It’s a very fantastic time for me to work here because Ball State is very supportive of faculty in their research. They give a lot of effort to encourage people.”

Wu’s vision: smart houses

“My biggest hope is to help people live in much better environments with my research. The drone is just part of my vision.”

Wu hopes to create smart houses in which “each room will know your individual preferences and that of your family members,” he says. “The system will learn everything by itself, so we call it artificial intelligence.”

With the use of smart vents, a smart house will automatically adjust room temperature based on individuals’ preferences. “Using various sensors placed in the room, the system will communicate with the vents and tell them to open or close, thus automatically adjusting each single room’s vents according to the set temperature,” Wu says.

“The next step is for the system to recognize that’s you, not your dad. The system will memorize your preferences from the past and learn,” he says. “After that, it’s about activity recognition. Sitting, walking, and exercising. Once we recognize it’s you, but you are doing different activities and your preference is different, the system will learn from you. It’s ongoing.”

This system senses motion in a way similar to Kinect for Xbox 360. “We can use different sensors to detect your gestures. We do not want to use anything that you have to wear on your body, because that’s not convenient,” Wu says. “We would like to use sensors embedded in the environment, so the environment will be smart. You do not have to do anything.”

His smart house system is similar to the Nest thermostat, but Wu says his system is more advanced. “The Nest only replaces the manual programmable thermostat widely available in markets, in that it intelligently learns the user preference on temperature and then makes adjustments without manually programming the preference.”

However, Wu’s system would learn to differentiate between household members, as well as the various activities of the different members. “It then adjusts the temperature based on a particular user and his or her activity. So, it is smarter and more user friendly. It is based on artificial intelligence algorithms. It will not only save energy, but also definitely result in a completely different user experience in our future housing.”

Wu’s smart house system can make a big difference in energy use. “Energy is wasted when unoccupied rooms have as high of a temperature as the occupied ones. With this … system, in which each room is individually controlled, if there are no people in a room, the system automatically turns off. That saves energy. It is hard to say, but from the report of what we have so far, at least 20-35% of energy can be saved.

“The long-term goal is for the whole world to be smart,” Wu says. “That’s the final goal for CPS—to use computing technology in the physical world. If we can make a contribution to the whole world, I would think that’s the most rewarding part.”