Three years ago Neil Deshmukh ’20 had never heard of artificial intelligence (AI). Today, utilizing the technology, he has developed two apps that are being used around the world and is currently working on a third. And it all started because he wanted to keep his brother out of his bedroom.

One day while reading about applications of AI in the real world, Neil decided to try to build a device that would recognize the difference between both him and his brother’s faces. Utilizing the coding skills he learned while on Moravian Academy’s Pennsylvania Junior Academy of Science (PJAS) team in middle school, he inputted a few lines of code into his computer and put his device to the test.

“If it was my brother, the computer would lock the door and if it was me, it would unlock the door,” said Neil. “As soon as I learned about AI, this was the first thing I wanted to try.”

Luckily for people the world over, the device worked, and Neil’s curiosity in AI was solidified.

“When I first started, I thought AI was like a superpower because you can basically do anything that’s within the realm of knowledge,” said Neil. “We can use artificial intelligence to replicate human thinking and solve problems that we once thought were impossible to solve. We can create solutions to help people who really need help.”

Fascinated, Neil dove into exploring more about the technology, creating apps, within the last year that have life-changing implications for underserved populations all over the world. Neil was motivated to create the apps, PlantumA, which helps detect and diagnose plant diseases, and VocalEyes, which helps visually impaired users navigate their environment, by significant people in his life.

“When you have a personal connection to a problem, you have better knowledge of the [intricacies of the] problem and how to solve it,” added Neil. “Otherwise, you are coming at it from the outside and, sometimes, getting into the perspective of people who have the problem, that’s the hard part.”

PlantumAI: Treating Plants with a Single Tap

Inspired by his grandfather, who is a farmer in India, Neil launched his first app, PlantumAI, in May 2017. Through frequent family trips to India, Neil met rural farmers and learned firsthand about crop diseases and the impact it has on farmers’ lives.

“I have a lot of friends in India and for most of them, their entire way of life is dependent on whether their crops succeed or not,” said Neil. “When farmers are trying to get rid of a [plant] disease, they’re worried about getting food out to their family. At the time, there was no way for them to solve this problem and it was all based on guess and check. It was like flipping a coin to see if they could get their crops to grow.”

Designed by Neil to be “an instantaneous disease expert in your pocket” for farmers, PlantumAI is able to detect, diagnose, and provide treatment options for plants with a push of a button. The app allows users to take a picture and immediately figure out what strain of disease, or what nutritional deficiency, is destroying the harvest. The app also runs completely offline, benefitting remote users that often don’t have reception.

“In testing the app, the most interesting feedback I received was from one farmer ”who told me he wasn’t using [the app] because he had a friend at the university. For the past ten years, whenever he came across a disease, he would travel to meet his friend who is a plant pathologist. He told me that the app was amazing, but I realized he wasn’t looking for a single-solution, he was also interested in maintaining the human connection,” concluded Neil.

To bridge this gap, Neil established a partnership with a local university in India that works with the farmers. Now, professors teach farmers how to use the app to treat their crops, which in turn minimizes the workload on the university, while still maintaining a cooperative experience.

Currently, Neil is working with a group at Penn State University to work with farmers in Africa to see if the technology will work there.

“So far, the app is doing a lot better than I thought it would. The farmers have picked it up really fast because they are seeing that it can help them,” said Neil. “The cool thing is that because they are taking so many pictures with it, the app is getting smarter and smarter. The way that AI works, the more images you get, the better and smarter the algorithm becomes.”

VocalEyes AI: Computer Vision for the Blind

Shortly after launching PlantumAI in June 2017, Neil began developing a second app to aid the blind and visually impaired. Working with two other students while attending MIT Launch, a summer entrepreneurship boot camp for high school students, Neil convinced the team to work on a solution for his grandmother.

“The problem that I saw was that for a lot of visually-impaired people, my grandmother, in particular, it’s hard for them to tell the difference between a bottle of water and a bottle of Coke. Also, If there’s text she wants to read, it’s hard for her to do so,” said Neil.

To help her and others like her, the team created VocalEyeAI, which utilizes a cell phone to take a photo then describe it to the user via the phone’s speaker. During the creation process, Neil had the opportunity to visit the Perkins School for the Blind, located in Boston, Massachusetts, to test out the app with students at the school.

“The kids loved it so much. They were taking pictures of everything just to see what was around them,” described Neil. “One student described [his existence] , before the app as living in a sphere because he had to feel around to sense objects. With the app, he could take a [broad spectrum picture and know that there were two people walking in the distance, a room over there, and a kitchen over there… It broadened their scope of the world.”

Just as with PlantumAI, Neil received insightful feedback from the students.

“I originally built the app with a button, settings you could change, and a flash feature,” said Neil. “But when I talked to the kids, they explained that they didn’t need any of that. They were just tapping the screen. I realized I didn’t need multiple settings for different types of objects. I needed to make a singular experience that would work for any scenario. This changed the app to a simple whole camera view. Anywhere you tap on the screen will take a picture and it will tell you all of the information you need at once.”

VocalEyes is available for Apple and Android devices and already has several thousand downloads. The app, which has been featured on CNN and other major media outlets, is continuing to be tested at several institutions for the visually impaired across America, India, and Egypt.

Creativity’s Involvement in AI

“Farmers in India are a completely different population than kids in Boston, but in working with them, both groups had a childlike fascination that technology can do this,” said Neil. “AI is capable of anything, but it’s really up to the person that makes it. The creativity of people in the field is super important right now. It’s not based on what you know, but on how creative you are with what you know. By using creativity in new fields, we can create solutions that don’t necessarily need new technology or new ideas.”

Neil credits his passion for coding with his experience at Moravian Academy, particularly his involvement in PJAS and the influence of his Middle School science teacher Mrs. Joanne Daniels.

“I wouldn’t have started coding at all if it wasn’t for PJAS and the entire program at the Middle School. Mrs. Daniels helped me figure out that I was interested in building things,” said Neil. “Moravian stands out because, in every subject, we aren’t focused on memorizing information, but on how the knowledge can be used in the real world. The school teaches you the fundamentals of what you need to know, along with the skills.”

When asked to give advice to younger students pursuing their passions, Neil stated, “If you really want to make a change, it’s not dependent on how smart you are. It’s dependent on how passionate you are about learning something and actually making that change. If you are passionate enough, the resources are all around you.”

This piece, highlighting Neil Deshmukh ’20, was featured in the Winter 2019 Moravian Academy Journal.