This article was written by Jim Solderitsch.
Jim is an Adjunct Professor, Department of Computing Sciences, Villanova University.
In this article, I describe an ESP32 project that I started developing in support of a Special Topics course on CyberPhysical Systems that I taught in the Computing Sciences department at Villanova University in the Fall of 2022. I tell my students that safely controlling a drone in flight is a core example of a CyberPhysical system.
One of the drones most commonly used in education is the Ryze/DJI Tello drone. This drone does not have separate flight controller hardware and requires users to download and learn to use a phone or tablet app to set up and fly it. But, the Tello supports a rich API where flight commands can be sent to the drone over a WiFI connection. The API has been used in several programming environments, including Python (e.g. DJITelloPy) and graphical programming environments (e.g. DroneBlocks) widely adopted in STEM programs in middle and high schools worldwide.
What’s missing in education is the means to explore an open-source hardware approach to control the drone. This approach allows custom control of flight parameters and flight characteristics at the hardware level: sensors, switches, LED status indicators, and a display screen. Many schools now use Arduino devices and development environments to learn Internet of Things concepts and demonstrations. EDC: EduDroneControl is my approach to leverage this interest and combine it with how drones are used elsewhere in education. It offers the thrill of the Tello flying experience to younger users safely and offers their teachers and family members the means to understand and change this experience as they learn Arduino and ESP32 applied to the Tello API.
Understanding the EDC
The EDC project is still a work in progress, but enough features and building blocks are now in place that the EDC can provide a rich and empowering learning playground for drone control. In this section, I outline my own learning objectives as I developed the EDC hardware and software. I then present the main technical aspects of the EDC, including its main hardware and software components and some of the engineering challenges that were overcome to achieve the EDC’s current state.
EDC Development Learning Objectives
I came to the Villanova course, having already learned much about the Tello Drone. I acquired examples of all three Tello models starting in 2018 when the Tello first became available, and in 2018 developed a Javascript-powered software module to fly the Tello via its API: https://github.com/jsolderitsch/tello-nodejs. This was used in earlier Villanova classes to show program-based drone control.
However, it wasn’t until the Spring of 2022 that I learned of two other projects (linked to from the EDC GitHub page: https://github.com/jsolderitsch/ESP32Controller) that described using an ESP32 for Tello flight control and an accelerometer-based means to convert breadboard gestures to flight control actions. These experiences convinced me that students can learn to use this approach in a more controlled and systematic manner.
My Villanova class completed Breadboard prototypes, and we learned to harness the Tello API from Arduino sketches of increasing complexity. It became clear that the designs could be turned into Printed Circuit Board (PCB) realizations, and several versions were produced with the help of a KiCAD course offered by Peter Damaris at Tech Explorations.
As the Tello and the controller hardware were refined, WiFi connectivity and the need to pair a single Tello with a single ESP32 controller in a group classroom setting were investigated, leading to a breakthrough where a WiFi captive portal is used to bind a Tello with a distinct Wireless SSID to a single controller.
Some Tello API commands send back a response that may take some time to be generated. Other commands do not generate a response, and the EDC logic must be tuned to handle these different commands. Recently, a Bluetooth Arduino library for the ESP32 was integrated into the EDC software baseline, allowing the EDC to be paired with gamepads from Video Game systems such as PS3, PS4, Switch and other video game consoles.
I believe the time is right for this knowledge to be shared with a community of like-minded educators who can empower their students to learn how to assemble the EDC, understand its programming elements and fly their drones confidently, and then extend the EDC with added features and capabilities.
Technical Aspects
The EDC project comprises several components: the Tello drone and EDC controller and the software currently implemented using the Arduino IDE. There are several Tello models, and while the project has been under development, the Arduino IDE moved from version 1 to version 2. Keeping up with the model differences and IDE changes provided a challenge.
Although when the EDC circuit is broken down, it is not very complex, with less than 30 components, students were challenged to deal with wires of various lengths. So recent care has been taken to provide wire segments of custom lengths for completing the from-scratch breadboard assembly. PCB controller versions are now available that do not require manual wire connections.
Unassembled Controller
Because we wanted the controller not to require a wire connected to a computer when flying a drone, ESP32 models with embedded battery support were identified.
Hardware
The controller requires a compatible drone. The Ryze/DJI Tello was chosen because of its prominence in educational program environments.
Original and EDU Tello models
The original white and EDU models are shown in the picture, but the EDC supports all 3 models, including the Tello Talent. The white model is readily available through online stores like Amazon, and refurbished models are available at discounted prices. The EDU model supports a richer API.
The controller hardware can be assembled from off-the-shelf components or acquired from me in kit form with all parts included.
Assembled Controller
A complete assembly video is available on YouTube here.
The top row of LEDs is used to indicate connectivity with Tello Drone:
- Red means no connection
- Blue means Tello is connected and is in Flight
- Green means EDC is connected to Tello
The bottom row of LEDs is used to indicate Tello battery strength:
- Red: battery percentage is below 20%
- Yellow: battery percentage below 60%
- Green: battery percentage is above 60%
The small circuit board to the right of the LEDs is an MPU6050 supporting Gesture-based flight. Tilting the EDC causes the Tello to fly in the direction of tilt: forward, back, left, right.
The bigger circuit board to the right of the MPU is an OLED display used to display flight and Tello status information.
Six push buttons control Tello’s take-off, landing, and other dynamic flight actions.
- Buttons in a diamond shape on the right of the board cause Up, Down, Clockwise rotation, and Counterclockwise rotation motions
- Buttons act as toggles so that pushing one again causes the current flight motion to stop
- Button on the top left controls Takeoff (first press) and Land (second press if Tello is flying)
- The button on the bottom left is a Kill switch:
- Press it to make Tello’s motors stop and force it to fall to the ground.
- This is a safety measure for bad flight behaviour.
A black velcro strap is used to secure the Lithium battery on the underside of the breadboard (the battery is not shown, but the connector is visible in the assembled breadboard picture).
In addition, two PCB versions of the controller are available at affordable prices. These are shown in the next few pictures.
PCB Controller SMD Version
The Surface Mount Device (SMD) version requires no wiring and no soldering and is available from the author.
LEDs are the smaller white rectangles to the left of the MPU and have the same colours as the breadboard LEDs. All components are SMDs except the MPU, the OLED and two header strips that the ESP32 plugs into. This version is available fully assembled, ready to plug the ESP32 in. The black velcro strap is used to secure the battery on the bottom of the PCB.
THT PCB Controller Before Soldering
Pin header strips are provided to plug the ESP32 into the board rather than soldering the ESP32 directly. All other holes are used to insert wire leads from components: LEDs, resistors, push buttons, MPU and OLED and then soldered into place.
THT PCB Controller After Soldering
The Through the Hole (THT) version of the PCB will reuse all breadboard kit components: LEDs, resistors, push buttons, MPU and OLED, but the user must solder them onto the PCB. This version can make the experience begin with the breadboard kit into something that is more permanent and resistant to frequent disconnections or loose wires.
Game system Gamepad controllers
The EDC includes support for interfacing video game systems, “gamepads”. Many people will have one or more of these already available. Supported models include PS3 (some clones work), PS4, Switch and others. Two examples are shown below.
Playstation 4 Dualshock and 8BitDo Controllers Compatible with EDC
Software
All of the software used within the EDC was developed using the Arduino IDE extended to support ESP32 boards and with several libraries installed to enable MPU usage, OLED display, and WiFi management. At this time, Arduino IDE v2.1.1 is known to support EDC usage and development and is the recommended version. Advanced Arduino knowledge is not assumed, and various source code examples are provided within the GitHub project to bring the user along gradually.
We use the Arduino Board Manager and Library Manager to install support for the ESP32 hardware and library software used in and by the EDC Arduino sketch files. Video gamepad support requires another Arduino combo board support installation: Bluepad. The EDC is highlighted as an Arduino Community project on the Bluepad page.
The Tello API is documented in several PDFs available from Ryze/DJI, and commands are sent to the Tello from the EDC firmware running on the EDC’s ESP32 MPU.
To learn how to use the EDC with the Tello, we start simple and create more complex iterations exposing new Arduino libraries and ESP32 features.
- Serial Echo function
- Tests that the OLED display is working
- Gesture Tester function
- Tests that MPU and OLED are both working
- Simple flying
- No Gamepad support
- Complex flying
- Including a re-play of the last successful manual flight using EDC
- Gamepad flying
After each iteration, the user is guided by the Control/Monitor/Fly experience. We learn hardware features as we go: OLED display, MPU for gestures, advanced Tello API features, adjusting the WiFi interface and more. Eventually, we learn how to pair video game system gamepads with the EDC via Bluetooth.
One of the innovative aspects of the EDC is how a WiFi captive portal is spawned by the EDC firmware when a new Tello needs to be bound to the controller firmware. This is illustrated in a series of screenshots documenting the process.
ManageTello Captive Portal in the Browser window
The user clicks the Configure WiFi button in the portal
Local Wireless SSID List
One Tello SSID is in the list, and the user clicks that one.
Tello SSID Selected
The Tello SSID appears in the SSID text box automatically, and the user clicks the Save button to make the EDC to Tello binding.
Tello Connected
Once connected, the physical buttons, EDC board gestures, or a connected gamepad can interact with the Tello with no physical connection to the host computer used to program the EDC. In addition, if the EDC is connected with a serial USB connection to the host, commands can be sent to the EDC and then to the Tello using the Arduino Serial Manager. The serial manager can also change which Tello is bound to the controller without using the captive portal.
A video of the EDC showing the breadboard version, Arduino IDE and Tello in flight is available (see below). Gamepad usage is not shown here.
Feedback and Community Engagement
I hope this article will inspire you to share your thoughts on the project and consider investigating drones and controllers with makers and educational audiences you know and are members of. I welcome early adopters who might see the potential in the EDC project to contact me and perhaps acquire the needed hardware.
In addition to the recently added gamepad support, I am considering adding extensions such as Speech Recognition with a Python to serial implementation mechanism. A prototype using the Deepgram speech API was built, and a rough-cut video is available (see below).
Work has also been done to support OTA (Over the Air) updating of EDC firmware without the need for an end user to have the Arduino IDE available.
I encourage readers to think beyond what is already implemented and suggest improvements. One way to interact with me is through the Discussions element of the EDC GitHub project page: EDC Discussions. Another way is to mail to a domain address I have created to facilitate this and other IoT experiences: Email Blog Feedback.
I am particularly interested in whether a full course supporting the EDC, such as those offered on the Tech Explorations site, would be of interest. Also, if anyone would like to commission me to work with you and your group to help you get the most out of your drone program, please get in touch with me.