I didn't expect to be doing this!

Look What We Can Do
Bodo Hoenen

A few months back, I was on a panel discussion in Cannes talking about girls empowerment. I ended that talk with a statement that I did not think would apply so personally just a few months later, “Imagine the joy in a young girl’s eyes when she realizes, by her work, how empowered she is.” A few minutes after walking off the stage my wife called, she was in the hospital with our daughter.

When life happens

My daughter Lorelei got a rare illness, a viral infection, that was causing swelling on her spine and brain. Over the next few hours, she turned from being the cheeky, playful girl she has always been, to a girl who was struggling to breathe, and whose body was becoming paralyzed. By the time I got to her in the hospital, she had lost the use of her left arm, her core muscles were seriously weakened, she could not walk, stand or sit and struggled to breathe and speak.

She was diagnosed with Acute Flaccid Myelitis, a rare polio-like syndrome, and the prognosis was not good. Out of the few hundred cases so far seen in the US, some children had died, most had continuing paralysis and only a handful (less than 5%) had made a full recovery. We could not accept this!

I’m not broken!

My daughter and I decided to tackle this challenge head on. In addition to doing everything the doctors and specialists recommended, we decided to build a robotic assistive arm she could wear. We got this idea from research in exoskeleton assistive devices that have helped paralyzed people walk again. We would do the same by building an exoskeleton arm to stimulate rehabilitation by allowing her to use her paralyzed arm as normal. We would make it open source, reach out to people much smarter than we are and document what we do as we go. This way, we could also help other children we had come to meet over the last few weeks who are also paralyzed.

Through research, we knew big companies were building similar devices, we just needed to figure out how to build it ourselves. Build it 10 times cheaper and make it 10 times lighter, so that Lorelei, a five-year-old, could use it wherever she goes.

“So, that is what we set out to do, we shot for the moon!”

Now, I should probably tell you that we had little to no experience in robotics. In fact, we had little experience in almost everything we have now done. And that is one of the joys of this story. Through open innovation and the help of others, we can be empowered to do amazing things!

We boiled down the challenge

During the hospital stay, my daughter’s body was getting stronger, but her left arm remained paralyzed. She could move her fingers and twist her wrist, but it was unlikely that much movement would return to her arm and shoulder. She could accomplish a very slight movement when her arm was in a zero gravity state like when submerged in water. She was able to move it about 10 degrees in either direction unassisted. This meant that at least a very weak signal was reaching her muscles and that not all the motor neurons were completely damaged. If we could figure out how to pick up those signals, we could use it to control the robotic assistive arm and encourage new neural connections to be made.

This assistive arm would also need to fulfill some functional requirements. Lorelei’s shoulder is really weak, therefore, whatever we built could not weigh more than 150 grams. It would also need to be mobile and be able to pick up at least 400 grams many times an hour for at least 5 hours. It would need to be easily modified and, if it was going to be worn often, at school and with her friends, it was going to need to be light, comfortable, and beautiful!

Before too long, we had a rough idea of what we wanted to do: scan Lorelei’s arm to get precise measurements, print out a 3D printed arm brace for her forearm and upper arm, and connect those two with an actuator that would be controlled through sensors that we would embed into the 3D printed braces.

Asking for help

We reached out by recording videos . To start with, we needed advice on what the fundamentals of this device would need. We created a simple design to explain what we wanted to achieve and reached out for advice. Within days, we started getting the support we needed. From Mexico to Hong Kong, people reached out to help us.

After a few weeks, we had a pretty good understanding of all the components we needed to get something working. With a tight budget, we went shopping. We bought an Arduino, an EKG board, various sensors and built an arm rig to test it all out on.

After much help from many people, we moved to creating a few prototypes, ending up with one that worked quite well. It used Lego type building blocks that we put together to test things out with. And now we were ready to move onto the next step.

Learning as we go

We needed to move onto 3D printing. However, we had some challenges with accurately scanning Lorelei’s arm to get the exact measurements we needed. The trouble was keeping her steady on a rotating axis. Usually, you would use an expensive rotating platform but we did not have that luxury. Instead, we used a plate. I would lie on the floor slowly rotating it and soon got a great scan. Sometimes, simple solutions are the best! With the 3D scan, we had all the measurements we needed.

We ended up designing the braces and printing them flat using PLA plastic —PLA plastic can be heated up easily to become flexible, then when it cools down again becomes rigid. This would allow us to heat it and mold it around Lorelei’s arm, allowing for small changes in the design and fit to be implemented on the fly, simply by heating and bending. To ensure we did not burn her arm with heated plastic we created casts of her arm so that we could mold the hot plastic around the casts instead. [VIDEO]

Hitting a roadblock

The muscle sensor we initially started using would sit on the muscle and measure the electric signal using two electrodes. This is a chart of what my arm signal looks like. You can clearly see when it’s relaxed and when I actively pull my muscle.

What the sensor would then do is filter the raw signal and normalize it. We could then set a threshold that would trigger the robotic arm. When I pulled my muscle, the robotic arm would pull, and when I relaxed, the arm would drop down again.

But, the challenge came in when trying to get it to work for my daughter. Her muscle signals were just too weak. We had to set the threshold so low that it would unintentionally be triggered by her heart and other muscle signals.

Creative problem-solving

I explained to Lorelei that her muscle signals were like trains traveling down a track and that we could not get a reliable signal because the track was broken. A few days later, while at a train station, she looked at me with a spark of genius and said:

“Why are we only looking for a train?”

That’s it! Let’s not filter out and normalize the signal, let’s look at the full signal. Coming from a background in computing I knew a bit about machine learning. With machine learning, you can train an algorithm to sift through vast data sets and learn to pick out data points of significance. This is how image recognition and voice recognition works. What if we used the same process on Lorelei’s raw signals coming from her arm, instead of filtering and normalizing the signals. Could we train an algorithm to recognize when she is trying to pick up her arm and move it?

We again created a video and posted this challenge to ask for advice. Soon, we found some interesting options and a company that was doing something similar. After a few emails and a phone call, they sent us an evaluation unit. We attached 17 electrodes to her upper arm: 8 pairs, and a baseline electrode. Each pair would provide us a separate unfiltered signal. Through these multiple sensors, we would be able to look at all the signals traveling through Lorelei’s arm.

Now we could train the algorithm. Lorelei would pull her arm up and it would record that signal data. She would then relax, and try to push her arm down and it would record that data. After a few training sessions, the algorithm was able to recognize patterns out of the mass of signal noise and we could then use it to control a virtual arm. [VIDEO]

That was it! On to the next challenge.

A call for help

We now knew this approach could work. However, the algorithm and technology we used to pick up the signal is proprietary and the costs were well above our budget to license.

Why should this great technology not be available to everyone? We want to build an open source myoelectric signal recognition technology. Over the course of the last few months, a group of innovators from around the world have joined my daughter to build an open source robotic arm that is 10 times lighter and 10 times less expensive than what’s out there. This is our story so far, come join us and make it your story too!

We stand on the shoulders of giants!

I have a philosophy in life that I live by. Everything we do is thanks to the billions that have lived before us. We have the ability to create, innovate and invent but we cannot claim that anything is our creation alone. So much of the inspiration we use to create comes from others. It’s the shoulders we all stand on. Right from the start of this project my daughter and I knew that we wanted to do this, not only for us but for all the kids we have got to know who are also paralyzed by this illness. We hope that soon this prototype will be reliable enough to share with thousands of children like Lorelei.

By openly sharing what we have done and learned, we can help many others just like we have been helped.

You can find us here: https://www.facebook.com/OurKidsCanDoAnything