Digital coach corrects movements in real time

Fitness within your own four walls is popular. According to figures from the Federal Statistical Office, there were almost 13 million fitness devices in German households in 2023, i.e. around one in four households had one device. One argument for this for many people is the time factor: If you train at home, you save yourself the trip to the gym. But there is also a disadvantage. Beginners in particular often risk accidents and injuries due to incorrect loading and movement sequences, especially during strength training.

Instructions from books or YouTube tutorials can counteract this, but they cannot provide individual tips and suggestions for improvement. This requires human trainers – or artificial intelligence, such as that developed by scientists at Drexel University and Michigan State University.

If the personal trainer is an AI

They recently presented a digital fitness trainer for the home under the name Biocoach at a specialist conference and in a preprint paper. Biocoach uses AI and computer vision to analyze exercises in real time via a camera and give athletes feedback on their posture.

This type of computer-based training is not new in principle; With the Qualcomm Exercise Video Dataset (QEVD), there is already a framework that can use video recordings to assess whether an athlete is performing exercises correctly and, if necessary, provide assistance – such as “legs wider apart” or “straightening the hips”. This is already used by some fitness apps.

More precise training thanks to the “biomechanical eye”

The special feature of Biocoach is that the prototype uses a second complementary system to assess movements in addition to pure video analysis, namely a “biomechanical eye” that creates a virtual 3D skeleton of the body in real time and is therefore presumably able to give more precise instructions. “Our goal was to develop a system that could do more than just look at pixels and make a general comment,” says Feng Liu.

For training, the team first took the QEVD data set and added around 2,400 additional annotations. Instead of just saying “go deeper,” Biocoach gives more specific instructions like “bend your knees to 90 degrees” or “keep your back straight to relieve stress on the spine.” In the best case scenario, it not only explains to the trainees what they are doing wrong, but also why correction is important to avoid injuries.

Knees and elbows in view – thanks to a 3D model of the skeleton

The system then uses the improved data set to assess the movement sequences. This works via a two-stage process: The training recordings are transferred in real time to the biomechanical module, where additional information is added and evaluated. It uses the user’s individual body measurements such as height, limb length and body shape, and creates a 3D model of the user’s skeleton “to capture biomechanically based body movements,” as the study states.

Thanks to this 3D model, it is possible to relate the joint angle to the specific stature of the athlete – and at the same time to identify the most important joints and movements for the respective exercise. When doing squats, the bending of the knee is important; when doing push-ups, the elbows play an important role.

Great potential for AI fitness applications

For the study, researchers from Drexel University and Michigan State University focused on 23 popular fitness exercises, including squats, lunges, jumping jacks and push-ups. To test the effectiveness of Biocoach, they pitted it against several well-known computer vision models. The prototype performed better than generalized models when assessing the movements and suggesting improvements.

“We believe our work could ultimately support exercise and physical therapy apps to complement the expertise of human trainers and caregivers,” says study leader Liu. In fact, artificial intelligence is said to have great potential in the area of ​​personal training and many people are already using AI to create training plans – even if this is not necessarily recommended, as a colleague discovered.

For now, the developers at Biocoach point out that accuracy depends on the quality of the extracted 3D skeletons. Clothing that is too loose or poor quality video recordings could distort the analysis and, in the worst case, even lead to incorrect recommendations. If you want to do sports at home, you should probably continue to seek advice from a professional at least at the beginning.