Put Me in, Coach: AI May Soon Assist Pro Football Coaches

by Jamie Beckett

Bill Belichick doesn’t have to worry. Artificial intelligence can’t coach football. At least not yet.

But the New England Patriots head coach and his NFL counterparts could one day consider AI as essential to their jobs as running practices or watching game tape.

“AI will revolutionize sports,” said Alan Fern, a computer science professor at Oregon State University who is using game videos and AI to teach computers to understand and coach football.

If Fern is successful, teams would boost their brainpower with a new type of assistant coach – one powered by GPUs and a branch of AI called deep learning. It could supply strategic insights that might elude even the savviest coach.

AI Goes Pro

AI is already widely used in the sports industry. A growing number of betting services predict game winners with deep learning, a way of teaching computers to perform at near- or better-than-human level of accuracy. Microsoft offers free game predictions through its Bing Predicts, which picked winners in 65.5 percent of last year’s NFL regular season games.

Deep learning is taking on some jobs typically held by people. The Washington Post recently employed the technology to produce short updates of the Summer Olympic Games. And researchers are experimenting with automating commentary for cricket and tennis matches.

Some sports are deploying or experimenting with AI for strategy help. NBA teams, for example, use AI and computer vision techniques to track players and plays. This lets coaches collect fine-grained statistics they can use to evaluate strategy and player performance.

But creating a computer that can coach football is much more daunting, Fern said.

“Football is the most strategic sport out there. Every play is a different collection of actions for 11 people. That gives rise to lots of complex strategies,” said Fern.

An artificial intelligence football coach could help the pros evaluate and plan plays like this between the Kansas City Chiefs and the Minnesota Vikings.
An artificial intelligence football coach could help the pros evaluate and plan plays like this during this during the 2013 Pro Bowl .

Training Camp for AI

Before a computer can attempt strategy, it has to understand the game at least as well as an average fan. Fern, by showing his neural network video of Oregon State football plays and hundreds of hours of high school football, trained it to detect the snap, the difference between kicking, passing and running plays, and whether a team is on offense or defense.

It can even design better plays for a football video game, thanks to Fern’s work in automated planning,  a branch of AI that studies automatically computing action plans or strategies to achieve goals.

But computers still possess only a rudimentary understanding of different types of plays. And they still have a hard time keeping track of all 22 players on the field based on just video.

To remedy that, Fern is using NVIDIA Tesla K80 GPU Accelerators and our CUDA programming model to develop a deep learning algorithm that tracks players and understands plays in the NFL’s “all 22” videos. Shot from above the field (instead of from the sidelines like broadcast videos), these let coaches see all players at the same time so they can analyze plays and plot strategy.

Suiting Up with Deep Learning

Although Fern doesn’t expect computers to out-coach the pros anytime soon, he sees promise in deep learning and new sensor data becoming available.

Some European soccer teams are using wearable GPS systems to monitor metrics like a player’s heart rate, distance run, speed and acceleration. In 2015, the NFL equipped all players with RFID tags embedded in their shoulder pads. The coin-sized chips track each player’s location, distance traveled, acceleration and movement on the field.

If researchers can gain access to this data, Fern said, they can use deep learning to gain insights into the game that weren’t available before. For example, knowing how players move during a game could help coaches plan an athlete’s training so he reaches peak performance. It could also shed light on the best matchups between receivers and cornerbacks and measure the contribution of each player to every play.

With this kind of know-how, an AI assistant coach would become a must-have for teams, Fern said.

“Once we figure this out, there will be an AI arms race,” he said. “If teams aren’t using this, they’ll be at a major disadvantage.”