Real-Time Remote Excavation

Telecommuting may seem like a white-collar job, requiring an office full of webcams, speakerphones and high-resolution displays just to remotely attend a staff meeting. But as smart connected cameras and other sensors permeate industrial facilities and equipment, it’s becoming more and more plausible that we’ll one day think it’s normal to operate heavy machinery remotely.

Last year a team of researchers at Ericsson decided to find out just what it would take to allow someone to safely and efficiently control an excavator -- the kind of earth-moving machine you might see at a mine or construction site -- without ever leaving home. They demonstrated a proof-of-concept model at last year’s Mobile World Congress (which is happening again this week in Barcelona).

As you might expect, an industrial worker’s telecommute is much more high-stakes than that of your typical office drone. An excavator can be dangerous enough when a trained operator has their hands directly on the controls -- for someone to run it remotely requires, in the words of Ericsson researcher Robert Swain, “an experience of controlling the machine that is as good as, or better than actually sitting on-site in the driver’s seat.”

To create that better-than-the-real-thing experience, the operator needs to be fully aware of situation around the worksite. That means real-time streams of video and audio, as well as machine performance data and other environmental sensor feedback.

In the Ericsson Research prototype, the model excavator (a small robot arm in a sandbox landscape) is outfitted with a smartphone with a 360-degree camera lens. Audio and video from the phone are streamed to an Oculus Rift headset, which maps the full-circle images onto a virtual environment. A standard video game controller sends commands to the digger.

Real-TIme Remote Excavation

Though it’s only a rudimentary set-up, the model demonstrates that it is possible to control the machinery pretty smoothly even without perfect feedback. “It really is a surprise, even to researchers and engineers working on the project, just how quickly you become immersed in the experience,” Swain wrote at the Ericsson Research blog. Before it could be deployed in the field, however, the system would need improvements to gather more and better-quality data, such as a second camera to present the worksite in stereoscopic 3-D and allow for depth perception.

The other challenges with remotely-operated machinery are latency -- the time between moving the controls, the machine’s response, and the results being sent back to the operator -- and the reliability of the connection. Latency must be extremely low (less than 50 milliseconds), and reliability high, so the operator can detect and respond to obstacles, malfunctions, and other hazards. That means future telecommuters will depend on next-generation networks, such as the 5G mobile standard, to prove that the infrastructure is up to the task.

But with the right sensors, the right software, and the right network, it might soon be possible to work in heavy industry while sitting on your couch in your pajamas (hard hat optional).

See for yourself in the video below.

Related: Immersive Data Analytics


Featured in Channel:

Smart Mining Guide

A spotlight on the products and companies