What Is Edge Computing?

Edge computing is the concept of capturing and processing data as close to the source of the data as possible via processors equipped with AI software.
by Scott Martin

Edge computing and donuts have one thing in common: the closer they are to the consumer, the better. A trip to the corner donut shop may take a bit, but a box of donuts within reach is instant gratification.

The same holds true for edge computing. Send data to an AI application running in the cloud, and it delays answers. Process that data on an edge device, and it’s like grabbing directly from that pink box of glazed raised and rainbow sprinkles.

With edge computing, as with donuts, proximity matters.

Edge computing — a decades-old term — is the concept of capturing and processing data as close to the source of the data as possible via processors equipped with AI software. Because edge computing processes data locally — on the “edge” of a network, instead of in the cloud or a data center — it minimizes latency and bandwidth needs, allowing for real-time feedback and decision-making by autonomous machines.

Frequently, the processors are in the form of intelligent sensors embedded in Internet of Things devices. These sensors could be on heavy machinery in a factory, processing data from the machines and alerting supervisors when malfunctions could result in an accident.

Businesses often place edge servers in close proximity to the sensors, usually in a server room or closet within a store, hospital or warehouse.

The always-on, instantaneous feedback that edge computing offers is especially critical for applications such as autonomous vehicles, where saving even milliseconds of data processing and response times can be key to avoiding accidents. Instantaneous feedback at the edge is also important in hospitals, where doctors rely on accurate, real-time data to treat their patients.

Edge computing is everywhere — used in everything from retail stores for smart self-checkout, to warehouses where it assists with supply-chain logistics and quality inspections.

Why Is Edge Computing Needed?

By 2025, it’s estimated that 150 billion machine sensors and IoT devices will stream continuous data that will need to be processed. These sensors are on all the time — monitoring, picking up data, reasoning about what they are sensing and taking action.

Edge computing processes this data at the source, reducing latency, or the need to wait for data to be sent from a network to the cloud or core data center for further processing, allowing businesses to gain real-time or faster insights.

The surge of data used in these compute-intensive workloads that require high efficiency and speed in data collection and analysis is demanding high-performance edge computing to deploy AI.

Moreover, emerging technologies such as the unveiling of 5G networks, which are expected to clock in 10x faster than 4G, only increases the possibilities for AI-enabled services, requiring further acceleration of edge computing.

How Does Edge Computing Work?

Edge computing works by processing data as close to the source or end user as possible. It keeps data, applications and computing power away from a centralized network or data center.

Data centers are centralized servers often situated where real estate and power is less expensive. Even on the zippiest fiber optic networks, data can’t travel faster than the speed of light. This physical distance between data and data centers causes latency. By bringing computing to the edge or closer to the source of data, edge computing reduces the issue surrounding latency.

Edge computing can be run at multiple network nodes to literally close the distance between where data is collected and processed to reduce bottlenecks and accelerate applications.

At the periphery of networks, billions of IoT and mobile devices operate on small, embedded processors, which are ideal for basic applications like video.

That would be just fine if industries and municipalities across the world today weren’t applying AI to data from IoT devices. But they are.

By using edge AI, a device would not need to be connected to the internet at all times. Instead, a device could process data and make decisions independently without a connection.

For example, an edge AI application on a microprocessor in a robot could process data from the robot in real time and store results locally on the device. After some time, the robot could connect to the internet and send specific data to the cloud for storage or further processing. If the robot wasn’t operating on the edge, it would continuously stream data to the cloud (taxing its batteries), take longer to process data and require a constant internet connection.

What Are the Benefits of Edge Computing?

The shift to edge computing offers businesses new opportunities to glean insights from their large datasets. The four main benefits of edge computing are:

  • Reduced latency: Bringing AI computing to where data is generated, rather than collecting and uploading data to a centralized data center or cloud, reduces latency.
  • Improved security: As edge computing allows for data to be processed locally, the need to send sensitive data to the public cloud is decreased.
  • Lowered expenses: Creating more and more data increases bandwidth and data storage costs. Using edge computing and local data processing means less data needs to be sent to the cloud.
  • Greater range: Internet access is required for traditional cloud computing. But edge computing processes data without internet access, extending its range to previously inaccessible remote locations.

Edge Computing vs. Cloud Computing vs. Fog Computing

These methods of computing are often used and mentioned together for strengthened computing power. However, they’re distinctly different:

  • Cloud computing uses a network of remote servers hosted on the internet.
  • Edge computing uses the edge of a device or server.
  • Fog computing uses the local area network (LAN) of a network architecture.

In recent years, cloud computing has been the preferred processing method due to its capacity, elasticity and ability to store and process data without physical hardware.

But cloud computing is limited by the speed of light and internet bandwidth. As more businesses deploy AI within their offerings, the demand for faster and more reliable data increases, putting a strain on cloud computing’s networking bandwidth.

To lighten this strain, edge computing has been incorporated into many IoT devices for faster data processing and response times.

Similar to edge computing is fog computing. The difference is that while edge computing processes data at the network edge, fog computing processes data on a device’s local area network. Its strength lies in its ability to process more data than edge computing but it’s limited to its physical connection to devices in the LAN.

Edge Computing: IoT and 5G

Edge computing plays a critical role in the recent advancements in technologies such as the 5G network and IoT applications.

Edge & IoT 

With the flood of data coming from IoT devices, manufacturers have realized both the financial and operational benefits of processing data at the edge. With edge computing, IoT devices and sensors can operate with reduced latency and less dependence on the cloud for costly data storage and processing.

For example, with the NVIDIA Metropolis platform for intelligent video analytics, data from trillions of sensors and IoT devices can be analyzed in real time. This can provide actionable insights for applications such as public services for anomaly detection and disaster response, logistics for supply forecasting, and traffic management for incident detection and traffic light optimization.

For an effective disaster response, acting in a timely manner is crucial. By implementing an edge platform like NVIDIA Metropolis, instantaneous and constant data on the location of personnel, vehicles and equipment needed for first responder efforts is available to help ensure the safety of citizens. Moreover, by implementing edge computing that gathers data from IoT sensors and devices and not through cellular networks or internet connection, a more reliable and efficient disaster response plan is possible with the potential to save lives.

Edge & 5G 

The amount of data being generated at the edge is growing exponentially and with the rollout of 5G infrastructure, new breeds of applications are emerging.

While AI is enabling insights from mass data, these applications will rely on 5G’s fast bandwidth, low latency and reliability to provide access to that data.

With the rollout of 5G, a wide portfolio of services is emerging to run AI workloads at the edge and make real-time analysis possible. These range from remotely controlling equipment and machines with cameras and other sensors to using cameras to improve site security and operational safety — all while supporting billions of media-rich devices that will collectively consume and produce zettabytes of data.

Edge computing is critical for such technological innovations and is the only way to meet the latency requirements needed for 5G to operate. It also helps to securely and efficiently virtualize multi-tenant 5G edge nodes, such as 5G.

4 Edge Computing Examples

Not only does edge computing reduce latency, but it also provides end users with better, more seamless experiences. Here are a few examples of edge applications across multiple industries.

Edge Computing for Retailers 

The world’s largest retailers are enlisting edge AI to become smart retailers. Intelligent video analytics, AI-powered inventory management, and customer and store analytics together offer improved margins and the opportunity to deliver better customer experiences.

For example, using the NVIDIA EGX platform, Walmart is able to compute in real time more than 1.6 terabytes of data generated a second. It can use AI for a wide variety of tasks, such as automatically alerting associates to restock shelves, retrieve shopping carts or open up new checkout lanes.

Connected cameras numbering in the hundreds or more can feed AI image recognition models processed on site by NVIDIA EGX. Meanwhile, smaller networks of video feeds in remote locations can be handled by Jetson Nano, linking with EGX and NVIDIA AI in the cloud.

Store aisles can be monitored by fully autonomous and capable conversational AI robots powered by Jetson AGX Xavier and running NVIDIA Isaac for SLAM navigation. All of this is compatible with EGX or NVIDIA AI in the cloud.

Whatever the application, NVIDIA T4 and Jetson GPUs at the edge provide a powerful combination for intelligent video analytics and machine learning applications.

With edge AI, telecommunications companies can develop next-generation services to offer their customers, providing new revenue streams.

Using NVIDIA EGX, telecom providers can analyze video camera feeds using image recognition models to help with everything from foot traffic to monitoring store shelves and deliveries.

For example, if a 7-Eleven ran out of donuts early in the morning on a Saturday in its store display, the convenience store manager could receive an alert that it needs restocking.

Edge Computing for Cities

Fortune 500 companies and startups alike are adopting AI at the edge for municipalities. For example, cities are developing AI applications to relieve traffic jams and increase safety.

Verizon uses NVIDIA Metropolis, the IoT application framework that, combined with Jetson’s deep learning capabilities, can analyze multiple streams of video data to look for ways to improve traffic flow, enhance pedestrian safety, optimize parking in urban areas, and more.

Ontario, Canada-based startup Miovision Technologies uses deep neural networks to analyze data from its own cameras and from city infrastructure to optimize traffic lights and keep vehicles moving.

Miovision and others’ work in this space can be accelerated by edge computing from the NVIDIA Jetson compact supercomputing module and insights from NVIDIA Metropolis. The energy-efficient Jetson can handle multiple video feeds simultaneously for AI processes. The combination delivers an alternative to network bottlenecks and traffic jams.

Edge computing scales up, too. Industry application frameworks like NVIDIA Metropolis and AI applications from third parties run atop of the NVIDIA EGX platform for optimal performance.

Edge Computing for Automakers and Manufacturers

Factories, retailers, manufacturers and automakers are generating sensor data that can be used in a cross-referenced fashion to improve services.

This sensor fusion will enable retailers to deliver new services. Robots can use more than just voice and natural language processing models for conversational interactions. Those same bots can use video feeds to run on pose estimation models. Linking the voice and gesture sensor information can help robots better understand what products or directions customers are seeking.

Sensor fusion could create new user experiences for automakers to adopt for competitive advantages as well. Automakers could use pose estimation models to understand where a driver is looking along with natural language models that understand a request that correlates to restaurant locations on a car’s GPS map.

Edge Computing for Gaming

Gamers are notorious for demanding high-performance, low-latency computing power. High-quality cloud gaming at the edge ups the ante. Next-generation gaming applications involving virtual reality, augmented reality and AI are an even bigger challenge.

Telecommunications providers are using NVIDIA RTX Servers — which deliver cinematic-quality graphics enhanced by ray tracing and AI — to gamers around the world. These servers power GeForce NOW, NVIDIA’s cloud gaming service, which transforms underpowered or incompatible hardware into powerful GeForce gaming PCs at the edge.

Taiwan Mobile, Korea’s LG U+, Japan’s SoftBank and Russia’s Rostelecom have all announced plans to roll out the service to their cloud gaming customers.

NVIDIA’s Edge Solutions

At NVIDIA, we have taken edge computing to the next level with the EGX platform, which makes it easier for enterprises to implement edge solutions that combine scalable software, IoT sensors and edge servers.

In place of traditional edge servers running racks of CPUs, the lighter footprint of NVIDIA EGX offers compatibility across NVIDIA AI, from the Jetson line of supercomputing modules, to full racks of servers equipped with NVIDIA T4 GPUs and the recently announced NVIDIA EGX A100.

Businesses running edge computing for AI are getting the flexibility of deploying low-latency AI applications on the tiny NVIDIA Jetson Nano. The compact supercomputer takes just a few watts to deliver a half-trillion operations per second for such tasks as image recognition.

A rack of NVIDIA T4 servers delivers more than 10,000 trillion operations per second, fit for even the most demanding real-time speech recognition algorithms and other compute-heavy AI tasks.

With NVIDIA EGX, edge applications can be remotely deployed and managed and, using NVIDIA’s NGC software hub, AI and IT teams can get started quickly with a wide variety of pre-trained AI models and Kubernetes-ready Helm charts to implement into their edge AI solutions.

Also, updates at the periphery of the AI-driven edge network are simple. The EGX software stack runs on Linux and Kubernetes, allowing remote updates from the cloud or edge servers to continuously improve applications.

NVIDIA EGX servers are tuned for CUDA-accelerated containers available on NGC.

The Future of Edge Computing

According to market research firm IDC, the edge computing market will be worth $34 billion by 2023. The emergence of 5G will enable the transition from computing at centralized data centers to computing at the edge, unlocking potential for opportunities that were not previously available.

From video analytics to autonomous vehicles to gaming, edge computing is creating more possibilities to deliver immersive, real-time experiences that have low-latency and connectivity requirements.

Learn more about using edge computing for your business