Editor’s note: This blog was updated on Nov. 15, 2021.
Smart cities. Remote surgeries. Fully autonomous vehicles. Voice-controlled home speakers. All of these innovative technologies are made possible thanks to edge computing. So …
What Is Edge Computing?
Edge computing is the practice of moving compute power physically closer to where data is generated, usually an IoT device or sensor. It is named for the way compute power is brought to the “edge” of a device or network. Edge computing is used to process data faster, increase bandwidth and ensure data sovereignty.
By processing data at a network’s edge, edge computing reduces the need for large amounts of data to travel between servers, the cloud, and devices or edge locations. This solves the infrastructure issues found in conventional data processing, such as latency and bandwidth. This is particularly important for modern applications such as data science and AI.
For example, advanced industrial equipment increasingly features intelligent sensors powered by AI-capable processors that can do inferencing at the edge, also known as edge AI. These sensors monitor equipment and nearby machinery to alert supervisors of any anomalies that potentially jeopardize safe, continuous, and effective operations. In this use case, having AI processors physically present at the industrial site results in lower latency and the industrial equipment reacting more quickly to their environment.
The always-on, instantaneous feedback that edge computing offers is especially critical for applications where human safety is a factor, such as self-driving cars where saving even milliseconds of data processing and response times can be key to avoiding accidents. Or in hospitals, where doctors rely on accurate, real-time data to treat their patients.
Edge computing can be used everywhere sensors collect data — from retail stores for self-checkout and hospitals for remote surgeries, to warehouses with intelligent supply-chain logistics and factories with quality control inspections.
How Does Edge Computing Work?
Edge computing works by processing data as close to its source or end user as possible. It keeps data, applications and computing power away from a centralized network or data center.
Traditionally, data produced by sensors is often either manually reviewed by humans, left unprocessed or sent to the cloud or a data center for processing, and then sent back to the device. Relying solely on manual reviews results in slower, less efficient processes. Cloud computing provides computing resources, however, data travel and processing puts a large strain on bandwidth and latency.
Bandwidth is the rate at which data is transferred over the internet. When data is sent to the cloud, it travels through a wide area network, which can be costly due to its global coverage and high bandwidth needs. When processing data at the edge, local area networks can be utilized, resulting in higher bandwidth at lower costs.
Latency is the delay in sending information from one point to the next. It’s reduced when processing at the edge because data produced by sensors and IoT devices no longer needs to send data to a centralized cloud to be processed. Even on the zippiest fiber-optic networks, data can’t travel faster than the speed of light.
By bringing computing to the edge, or closer to the source of data, latency is reduced and bandwidth is increased, resulting in faster insights and actions.
Edge computing can be run on one or multiple systems to close the distance between where data is collected and processed to reduce bottlenecks and accelerate applications. An ideal edge infrastructure also involves a centralized software platform that can remotely manage all edge systems in one interface.
Why Is Edge Computing Needed?
Today, three technology trends are converging and creating use cases that are requiring organizations to consider edge computing: IoT, AI and 5G.
IoT: With the proliferation of IoT devices came the explosion of big data that businesses started to generate. As organizations suddenly took advantage of collecting data from every aspect of their businesses, they realized that their applications weren’t built to handle such large volumes of data.
Additionally, they came to realize that the infrastructure for transferring, storing and processing large volumes of data can be extremely expensive and difficult to manage. That may be why only a fraction of data collected from IoT devices is ever processed, in some situations as low as 25 percent.
And the problem is compounding. There are 40 billion IoT devices today and predictions from Arm show that there could be 1 trillion IoT devices by 2022. As the number of IoT devices grows and the amount of data that needs to be transferred, stored and processed increases, organizations are shifting to edge computing to alleviate the costs required to use the same data in cloud computing models.
AI: Similar to IoT, AI represents endless possibilities and benefits for businesses, such as the ability to glean real-time insights. Just as quickly as organizations are finding new use cases for AI, they’re discovering that those new use cases have requirements that their current cloud infrastructure can’t fulfill.
When organizations have bandwidth and latency infrastructure constraints, they have to cut corners on the amount of data they feed their models. This results in weaker models.
5G: 5G networks, which are expected to clock in 10x faster than 4G ones, are built to allow each node to serve hundreds of devices, increasing the possibilities for AI-enabled services at edge locations.
WIth edge computing’s powerful, quick and reliable processing power, businesses have the potential to explore new business opportunities, gain real-time insights, increase operational efficiency and to improve their user experience.
What Are the Benefits of Edge Computing?
The shift to edge computing offers businesses new opportunities to glean insights from their large datasets. The main benefits of edge computing are:
- Lower Latency: By processing data at a network’s edge, data travel is reduced or eliminated, accelerating AI. This opens the door to advanced use cases with more complex AI models such as fully autonomous vehicles and augmented reality, which require low latency.
- Reduced Cost: Using a LAN for data processing means organizations can access higher bandwidth and storage at lower costs compared to cloud computing. Additionally, because processing happens at the edge, less data needs to be sent to the cloud or data center for further processing. There is also a decrease in the volume of data that needs to travel, which reduces costs further.
- Model Accuracy: AI relies on high-accuracy models, particularly for edge use cases that require instantaneous responses. When a network’s bandwidth is too low, it’s typically mitigated by reducing the size of data used for inferencing. This results in reduced image sizes, skipped frames in video and reduced sample rates in audio. When deployed at the edge, data feedback loops can be used to improve AI model accuracy and multiple models can be run simultaneously resulting in improved insights.
- Wider Reach: Internet access is required for traditional cloud computing. But edge computing processes data without internet access, extending its range to remote or previously inaccessible locations.
- Data Sovereignty: When data is processed at the location it is collected, edge computing allows organizations to keep all of their data and compute inside the LAN and company firewall. This results in reduced exposure to cybersecurity attacks in the cloud, and strict and ever-changing data laws.
Edge Computing Use Cases Across Industries
Edge computing can bring real-time intelligence to businesses across industries, including retail, healthcare, manufacturing, hospitals and more.
Edge Computing for Retail
In the face of rapidly changing consumer demand, behavior and expectations, the world’s largest retailers enlist edge AI to deliver better experiences for customers.
With edge computing, retailers can boost their agility by:
- Reducing shrinkage: With in-store cameras and sensors leveraging AI at the edge to analyze data, stores can identify and prevent instances of errors, waste, damage and theft.
- Improving inventory management: Edge computing applications can use in-store cameras to alert store associates when shelf inventories are low, reducing the impact of stockouts.
- Streamlining shopping experiences: With edge computing’s fast data processing, retailers can implement voice ordering so shoppers can easily search for items, ask for product information and place online orders using smart speakers or other intelligent mobile devices.
Learn more about the three pillars of edge AI in retail.
Edge Computing for Smart Cities
Cities, school campuses, stadiums and shopping malls are a few examples of many places that have started to use AI at the edge to transform into smart spaces. These entities are using AI to make their spaces more operationally efficient, safe and accessible.
Edge computing has been used to transform operations and improve safety around the world in areas such as:
- Reducing traffic congestion: Nota uses computer vision to identify, analyze and optimize traffic. Cities use its offering to improve traffic flow, decrease traffic congestion-related costs and minimize the time drivers spend in traffic.
- Monitoring beach safety: Sightbit’s image-detection application helps spot dangers at beaches, such as rip currents and hazardous ocean conditions, allowing authorities to enact life-saving procedures.
- Increasing airline and airport operation efficiency: ASSAIA created an AI-enabled video analytics application to help airlines and airports make better and quicker decisions around capacity, sustainability and safety.
Download this ebook for more information on how to build smarter, safer spaces with AI.
Edge Computing for Automakers and Manufacturers
Factories, manufacturers and automakers are generating sensor data that can be used in a cross-referenced fashion to improve services.
Some popular use cases for promoting efficiency and productivity in manufacturing include:
- Predictive maintenance: Detecting anomalies early and predicting when machines will fail to avoid downtime.
- Quality control: Detecting defects in products and alerting staff instantly to reduce waste and improve manufacturing efficiency.
- Worker safety: Using a network of cameras and sensors equipped with AI-enabled video analytics to allow manufacturers to identify workers in unsafe conditions and to quickly intervene to prevent accidents.
Edge Computing for Healthcare
The combination of edge computing and AI is reshaping healthcare. Edge AI provides healthcare workers the tools they need to improve operational efficiency, ensure safety, and provide the highest-quality care experience possible.
Two popular examples of AI-powered edge computing within the healthcare sector are:
- Operating rooms: AI models built on streaming images and sensors in medical devices are helping with image acquisition and reconstruction, workflow optimizations for diagnosis and therapy planning, measurements of organs and tumors, surgical therapy guidance, and real-time visualization and monitoring during surgeries.
- Hospitals: Smart hospitals are using technologies such as patient monitoring, patient screening, conversational AI, heart rate estimation, radiology scanners and more. Human pose estimation is a popular computer vision task that estimates key points on a person’s body such as eyes, arms and legs. It can be used to help notify staff when a patient moves or falls out of a hospital bed.
View this video to see how hospitals use edge AI to improve care for patients.
NVIDIA at the Edge
The ability to glean faster insights can mean saving time, costs and even lives. That’s why enterprises are tapping into the data generated from the billions of IoT sensors found in retail stores, on city streets and in hospitals to create smart spaces.
But to do this, organizations need edge computing systems that deliver powerful, distributed compute, secure and simple remote management, and compatibility with industry-leading technologies.
NVIDIA brings together NVIDIA-Certified Systems, embedded platforms, AI software and management services that allow enterprises to quickly harness the power of AI at the edge.
Mainstream Servers for Edge AI: NVIDIA GPUs and BlueField data processing units (DPUs) provide a host of software-defined hardware engines for accelerated networking and security. These hardware engines allow for best-in-class performance, with all necessary levels of enterprise data privacy, integrity and reliability built in. NVIDIA-Certified Systems ensure that a server is optimally designed for running modern applications in an enterprise.
Management Solutions for Edge AI: NVIDIA Fleet Command is a cloud service that securely deploys, manages and scales AI applications across distributed edge infrastructure. Purpose-built for AI lifecycle management, Fleet Command offers streamlined deployments, layered security and detailed monitoring capabilities.
For organizations looking to build their own management solution, there is the NVIDIA GPU Operator. It uses the Kubernetes operator framework to automate the management of all NVIDIA software components needed to provision GPUs. These components include NVIDIA drivers to enable CUDA, a Kubernetes device plugin for GPUs, the NVIDIA container runtime, automatic node labeling and an NVIDIA Data Center GPU Manager-based monitoring agent.
Applications for Edge AI: To complement these offerings, NVIDIA has also worked with partners to create a whole ecosystem of software development kits, applications and industry frameworks in all areas of accelerated computing. This software is available to be remotely deployed and managed using the NVIDIA NGC software hub. AI and IT teams can get easy access to a wide variety of pretrained AI models and Kubernetes-ready Helm charts to implement into their edge AI systems.
The Future of Edge Computing
According to market research firm IDC’s “Future of Operations-Edge and IoT webinar,” the edge computing market will be worth $251 billion by 2025, and is expected to continue growing each year with a compounded annual growth rate of 16.4 percent.
The evolution of AI, IoT and 5G will continue to catalyze the adoption of edge computing. The number of use cases and the types of workloads deployed at the edge will grow. Today, the most prevalent edge use cases revolve around computer vision. However, there are lots of untapped opportunities in workload areas such as natural language processing, recommender systems and robotics.
The possibilities at the edge are truly limitless.