2 Simulation and Modeling of Edge and Mist Computing: Related Work
In the previous chapter, we presented edge computing and its potential to address the identified challenges posed by the rise of IoT, as well as the various concepts of simulation and energy efficiency. As we have seen before, while idle, servers may use up to 60% of their peak consumption (Barroso & Hölzle, 2009). Luckily, cloud energy efficiency has improved quickly. However, as with any technological efficiency improvements, due to the rebound effect, the cloud’s total power demands continue to rise. This improved energy efficiency is undoubtedly a contributing factor, but it is by no means the only one. Studies have also found that network equipment power consumption may represent more than 90% of their peak consumption (Barroso & Hölzle, 2009; Vishwanath, Zhu, et al., 2013).
In theory, edge computing can help in this circumstance since it allows the local processing of large amounts of data by putting computation and storage closer to their source. As a result, it is expected to enhance network scalability while also lowering energy costs associated with data transfer to the cloud (Ai et al., 2018). It also makes use of small servers, access points, gateways, switches, or routers, that ensure enough resource availability and consume less energy (Jalali et al., 2016). In practice, though, things get more complicated. Even if distributed applications can certainly benefit from its distributed architecture, it does present several challenges, like resource constraints, the trade-off between interdependence and autonomy, and the need for adaptation that is imposed by the evolving environment. As a result, its simple adoption cannot guarantee compliance with the QoS. It is also unknown if a performance per watt comparable to that of hyperscale data centers can be attained (Google, 2019; Miller, 2019).
For these reasons, current research efforts are focusing on how to properly utilize the capabilities of the network edge to meet these applications' demands. However, as the actual deployment of edge computing have not yet begun (Yousefpour et al., 2019), and because experiments on a truly distributed system are impossible due to financial and scalability constraints (Santoro et al., 2017), the only feasible way to assess these solutions, is through simulation. Another advantage of simulation is that it does not necessitate real-time execution. An abstract simulation can reproduce the behavior of an edge computing system across several hours in a matter of minutes. Nevertheless, modeling has a number of drawbacks as well: inaccurate or skewed assumptions might result in an erroneous outcome. This is especially true in the case of edge computing, where there are no comparable real-world infrastructure deployments (Kecskemeti et al., 2017; Svorobej et al., 2019).
This chapter examines the literature and puts the contributions of this dissertation in the right context. More specifically, it aims to demonstrate that edge computing modeling requirements cannot be met by any of the current simulation tools and analytical models. It reviews the existing simulation tools and analytical models and concludes with the strengths and drawbacks of each one of them.
Due to the lack of a flexible and scalable simulation tool for edge computing and comparable real-world infrastructure deployment, researchers were forced to choose between building a testbed and developing their own models so they could evaluate their approaches. Cost and scalability considerations compelled the majority to select the second option, which involves defining all the aspects of edge computing, including resource specification, networking, energy, and mobility. To save time and effort, as seen in the previous section, they relied on oversimplified system models with unrealistic assumptions, raising doubts about the accuracy of the results. All this has motivated the development of numerous simulators.
This section describes the simulation frameworks and tools that have been developed for edge computing. First, the commonly used cloud computing simulators are described with a special focus on how they model network and power consumption. Then, the characteristics of the existing edge computing simulators are outlined, and their approaches to model the network and power consumption are investigated. Finally, analytical approaches for modeling the global power consumption of edge computing infrastructures are discussed.
Following the importance of cloud computing as an area of research in previous years, several simulators have emerged, each serving a different purpose. This subsection presents four of the most popular cloud computing simulators: GreenCloud (Kliazovich et al., 2012), iCanCloud (Núñez et al., 2012), CloudSim (Calheiros et al., 2011), and CloudSim Plus (Silva Filho et al., 2017). While these tools are not intended to simulate heterogeneous edge environments, they often serve as the foundation for edge simulators.
2.2.1.1 GreenCloud
GreenCloud (Kliazovich et al., 2012) is a simulator that is specialized in cloud networking and power consumption. It extends NS-2 , a discrete event network simulator, and is implemented using C++ and TCL scripts. Its primary objective is to provide an energy-aware analysis of cloud data centers, which includes measurements of power consumption of system components (i.e., switches, servers, and links). It also provides a complete TCP/IP implementation. However, realistically modeling power consumption and network is not costless. The experiments performed using this simulator are much slower than similar experiments in the other simulators, making large-scale scenarios unfeasible.
2.2.1.2 iCanCloud
iCanCloud (Núñez et al., 2012) is designed for simulating and modeling cloud computing systems. It is built upon OMNeT++ (Varga & Hornig, 2008), a discrete event simulation library for network simulators that is programmed in C++. Unlike GreenCloud, it has a strong focus on performance and usability, including the ability to run large-scale experiments. To model power consumption in fine detail, Castañé et al. (2013) has enriched iCanCloud with a realistic energy model. With this model, it became possible to simulate resource management policies that balance between power consumption and performance. However, despite being performance-oriented, modeling cloud computing systems in a very detailed way translates into relatively low scalability. In addition, just like GreenCloud, this project is difficult to extend, and it has ceased to be maintained.
2.2.1.3 CloudSim
CloudSim (Calheiros et al., 2011) is undoubtedly the most widespread cloud simulator. It covers all cloud computing modeling needs, like resources (i.e., data centers, hosts, and virtual machines) and CPU, storage, memory, and bandwidth allocation policies and utilization models. Its architecture consists of five layers: network, cloud resources, cloud services, VM services, and user interface structures. As opposed to the above-mentioned simulators, CloudSim is not based on another network simulator. The network model is too basic, favoring performance at the expense of accuracy. Through its cost model, users can only define static usage costs for computing, storage, and networking resources.
CloudSim is a Java-based simulator that is a publicly accessible open-source project . It is conceived to be extensible, which has led to a series of extensions, among which several have been embedded in the main project. As of version 2.0, an energy model for modeling CPUs' energy usage has been introduced by Beloglazov & Buyya (2012). Besides being simplistic, its major drawback is that it is not compatible with CloudSim’s network model (Garg & Buyya, 2011). DartCSim+ (X. Li et al., 2013), another CloudSim extension, tackles this problem by implementing an energy model that works in conjunction with the network model and can even measure the power consumed during the migration of VMs. However, it adds significant computational and conceptual complexity. In addition to being extended by several cloud computing simulators, CloudSim also constitutes the basis of many edge computing ones (Gupta et al., 2017; Sonmez et al., 2018).
2.3.1.4 CloudSim Plus
Although widely used, the initial CloudSim project had experienced several persistent issues over the years, coupled with the lack of maintenance and its rather limited extensibility that forced its users to make changes in core classes. Silva Filho et al. (2017) introduced a fork of CloudSim 3.0 called CloudSim Plus . This new simulator offers unprecedented extensibility and accuracy, improved code quality and reusability, increased scalability and performance, as well as numerous new features that make it one of the most comprehensive cloud simulators, as shown in Figure 2.1.
Figure 2.1 CloudSim Plus project structure and organization. Packages with a stronger color contain exclusive features of CloudSim Plus. The lighter modules were added simply to improve the project organization and separation of concerns (Pressman, 2009), whereas contained classes underwent substantial refactoring and re-design. Reprinted from (Silva Filho et al., 2017)
The following are brief descriptions of the most important CloudSim Plus packages and their classes:
- Distributions: This package includes classes for generating pseudo-random numbers based on a variety of statistical distributions that are needed by the simulation API. Developers who want to create new simulations of their own can make use of them as well.
- Network: This package contains classes for building datacenter network architecture and simulating networks.
- Power: This package provides classes that allow the creation of energy-aware simulations, including energy utilization models that may be extended by users to construct customized simulations.
This is the reason why it has been chosen to be the basis for the prototypical implementation, presented in Chapter 3.
As research progressed in the area of edge computing, simulation started to attract a lot of interest, driving the development of numerous simulators. This subsection presents a brief introduction to the currently available simulators, highlights their characteristics, as well as their limitations. We focus on the following properties: (i) the network topology and layers, (ii) the simulator type (i.e., specialized network simulator, event-driven, generic), (iii) maintenance (to identify obsolete solutions), (iv) cost model applied in this simulator (e.g., predefined, static, or real), (v) mobility and location management. (vi) configurable network parameters and protocols, and finally (vii) energy models.
2.2.2.1 iFogSim
iFogSim (Gupta et al., 2017) is a Java-based simulation tool that relies on CloudSim and is conceived to model fog computing environments to measure latency and throughput. It handles both edge-ward and cloud-only application placement, which demonstrates the scalability of fog-based applications. Using this simulation tool, multiple types of components (e.g., gateways, actuators, sensors, etc.) can be added and inter-connected to shape a topology. However, as shown in Figure 2.2, this topology stays static, thereby lacking support for mobility, which is a key incentive for the adoption of fog computing. In addition, the simulation of scenarios involving hundreds (or even thousands) of devices means adding and connecting them individually, representing an impractical, time-consuming, and effortful process. Finally, every connection is given a constant delay, disregarding the impact of network load on transmission delays.
Figure 2.2 iFogSim topology and graphical user interface
2.2.2.2 IoTSim
IoTSim (Zeng et al., 2017) is another CloudSim-based simulation tool. It allows simulating batch-oriented IoT applications where data is fed in bulk to a processing system, using a MapReduce large data processing model. However, it does not support mobility nor the ability to model energy consumption.
2.2.2.3 SimIoT
SimIoT (Sotiriadis et al., 2014) is a simulator that focuses on the communication between IoT devices and the remote cloud. It enables experimenting with dynamic multi-user submission in IoT environments. Nevertheless, the power consumption and the heterogeneity of end devices are neglected. IoT will count more than 38.6 billion devices by 2025 (Strategy Analytics, 2019), all of which produce data constantly, raising power consumption as a primary concern.
2.2.2.4 FogNetSim++
FogNetSim++ (Qayyum et al., 2018) is a fog computing simulator that is built upon OMNeT++ discrete event simulator. It leverages many of its legacy features and offers the ability to simulate unreliable network links and packet failures. It allows to simulate complex scenarios and incorporate scheduling algorithms and mobility models. It supports various IoT-specific protocols, including Message Queue Telemetry Transport (MQTT), Constrained Application Protocol (CoAP), and Advanced Message Queuing Protocol (AMQP). Another key element of FogNetSim++ is the presence of an energy model for applications. So far, it is the only edge simulator to include such a feature. While it may seem promising at first glance, there are still many drawbacks. As with GreenCloud and iCanCloud, the detailed simulation of network traffic adversely affects performance and scalability. It should also be noted that this simulator does not include an energy model for the WAN, restricting the relevance of comparing edge to cloud architectures. Finally, the lack of documentation and updates since the first release has made it difficult for the community to give it proper attention.
2.2.2.5 EdgeCloudSim
EdgeCloudSim (Sonmez et al., 2018) is another extension of CloudSim that is targeted at multi-access edge computing scenarios. It supports multi-tier scenarios in which several edge servers operate in coordination with the distant cloud (see Figure 2.3). It considers some limitations of iFogSim and IoTSim, such as supporting the dynamic generation of mobile devices and providing a relatively realistic network modeling. Although it was designed specifically for multi-access edge computing, this simulator still lacks some of its main modeling pillars: it does not incorporate any energy model, its mobility model is very simplistic, and implementing a more realistic model involves modifying the code base, which is not practical and limits its potential.
Figure 2.3 The type of architectures simulated by EdgeCloudSim. Reprinted from (Sonmez et al., 2018)
Analytical models, much like simulators, have been used to examine the energy consumption of edge computing infrastructures. Typically, such models provide a complete view of the problem space and an in-depth analysis of the input variables' impact (see subsection 1.3.6). In addition, their rapid execution makes them well suited for large-scale simulations. However, they present a high degree of abstraction, often unrealistic. None of the methods described below can be applied to heterogeneous infrastructures because of their idealistic assumptions, for example, by assuming edge nodes to have identical computing capacities. Furthermore, because they fail to represent evolution over time, these analytical approaches cannot support mobility nor online decision making, which limits the studies on sustainable energy-saving strategies.
2.2.3.1 Sarkar and Misra Analytical Model
Sarkar and Misra (2016) have presented an analytical model for edge computing environments to evaluate service time and power consumption against the conventional cloud. While this model describes the power consumption of each component using mathematical equations, it does not differentiate between static and dynamic power consumptions, which further restricts the informative value of the analyses. In addition, no application concept is provided. As a result, only a high-level overview of the different infrastructures is possible. Based on their model, the researchers claim that edge computing architectures can improve the mean energy efficiency by 40% in comparison to conventional cloud infrastructures.
2.2.3.2 Jalali et al. Analytical Model
A further detailed approach was adopted by Jalali et al. (2016) to model the energy consumption of edge computing systems. It is composed of two parts: The first one is a flow-based model for highly shared devices like switches and routers, which calculates power consumption on the basis of the data flow. The second one is time-based for end-user devices that are typically not shared among numerous users. It determines the consumption according to the time spent while offering access to a service. The application model, on the other hand, focuses on offloading computations to edge nodes to deliver content. It is validated by measuring the actual data of a WordPress deployment that can offload content delivery to edge nodes. Nevertheless, the model cannot simulate different applications simultaneously. The results show that edge computing can extend the cloud and has the potential to reduce power consumption, but this achievement is still subject to design factors, including the type of applications and network architecture. The type of access network is notably another crucial factor: offloading computations over 4G LTE is significantly cost-intensive compared to, for instance, passive optical networks. Interestingly, the number of hops in the WAN did not impact the overall power consumption significantly.
2.2.3.3 Ahvar et al. Analytical Model
A model to evaluate the total power consumption of several edge and cloud-related architectures has been proposed by Ahvar et al. (2019). This model includes both static and dynamic energy consumption and considers the consumption of cooling and network equipment. A comparison of four architectures, ranging from entirely distributed to entirely centralized, was made. The completely centralized one was found to consume up to 33% more power than the completely distributed one due to intra-data center traffic and large-size cooling systems. Despite being the broadest analytical model discussed in this subsection, it still contains a number of important oversimplifications, such as equal distribution of VMs across data centers or the assumption that all edge nodes connect to the same number of devices. Also, no true concept of applications with interdependent tasks exists nor means to determine the relative energy requirements of VMs.
As we mentioned earlier in Section 1.3.4, a proper specification of goals is crucial to achieve the targeted model and a software product in general. It is crucial to decide which aspects and functionalities are necessary and under which limiting conditions. By examining the current use cases and models, we have identified a set of functional requirements that need to be satisfied by an edge computing model. However, non-functional ones, such as qualitative necessities, should not be neglected as well (Kleuker, 2011). Figure 2.4 highlights the modeling and simulation aspects of edge computing. Current simulation tools, as pointed out above, are not capable of properly modeling edge computing systems. Resorting to a simulator that realistically models such environments and helps to identify the causes of energy consumption and investigate power-efficient algorithms and architectures is essential to unleash the full energy-saving potential of this emerging computing model.
Figure 2.4 The aspects of modeling edge and mist computing environments
Based on the current usage scenarios and in view of the difficulties encountered in modeling edge computing environments (Kecskemeti et al., 2017; Noor et al., 2018; Svorobej et al., 2019), many functional requirements have been derived for the proposed simulator. These requirements are listed in Appendix A and divided into three categories in accordance with RFC 2119 . The MUST category contains features that are strictly mandatory for the simulator; the SHOULD category contains the features that are necessary to achieve adequate software; and the MAY category, which consists of features that are desirable but not obligatory. Besides, the WILL NOT category has been added, which enumerates features that will not be considered yet and should be part of future work. The primary requirements are:
-
Realistic Edge Computing Modeling: The simulator should allow modeling heterogeneous, geo-distributed, and resource-limited edge computing environments. It must ensure mobility of devices and location-awareness, where their positions change based on predefined models or in a totally random way, allowing them to join or quit the network at any moment. It also has to support modeling different types of applications, from linear data pipelines to more sophisticated ones that involve numerous sources of data, processing stages, and data sinks.
-
QoS-Aware Online Decision Making: Data will be generated, captured, and analyzed in quantities and varieties in order of magnitude greater than ever before. Effective implementation of the infrastructure will therefore require a number of important decisions: mainly how data is gathered and where it will be processed (e.g., the cloud or the edge). These decisions are influenced by two competing pressures: the use of the infrastructure and QoS required by the end-user (Svorobej et al., 2019). The simulator must accordingly be capable of seamlessly incorporating live data to be employed during the process of decision-making.
-
Performance Evaluation: To analyze performance, it becomes important to understand the bottlenecks and metrics at each level. The difficulty here is that each level requires a different approach to assess performance. For instance, edge level evaluation involves assessing network stability, bandwidth, fairness in resource allocation, delays, and the battery state of end devices. Such measures can vary significantly from those of cloud operators, who may be more concerned with platform scalability, response time, VMs utilization, and data migration cost (Kecskemeti et al., 2017). Thus, it is essential that the simulator provides a wide collection of metrics.
In addition to these functional requirements, the non-functional necessities, such as quality, should also be considered. ISO/IEC 250001 (formerly ISO/IEC 9126) provides guidelines for quality requirements. Typical quality attributes are correctness, reliability, usability, and efficiency (Kleuker, 2011). These attributes are described below:
-
Usability, Customization, and Extensibility: To guarantee that edge computing simulators fulfill all expectations, they are supposed to be easy to learn and manipulate. They must also allow users to implement their customized resource management policies and to add new kinds of devices, applications, and models without requiring to know or modify their internals (Kecskemeti et al., 2017). Thus, their applicability is broadened, allowing the customization of their components’ behavior, and opening the doors to a multitude of use cases and scenarios. However, being able to model unlimited combinations of workloads, topologies, and resources can be offset by the additional complexity of parameterization.
-
Efficiency and Scalability: The simulator must support various scenarios, including complex ones that involve a large number of devices. The simulation duration must be consistent with the size and complexity of the simulated environment and with the number of runs, while also being shorter than wall-clock-time.
-
Correctness and Reliability: The simulator must deliver correct, accurate, and reliable results, and also, be robust to user error and incorrect entries.
Although numerous state-of-the-art simulators (Gupta et al., 2017; Sonmez et al., 2018; Zeng et al., 2017) are built upon cloud computing ones like CloudSim (Calheiros et al., 2011) to efficiently simulate computing resources (e.g., CPU, storage, memory), a lot of them have failed to deliver realistic modeling of edge computing by omitting critical features such as the mobility of end devices (Gupta et al., 2017; Zeng et al., 2017). Similarly, employing constant transmission delays or bandwidth as in some simulation tools is impractical (Calheiros et al., 2011; Gupta et al., 2017; Sonmez et al., 2018), particularly when analyzing the platform scalability, considering that these delays are in direct proportion to the network load. Taking power consumption as another example, the fact that the few simulators that supply energy models are either excessively simplistic (Gupta et al., 2017) or limited to the consumption of certain entities like data centers or constrained edge devices (Flores et al., 2018; Hirsch et al., 2020), is maybe the most obvious limitation. To achieve a global view, it becomes necessary to analyze, besides computational resources, the power consumed by the network and applications.
While network simulators allow modeling the network efficiently, they require users to specify every remaining aspect of the simulation (refer to Figure 2.4). This includes building a workload generator, a task orchestrator, and mobility and resource utilization models to assess their solutions, which involves a considerable amount of time and work. On the other hand, tools that allow modeling computing and network environments, like FogNetSim++, do so in great detail by modeling each packet individually, and in certain cases, even implementing full communication protocols. Because of this high degree of detail, they are rarely capable of handling scenarios with huge numbers of devices given performance limitations. Analytical models, in contrast, are well-suited for large-scale experiments because of their great level of abstraction. However, they are not capable of realistically modeling edge computing systems, owing to their lack of support for online decision-making. Furthermore, modeling heterogeneous infrastructures is far from being simple. For these reasons, all the analytical models presented in this section are oversimplified and rely on a number of unrealistic assumptions. An overview of the strengths and limitations of the models that have been discussed in this section is provided in Table 2.3. The results are divided into five groups representing the modeling requirements of edge computing. The goal is to highlight if the model supports realistic modeling of edge computing environments, to show its support for online decision-making, its extensibility, and its scalability. ✓ indicates that it reasonably satisfies that criterion, while ❏ reports that it only does so in part.
1 ISO/IEC 25000 Software Quality Requirements and Evaluation standard family.