Big data technologies are nowadays being integrated in systems that are required to process a vast amount of information from geographically distributed data sources. A big challenge in this context is to fulfill non-functional properties (such as real-time guarantees, energy-efficiency, communication quality or security) inherited from the domain in which analytics are applied. Examples include smart cities or smart manufacturing domains. ELASTIC is developing a novel software architecture to help system designers to address this challenge.
ELASTIC’s software architecture is integrating the most advanced information and communications technologies (ICTs) from multiple computing domains into a single development framework. This technology will enable to design, implement and efficiently execute extreme-scale big-data analytics. To do so, ELASTIC incorporates a new elasticity concept, distributing the data analytic workflows to the most appropriate computing resources across the compute continuum, from edge to cloud, with the objective of providing the level of performance needed to process the envisioned volume and velocity of data at an affordable development cost, whilst guaranteeing the fulfilment of the non-functional properties of the system.
The main features of the ELASTIC software architecture are:
-
Elasticity: ELASTIC develops a software architecture incorporating an innovative elasticity concept, in which computation is dynamically distributed across the compute continuum whilst fulfilling real-time, energy, communication and security requirements.
-
Fog computing: Elastic aims to significantly increase the capabilities of extreme-scale analytics by efficiently distributing computation in fog environments, leveraging the use of advanced hardware architectures at the edge side and integrating both responsive data-in-motion and latent data-at-rest analytics into a single solution.
-
An API for extreme-scale analytics: ELASTIC provides an API to enable programmers to easily design their analytics applications in a transparent way over the hybrid edge/cloud infrastructure. The API will provide the necessary mechanisms for task parallelization, load balancing and data distribution methods, integrating Map Reduce and task-based programming models.
The ELASTIC software architecture is being tested on a realistic yet visionary smart mobility use-case. It enables the design of a set of advanced mobility applications based on the real-time processing of huge amounts of data coming from a large set of IoT sensors distributed along the Florence tramway network.
The ELASTIC software architecture can be applied to heterogeneous fog computing environments, where computing resources are distributed across the compute continuum, from the edge to the cloud. The proposed framework is suitable for Internet of things (IoT) and applications that require real-time capabilities and the processing of big data.
One of the domains in which extreme-scale analytics can have a significant impact on people’s day-to-day life is Smart Cities. Cities generate a massive amount of data from heterogeneous and geographically dispersed sources including citizens, public and private vehicles, infrastructures, buildings, etc. Big data analytics is increasingly seen as an effective technology capable of controlling the available (and distributed) city resources in a safe, sustainable, and efficient way to improve the economical and societal outcomes.
There is therefore a need to develop software architectures capable of deploying federated/distributed, powerful and scalable big-data systems to extract valuable knowledge. This will open the door to a wide range of advanced urban mobility services, including public transportation and traffic management (with a clear impact on CO2 reduction and quality of life of the citizens).
ELASTIC addresses an important societal challenge: the mobility of citizens within modern cities. The ELASTIC software architecture is being evaluated on a realistic smart city use-case in the public tram network of Florence in Italy. Concretely, the ELASTIC use-case will enhance the tramway public transportation services provided by the Metropolitan City of Florence (Italy) as well as its interaction with the private vehicle transportation.
The expected benefits of the employment of the ELASTIC architecture in the local tram network are the generation in real-time of valuable knoweldge for a reduced number of yearly incidents in Florence tramway, traffic improvements and reduced preventive and standard maintenance costs.
Current big-data software architectures execute most of the data analytics computation into powerful cloud services, which heavily affects the capabilities of the system to provide real-time guarantees. This approach also imposes the need to increase the level of security to minimise potential attacks while the data is being transferred to cloud, which may end up affecting the overall safety assurance levels.
The ELASTIC technology addresses these challenges by efficiently distributing the big-data computation across the compute continuum in a holistic way, taking into account real-time guarantees, energy efficiency, communication quality and security requirements. Overall, ELASTIC aims to provide the technological background for the development of new and safe mobility services.