The AEGIS platform provides a multi-tenant data management and processing services for big data. The multi-tenancy behaviour allows different users and services to securely and privately access and process their data. The AEGIS platform enables users to share their data with other users on the platform and allow access for specific services. Also, users can use different data processing services that are supported by the platform to process and visualise their data. AEGIS is built on top of Hopsworks and Hops. It provides an integrated support for different data parallel processing services such as Spark, Flink, and MapReduce, as well as a scalable messaging bus with Kafka, and interactive notebooks with Zeppelin and Jupyter. Under the hood, the data is mainly stored in the AEGIS data store; however, the AEGIS data store APIs are kept hidden from users. Instead, the AEGIS platform provides a Project/Dataset service to allow users to upload/download, explore, and do analysis on their data in a secure way without interacting with the AEGIS data store directly.
The AEGIS platform includes the Metadata Service responsible for managing the rich metadata associated with a particular dataset within the AEGIS platform and posing the foundation for it processing. It is based on the AEGIS ontology and vocabulary. Another important service in AEGIS is the AEGIS Harvester. It enables the data import to the AEGIS platform offering the transformation, harmonisation and annotation functionalities required within the context of the platform as well as the rich metadata generation for the imported data. Furthermore, the AEGIS platform includes several tools for users with limited technical background, but potentially useful for all, as they simplify and accelerate the work with data. Among them are:
- Query Builder providing the capability to interactively define and execute queries on data available in the AEGIS system,
- Visualiser enabling the advanced visualisation capabilities of the AEGIS platform and
- Algorithm Execution Container accelerating analysis execution by simplifying the steps that data analysts perform, through eliminating the need to author code directly into the notebook.
Additionally, the platform includes the Brokerage Engine acting as a trusted way to record and keep a log of transactions over the AEGIS platforms, which mostly have to do with the sharing of the different data assets. AEGIS is an Open Source solution (https://github.com/aegisbigdata)
Delivering a Big Data IaaS platform to support a multilingual, cross-sector value chain on Public Safety and Personal Security. AEGIS will help EU companies to adopt a more data-driven mentality, extending and/or modifying their individual data solutions and offering more advanced data services.
Automotive and Road Safety Data; Smart Home and Assisted Living; Finance/Insurance Sector Services; Public Safety and Personal Security (PSPS)
The AEGIS platform is a holistic solution as a public/private cloud-based platform, the only European platform with small & big data integration (most of the competitors analysed above are either from the United States or outside Europe), allowing modular design, facilitating customization and therefore infrastructure/resources scarcity. Furthermore, AEGIS provide its users i) with a distribution of a powerful big data stack that can be deployed in a cluster hosted onpremise or over cloud-based infrastructure and ii) ability to put on public display datasets by operating in an environment of distributed AEGIS clusters and allowing partners of the ecosystems to monetize on them. Considering the opportunities, the solutions that are mainly general purposes or science/research oriented, not specifically considering Public Safety and Personal Security (PSPS) and social value; consequently, AEGIS can still become the key venue and the standard for open, social innovation in big data for the PSPS domain, also facilitating a new market on PSPS and nurturing new ventures based on PSPS data. Consequently, AEGIS has the opportunity to enable a balancing between social value creation and economic value capture through value streams from industries which interested to PSPS datasets.
The Aegis Big Data Platform aims to offer:
- Big Data processing, enrichment, storage, analysis and sharing
- Cross-domain batch and streaming data integration and harmonization
- DCAP-AP based metadata
- Data anonymization and semantic enrichment procedures
AEGIS aims at driving data-driven innovation that expands over multiple business sectors (e.g. public, environment, health, automotive, insurance, etc.) and taps structured, unstructured and multilingual data sets to create a novel data value chain around Public Safety and Personal Security (PSPS).
AEGIS offers novel services and applications that allow PSPS-related industries to generate: (a) more factual and evidence-based analytics, (b) improved decision support models, and (c) new business services focused on real-time data collaboration, knowledge sharing and notifications amongst the key stakeholders. Through the AEGIS Platform, the PSPS Data Value Chain Analysis is conducted at multiple levels including: (I) Data Privacy Enhancement, (II) Data Pre-Processing, (III) Big Data Analysis, (IV) Data Intelligence Sharing.
Although open data sources are leveraged toward enhanced PSPS services provision, the AEGIS data ecosystem is based on a trusted multi-level network, enabling proprietary and also private data sharing and seamless integration functionalities in a secure environment under clear terms.
AEGIS builds on:
- The latest advancements in the Linked / Big Data landscape to deliver a framework for semantically enriching and interlinking data.
- The concept of micro-services for enabling a modular and scalable big data architecture that facilitates the continuous integration of data from various sectors / formats / languages.
- The power of the blockchain technology to safeguard security, privacy, quality and IPRs of the data to be utilized.