Mikelangelo - Improving Responsiveness and Agility of HPC Cloud Infrastructure





What user need or pain point is your project addressing?

Mikelangelo targets providers of computational infrastructures, HPC, and big data services. We help providers to take full advantage of cloud computing on virtual infrastructures. Our users use cloud computing to highly utilise their infrastructure, to deploy applications easily, and to isolate applications between tenants as much as possible. Full machine virtualisation fulfills these requirements best, however it exhibits significant degradation of I/O performance, which leads to attrition of physical resources. In turn, poor I/O performance rules out I/O intensive applications. Furthermore, current virtual infrastructures do not handle bursts, which are common in cloud computing, well. Reasons for poor bursting performance are large operating system images and low I/O performance of VMs.

Two large use cases that will benefit from I/O efficient virtual infrastructures are HPC and big data. In HPC, virtual infrastructures are not used, because of the low I/O performance. Thus, end-users need to develop their applications against prescribed operating environments. In big data, end-users also need to adapt their applications to a prescribed big data platform with low flexibility. This lack in flexibility poses an entry barrier for many end-users to use HPC and big data.


Project's major results: 

The main goal of MIKELANGELO is to offer a unified virtualisation stack, able to power HPC, HPC Cloud and Cloud infrastructures. To achieve this ambitious goal, MIKELANGELO focuses first and foremost on performance: optimization of I/O stack, reduction of virtualisation overhead and overall manageability of high performance applications. The raw gains in each of the building blocks are combined with the overall architecture, which allows for IO optimizations on the hypervisor level, efficient telemetry and thus feedback and control of the complete system. The final goal is to integrate research results into existing open source software products and to offer them under well-known and well-supported umbrella of OpenStack platform.


Target stakeholders: 

Small & medium enterprises; Open Source developers; Technology providers; National government agencies; International agencies; Research institutions.



Resources: 

Open Source software available on

AppHub Directory

 
Project Start: 
01/01/2015
Project End: 
31/12/2017


How will your solution/service benefit the end-user?

Infrastructure providers that use Mikelangelo’s stack will be able to host I/O intensive applications in VMs. Providers will achieve better utilisation of their physical resources by providing new platforms to attract more end-users. End-users will gain flexibility, scalability, and elasticity for their applications in the cloud. Most of those benefits reside in the contexts of HPC and big data.

HPC infrastructure providers will run applications in VMs without loss of resources due to I/O inefficiencies. The VM approach to HPC also facilitates rapid scale out of HPC applications to a cloud, which further increases the utilisation of resources across the data centre. End-users will gain the flexibility to choose the best provider since they will be able to define the operating environment in a VM.

Big data infrastructure providers will be able to offer multiple custom platforms to end-users on demand. This flexibility will in turn attract more end-users to use big data applications on the provider’s premises.

Both, HPC and big data, will benefit from Mikelangelo’s service management system with multiple end-user interfaces and an elaborate monitoring system. This monitoring system will allow end-users to identify bottlenecks in their applications to tweak application performance.


Potential exploitation strategy: 
We expect business impact of MIKELANGELO to be on HPC Centers, integrators and owners of private clouds. These are going to benefit from flexible, easy to maintain, open source, standardised Cloud/HPC Cloud technology stack, which delivers performance, low footprint and has increased security. Turning to the delivery of applications in the HPC/HPC cloud area, the packaging of typical HPC applications (e.g. OpenFOAM) into OSv packages, and ensuring their standardized management, should have significant impact for infrastructure providers that will be relieved from having to maintain multiple application versions specific to their clusters, as well as for infrastructure users that will be able to build and deploy applications on top of any MIKELANGELO-compatible infrastructure. Furthermore, expanding the typical HPC applications to cover typical Big Data applications, will have impact in the High Performance Data Analytics field. Overall, we expect the contributions of MIKELANGELO to be accepted upstream into KVM community, thus delivering IO performance to the whole KVM community. The same holds for OSv, snap communities, where we expect all contributions to be accepted upstream, thus delivering value to all of the relevant communities.

Vertical Market: 

Energy, Engineering & manufacturing.