CLOUDLIGHTNING – Self-Organising, Self-Managing Heterogeneous Cloud
What user need or pain point is your project addressing?
CloudLightning aims to address the problems surrounding cloud infrastructure management, service specification, access to, and provisioning of, cloud resources faced by both consumers and providers of cloud services. CloudLightning’s initial target application domains are the oil and gas industry, genomics, and sectors that use ray tracing.
Currently, from a cloud service provider perspective, cloud service delivery models typically rely on over-provisioning to guarantee quality of service and accommodate unpredicted peaks in demand. The typical cloud server operates at approximately 20% computing capacity. The output capacity could be increased to 80% using heterogeneous processing resources with no change to the server’s energy consumption. This represents a model of inefficiency, coalescing in ongoing sub-optimal energy consumption and related costs for the service provider and ultimately the consumer. Additionally, service providers must grant users access to resources to allow them to configure the service to meet their requirements, thus requiring security and access protocols to manage user access and permissions.
From a cloud consumer perspective, when interfacing with their chosen cloud service provider, they are tasked with the provisioning, configuring and optimisation of resources to meet their requirements. This requires a commensurate level of IT expertise and knowledge of the provider’s infrastructure.
Project's major results:
The most important result of the project is the achievement of the project goal: the development of a new cloud service delivery model that provisions heterogeneous cloud resources to deliver cloud services. The most important results will be realised through the expected impacts on the chosen application domains - oil and gas exploration, ray tracing (image rendering) and genome processing – in addition to the wider cloud services market. The CloudLightning service delivery model will result in greater energy efficiency and reduction in associated costs through greater server and resource utilisation.
At present (beginning of Year 2) the achievements to date include:
• Conceived system architecture of the completed CloudLightning service.
• Methodology for integrating heterogeneous resources (such as a Numascale machine and Data Flow Engine) into the CloudLightning model.
• Development of a Service Gateway using a modified version of the Brooklyn Service Description Language.
• Development of a prototype simulator of the CloudLightning simulator.
• Successfully characterised the various heterogeneous resource types including GPUs, MICs, and DFEs.
• Report detailing the requirements and detailing the evaluation criteria of the three use cases based on the project’s three application domains.
The project has successfully engaged with domain experts including the formation of the project’s External Advisory Board. The Board met in March 2016 to provide feedback and guidance.
Further stakeholder engagement included interviews and surveys with ray tracing and image rendering user communities to provide key insights into the ray tracing use case.
Target stakeholders:
Small & medium enterprises, Open Source developers, Large companies, Technology providers, National government agencies, Research institutions, Other.
Project Start:
01/02/2015
Project End:
31/01/2018
How will your solution/service benefit the end-user?
Consumers now have improved accessibility to heterogeneous cloud resources. The self-organising, self-managing principles used by CloudLightning provide consumers with a user-friendly service level interface to explicitly declare their requirements for service delivery. They can do this by specifying their exact workflow, dataset and time deadline. Through the assembly of dynamic resource coalitions, the self-organising and self-managing system automatically and intelligently locates the required resources and presents the most appropriate configuration of that service to the user for selection.
Cloud service providers now enjoy power-efficient, scalable management of their cloud infrastructures and better overall utilisation of service. CloudLightning’s new standard in-service description language is not only easy to implement, but means that the service configuration presented by the self-organising, self-managing system will always be within the constraints of the service provider’s resources, thereby providing optimal resource allocation and energy consumption. The coalition formation and service deployment processes negate the need to grant users access to manually configure their service description, thus providing an additional layer of security to the provider.
CloudLightning benefits service providers with the freedom to use its resources optimally and frees the consumer of the need and expertise required to configure cloud service delivery.
Potential exploitation strategy:
The exploitation of the CloudLightning project will include both Academic and Commercial exploitation.
Academic Exploitation:
Furthermore, under the academic exploitation plan, the academic partners will integrate technical developments into the teaching curricula and research agendas of their respective institutions thereby providing their institutions and graduates a competitive edge over other higher education institutions. In addition to academic papers, this will include the production of the following:
• Lecture Script
• PowerPoint Presentation and Walk Through
• Reading List and Support Resources
• Open Courseware
Commercial Exploitation:
CloudLightning’s commercial exploitation activities will focus on stakeholders with the potential of licensing or investing in the CloudLightning system. As such, the focus of the commercial exploitation plan is to engage and collaborate with the following category of stakeholder as prioritised below:
Priority 1
• Independent Software Vendors
• Systems Integrators
• Cloud Service Providers
• Investors and Licensors
Priority 2
• Hardware Manufacturers
• Telecommunications and Network Service Providers
• Industry Analysts
• Open Source Initiatives
Vertical Market:
Digital health, Energy, Media, Other.
Website: http://cloudlightning.eu/
