The benefits of cloud seem an attractive option for HPC workloads. Features like flexibility plays an important role as it is not seen in dedicated machines. If on a dedicated machine, different environment is required, a new cluster has to be set up with the required environment which can take upto several days. On the other hand cloud provides variety of environment on-demand easily. The property of elasticity in the cloud is also useful for HPC applications as an HPC application may require scaling in between the computation. Availability is also one of the property useful for HPC workloads. Inspite of all the above advantages of cloud for HPC applications, there are some serious issues related with it. HPC applications are used in academia and scientific laboratories for research, hence need guarantee of resources and timely results. They tend to consume more than 90% of the available CPU cycles, require low latency and high bandwidth inter-process communication, which is difficult to achieve in a virtualized environment due to I/O overhead and extra CPU cycles for I/O processing. Previous work shows that Cloud is one or two magnitude worse as compared to Infiniband, which is currently used in supercomputers for interconnect network. The virtualization of interconnect becomes a bottleneck for HPC applications and increases latency of communication intensive applications. Thus the use of virtualization layer proves to be a major hindrance in performance of HPC applications. The notion of virtualization was brought in mainly to utilize all the existing resources, as virtual machines share the overall resources instead of having equal share of it. But for HPC applications who are capable of utilizing major portion of the available resources do not have the issue of under-utilization. Past results on evaluation of HPC applications on Cloud using IaaS platform with virtualization have been pessimistic. Literature indicates that not all types of HPC applications are suitable for Cloud. The HPC applications with less intensive communication patterns, having less sensitivity towards interference and applications with performance needs that can be met at small to medium scale execution (in terms of number of cores) are suitable for Cloud under virtualized environment. This work discusses about the idea of converging the benefits of Cloud like elasticity and flexibility with HPC platforms to create an environment for HPC workload. PaaS properties are explored for achieving this goal as PaaS has the ability to provide on-demand platform, abstracts and controls the underlying resources and gives choice of platform and guarantee of performance. Instead of improving the virtualization layer we propose a model where virtialization layer is completely removed. By elimination of virtualization layer and with PaaS model a framework is obtained for provisioning of dedicated platforms. On-demand environment is setup by booting a node with desired operating system and runtime libraries for building user-specific HPC platform. The key contributions of this work are as follows:
- Provision of a non-virtualized platform to obtain contention free and dedicated use of resources.
- Design of a PaaS model to provide on-demand and isolated environment to meet HPC application requirements.
- P. A. Dhuldhule, J. Lakshmi and S. K. Nandy, “High Performance Computing Cloud — A Platform-as-a-Service Perspective,” 2015 International Conference on Cloud Computing and Big Data (CCBD), Shanghai, 2015, pp. 21-28. doi: 10.1109/CCBD.2015.56 (PDF)(Talk)
- High Performance Computing Cloud – A Platform-as-a-Service Perspective, Prathima A Dhuldhule, MTech (Comp. Sc.) Project Report, 2015 (PDF)