Cloud Cost Modelling and Optimisation: A Taxonomy and Approaches for Storage Object Classification and Cloud Resource Placement

Abstract

The adoption of cloud services continues to rise due to their flexibility and potential cost savings. However, the cost structure of cloud services is complex and often leads to significant service wastage. Existing cost models are industry-specific and lack the ability to incorporate user requirements, which is why only a small percentage of cloud users actually use these models to optimise service costs. As a result, this often leads to higher cloud usage costs. The primary objective of this thesis is to explore approaches for cloud cost modelling and optimisation in order to aid understanding of the complex cost structure and cost management. This includes identifying varying cost dimensions with cost-saving potential and, based on that, developing industry independent approaches that are applicable to a wide variety of scenarios fulfilling the users’ requirements. In this respect, there are three main contributions produced in this thesis. First, a comprehensive cloud storage cost taxonomy is developed in relation to other cost elements. This taxonomy serves as a framework for understanding the various cost elements associated with cloud storage. It provides a structured approach to dissecting the cost ecosystem, making it more comprehensible and manageable. Moreover, different cost optimisation strategies are identified for storage cost, network usage cost, compute cost etc. Focusing on the first area of optimisation which is storage cost, the second contribution involves the development of two novel approaches for the classification of storage objects across different storage tiers. These approaches aim to optimise the allocation of storage objects, ensuring that each object is stored in the most cost-effective tier. This not only helps in managing storage costs but also enhances the efficiency of data retrieval processes. The proposed storage tier classification approaches are evaluated on both synthetic and semi-synthetic datasets that mimic real-world big data pipeline scenarios. The results show a considerable amount of cost savings compared to the scenario where data is not moved between tiers and is stored in the same tier for the entire duration. Third, in order to optimize network usage cost and address the trade-off between cost and performance, a graph-based approach is devised for the placement of cloud resources. This approach uses graph theory and allows for the optimal allocation of resources, taking into consideration factors such as cost, performance, and availability. For the graph-based approach, four different big data deployment scenarios are created to accommodate various situations. The deployment model is generated through the proposed approach and evaluvated. The evaluations not only demonstrate the potential for cost savings but also the ability of the approach to incorporate Quality of Service (QoS) elements. In summary, through the taxonomy and the proposed approaches, the thesis seeks to simplify the cost structure of cloud services and provides approaches that can lead to more efficient and cost-effective utilisation of cloud services. Based on the evaluations, the proposed approaches show the potential to significantly impact how organisations approach their cloud strategies, leading to notable cost savings and improved operational efficiency.