Automated lazy metalearning in introspective reasoning systems

Abstract

Machine learning systems are becoming increasingly widespread and important, and these days machine learning is used in some form in most industries. However, the application of machine learning technology still has a relatively high barrier to entry, requiring both machine learning expertise and domain knowledge. In this thesis, we present a metareasoning approach to multi-method machine learning that allows the system to adapt and optimize learning for a given domain automatically, without requiring human expert judgment. In contrast to popular deep learning methods for similar situations, the approach presented here does not require specialized hardware nor large data sets. Multiple machine learning components are continuously evaluated at run-time while solving problems, using a framework to analyze overall system performance based on observed prediction performance and time spent. The system automatically learns to prioritize the methods with the best empirical performance for a given domain. In experiments using data sets from the UCI machine learning repository and machine learning methods from the Weka suite, an example implementation outperformed individual methods and other metareasoning approaches.