Deploying OWL ontologies for semantic mediation of mixed reality interactions for human–robot collaborative assembly

Abstract

For effective human–robot collaborative assembly, it is paramount to view both robots and humans as autonomous entities in that they can communicate, undertake different roles, and not be bound to pre-planned routines and task sequences. However, with very few exceptions, most of recent research assumes static pre-defined roles during collaboration with centralised architectures devoid of runtime communication that can influence task responsibility and execution. Furthermore, from an information system standpoint, they lack the self-organisation needed to cope with today’s manufacturing landscape that is characterised by product variants. Therefore, this study presents collaborative agents for manufacturing ontology (CAMO), which is an information model based on description logic that maintains a self-organising team network between collaborating human–robot multi-agent system (MAS). CAMO is implemented using the Web Ontology Language (OWL). It models popular notions of net systems and represents the agent, manufacturing, and interaction contexts that accommodate generalisability to different assemblies and agent capabilities. As a novel element, a dynamic consensus-driven collaboration based on parametric validation of semantic representations of agent capabilities via runtime dynamic communication is presented. CAMO is instantiated as agent beliefs in a framework that benefits from real-time dynamic communication with the assembly design environment and incorporates a mixed-reality environment for use by the operator. The employment of web technologies to project scalable notions of intentions via mixed reality is discussed for its novelty from a technology standpoint and as an intention projection mechanism. A case study with a real diesel engine assembly provides appreciable results and demonstrates the feasibility of CAMO and the framework.