IN
DEGREE PROJECT MECHANICAL ENGINEERING, SECOND CYCLE, 30 CREDITS
STOCKHOLM SWEDEN 2020,
Digital Twin: Visualization- Assisted Corrective
Maintenance
AAMIR MALIK SHAIK
SIDHVIN DULEVALE MATADHA
KTH ROYAL INSTITUTE OF TECHNOLOGY
SCHOOL OF INDUSTRIAL ENGINEERING AND MANAGEMENT
Digital Twin:
Visualization-Assisted Corrective Maintenance
MF224X, Degree Project in Mechatronics, Second Cycle - KTH
AAMIR MALIK SHAIK
SIDHVIN DULEVALE MATADHA
Date: November 23, 2020
Supervisor: Jad El-Khoury (KTH) and Franz Waker (Scania) Examiner: Martin Edin Grimheden
KTH Royal Institute of Technology Host company: Scania AB
Swedish title: Digital Tvilling: Visualiseringsassisterat Korrigerande Underhåll
Examensarbete TRITA-ITM-EX 2020:594
Digital Tvilling: Visualiseringsassisterat Korrigerande Underhåll
Aamir Malik Shaik Sidhvin Dulevale Matadha
Godkänt
2020-11-19
Examinator
Martin Edin Grimheden
Handledare
Jad El-Khoury (KTH)
Uppdragsgivare
Scania CV AB
Kontaktperson
Franz Waker (Scania)
Sammanfattning
Denna avhandling utvärderar betydelsen av den datadrivna lösningen som baseras på en Digital Tvilling vilken hjälper korrigerande underhållstekniker att utnyttja sina tvärvetenskapliga tekniska färdigheter vid åtgärdande av komplexa mekatroniska problem.
Med anledning av felens komplexa mekatroniska natur är mänsklig inblandning nödvändig vid korrigerande underhåll. Även idag utför många industrier korrigerande underhåll genom att använda metoder som både är tidsineffektiva och felbenägna. Mjukvaru- eller AI-baserade lösningar har vida rapporterats ha misslyckats med anledning av ett försummande av den mänskliga aspekten i underhåll. Än så länge kan inte den mänskliga rollen helt ersättas av mjukvarusystem. Standardpraxis för underhåll såsom FMEA och RCA är kostsamma,
tidskrävande och känsliga för fel. Å andra sidan har lösningar som baserats på en Digital Tvilling (DT) visat på förbättrad hantering och effektivitet av underhåll genom att ta hänsyn till den mänskliga aspekten. För korrigerande underhåll är lösningen dock fortfarande i sitt konceptuella skede. Det finns ett behov av att praktiskt implementera en lösning baserad på en Digital Tvilling och kvantitativt utvärdera dess betydelse.
Nyligen utförda studier har visat att ett Digtal Tvilling-koncept, uppbyggt på ett modellbaserat tillvägagångssätt, har en enorm potential i att tillhandahålla alla nödvändiga data som krävs för att kontrollera beteendet hos ett nätverk av fysiska enheter och samtidigt virtuellt övervaka deras verkliga tillstånd effektivt. Denna avhandling försöker först utveckla användarcentrerade
visualiseringar uppbyggda på en helt integrerad Digital Tvilling av ett komplext Cyberfysiskt Produktionssystem (Cyber Physical Production System CPPS) och försöker sedan utvärdera dess effektivitet (när det gäller korrekthet och effektivitet) för att lösa problemet vid korrigerande underhåll.
Experimentella resultat visar att när uppgiften vid korrigerande underhåll assisteras av användarcentrerade visualiseringar från en i realtid motsvarande Digital Tvilling förbättrades underhållsteknikerns noggrannhet och effektivitet med cirka 24% respektive 52,4%. Vidare förklarar en post-experimentell kvalitativ analys att det inte är vilken visualisering som helst utan en datadriven visualisering baserad på en digital tvilling och uppbyggd på användarkrav som hjälpte till att utföra uppgiften för korrigerande underhåll mer effektivt.
Master of Science Thesis TRITA-ITM-EX 2020:594
Digital Twin: Visualization-Assisted Corrective Maintenance
Aamir Malik Shaik Sidhvin Dulevale Matadha
Approved
2020-11-19
Examiner
Martin Edin Grimheden
Supervisor
Jad El-Khoury (KTH)
Commissioner
Scania CV AB
Contact person
Franz Waker (Scania)
Abstract
This thesis evaluates the significance of the Digital Twin based data-driven solution, in helping corrective maintenance technicians leverage their multi-disciplinary engineering skills to solve complex mechatronic problems.
Due to the complex mechatronic nature of the faults, human involvement is necessary for corrective maintenance. Even today, many industries perform corrective maintenance by following methods that are both time inefficient and error prone. Software/AI based solutions have been widely reported to have failed due to neglect of human aspect in maintenance. The role of human cannot be completely replaced by software systems yet. Standard maintenance practices such as FMEA and RCA are costly, time consuming and susceptible to errors. On the other side, Digital Twin (DT) based solutions have shown to have improved management and effectiveness of maintenance by considering the human aspect. However, for corrective maintenance, the solution is still in its conceptual stage. There is a need to practically implement a Digital Twin based solution and quantitatively evaluate its significance.
Recent studies have shown that Digital Twin concept, built on model-based approach, has a tremendous potential in providing all the essential data required to control the behaviour of a network of physical devices, and at the same time, virtually monitor their real-world states effectively. This thesis first attempts to develop user-centric visualizations built on a fully- integrated digital twin of a complex Cyber Physical Production System (CPPS), and then it tries to evaluate its effectiveness (in terms of correctness and efficiency) in solving the corrective maintenance problem.
Experimental results show that when the corrective maintenance task is assisted by user-centric visualizations from a real-time Digital Twin, it significantly improved the accuracy and efficiency of the maintenance technician by about 24% and 52,4% respectively. Further, a post- experimental qualitative analysis explains that it is not any visualization but a Digital Twin based data-driven visualization, built on the user requirements that helped perform the corrective maintenance task more effectively.
Acknowledgements
We take this opportunity to express our deepest gratitude and appreciation to all those who have helped us directly or indirectly towards the successful completion of this thesis.
First and foremost, we would like to express our sincere appreciation and grat- itude to our esteemed guides Juan Luis Jiménez Sánchez, Project Engineer, and Lars Hansson, Project Manager, Smart Factory Lab, Scania CV AB for their insightful advice, encouragement, guidance, critique, and valuable sug- gestions throughout the course of our thesis work. Without their continued support and interest, this thesis would not have been the same as presented here.
A big thank you is also directed to the KTH supervisor Jad El-Khoury for his guidance through continuous reviewing of the work, keeping the project on track. We are also grateful to all our colleagues at Smart Factory Lab for their help, encouragement and invaluable suggestions.
Our special thanks to our parents, supporting families and friends who contin- uously supported and encouraged us in every possible way for the successful completion of this thesis. Last but not least, we thank God Almighty for His blessings without which the completion of this thesis work would not have been possible.
Aamir Shaik and Sidhvin Dulevale Matadha November 23, 2020, Stockholm
vi
List of Figures
2.1 Digital Model . . . 9
2.2 Digital Shadow . . . 10
2.3 Digital Twin . . . 10
3.1 Skill-based Approach . . . 15
3.2 Implementation Approach 1 . . . 16
4.1 Methodology workflow diagram . . . 21
5.1 ABB IRB 1200 [60] . . . 26
5.2 Yaskawa HC20 . . . 27
5.3 Connections between physical and virtual environments . . . . 29
5.4 An sample of the Smart Factory Lab environment in Unity . . 30
5.5 Yaskawa Motoman HC20 . . . 31
5.6 Integration and Processing in middleware Node-RED . . . 32
5.7 One-way Physical->Virtual . . . 33
5.8 Other-way Virtual->Physical . . . 34
5.9 User Interface of the Maintenance Technician . . . 35
5.10 A top down view of Smart Factory Lab . . . 37
6.1 Time Comparison for Individual tasks . . . 44
6.2 Total time taken for each task . . . 45
6.3 Number of errors in diagnosis for individual tasks . . . 48
6.4 Total number of errors in diagnosis for each task . . . 49
6.5 Post-experiment analysis for participants without DT . . . 51
6.6 Post-experiment analysis for participants with DT . . . 51
vii
List of Tables
3.1 Component RPN ranking . . . 13
6.1 User-Centric requirements mapping . . . 41
6.2 Time taken to solve the maintenance tasks in seconds . . . 42
6.3 ANOVA Analysis for time data . . . 43
6.4 Average time for each task in seconds . . . 45
6.5 Correctness : number of errors while solving the maintenance task . . . 46
6.6 ANOVA analysis on number of errors . . . 47
6.7 Average number of errors in diagnosing the fault . . . 50
viii
Acronyms
DT Digital Twin
CPPS Cyber Physical Production System RTLS Real Time Locating System IoT Internet of Things
AGV Automated Guided Vehicle OSI Open Systems Interconnect CPS Cyber Physical Systems
RF-ID Radio Frequency Identification ANOVA Analysis of Variance
HMI Human-Machine Interface
FMEA Failure Mode And Effects Analysis RCA Root Cause Analysis
FTA Fault Tree Analysis RPN Risk Priority Number AI Artificial Intelligence ML Machine Learning UI User Interface
TCP Transmission Control Protocol IP Internet Protocol
HTTP Hypertext Transfer Protocol URI Uniform Resource Identifier HTML Hypertext Markup Language REST Representational state transfer GUI Graphical User Interface VR Virtual Reality
ix
Contents
1 Introduction 1
1.1 Scania AB background . . . 2
1.2 Problem analysis . . . 2
1.3 Purpose . . . 5
1.4 Research Question . . . 5
1.5 Requirements . . . 5
1.6 Delimitation . . . 6
1.7 Reader’s Guide . . . 7
2 Background 8 2.1 Cyber Physical System and Digital Twin . . . 8
2.1.1 Levels of Integration . . . 9
3 Literature Review 11 3.1 Corrective Maintenance . . . 11
3.1.1 AI Approach . . . 11
3.1.2 Standard maintenance solutions . . . 13
3.1.3 DT based Approach . . . 14
3.2 Digital Twin . . . 17
3.3 User-Centric Visualizations . . . 17
4 Methodology 19 4.1 Project Management . . . 19
4.2 Project Organisation . . . 19
4.2.1 Digital Twin Creation . . . 22
4.2.2 Interviews . . . 22
4.2.3 Controlled Experiment . . . 23
5 Implementation 25
x
5.1 Digital Twin Creation . . . 25
5.1.1 Physical Model . . . 25
5.1.2 Virtual Model . . . 27
5.1.3 Communication between Physical and Virtual Models 28 5.1.4 Process of DT Creation . . . 29
5.2 Interviews . . . 34
5.2.1 User-Centric Visualizations . . . 35
5.3 Controlled Experiment . . . 37
5.3.1 Experiment Design . . . 37
5.3.2 Verification and Validation . . . 39
6 Results and Discussion 40 6.1 Interviews . . . 40
6.1.1 Mapping of individual requirements . . . 40
6.1.2 Experiment scenarios . . . 41
6.2 Controlled Experiment . . . 42
6.2.1 Time . . . 42
6.2.2 Correctness . . . 46
6.2.3 Qualitative post experiment interview . . . 50
6.2.4 Reliability and Validity of Study . . . 52
7 Conclusion and Future Work 54
Bibliography 56
A Interview Sheet A1
B Post Experiment Questionnaire B1
Chapter 1 Introduction
Industrial maintenance pertains to the actions that contribute towards the up keep of production equipment. The throughput of a production facility is a key factor in its profitability, which in turn is directly affected by the functional availability of production equipment. According to [1], apart from the eco- nomic factor, the maintenance of industrial equipment also impacts reliability, availability and security of the production system. The main aim of industrial maintenance is to maximise the availability and minimize the breakdowns of these equipment. [1] enlists three types of maintenance technologies namely :
• Predictive Maintenance : maintenance performed based on the condi- tion of the system at a given point in time.
• Preventive Maintenance : scheduled maintenance performed at fixed time intervals. The aim is to reduce system breakdowns by maintain- ing its health through regular maintenance.
• Corrective or Breakdown Maintenance : unscheduled maintenance per- formed when the system fails or there is an unexpected fault in the sys- tem.
It is estimated that the cost of corrective maintenance is 3-4 times higher than that of preventive maintenance. Undeterred, corrective maintenance is still un- avoidable and necessary because of the complications involved in predicting it due to the complexity of the production systems [2][3]. Thus, improving cor- rective maintenance would positively impact production cost and efficiency.
And this thesis focuses on improving corrective maintenance.
Since the main aim of corrective maintenance is to quickly restore the faulty
1
system back to its normal operation, time efficiency and correctness are inte- gral to the activity. In many industries even today, the corrective maintenance activity is mostly based on experience, gut feeling and luck through trial &
error method. This does not always guarantee an efficient outcome and is very much prone to human errors. In this digital era where we are surrounded by large volumes of useful data, what if we could tap the potential of this useful data to gather insights for making informed and rapid decisions? How do we make use of such data? What impact will it have on corrective maintenance?
are some of the questions this thesis aims to study.
1.1 Scania AB background
Scania is a major European manufacturer of trucks and buses and is now part of the Traton Group owned by the Volkswagen Group. It also manufactures diesel engines for heavy vehicles, marine and other industrial applications, and provides workshop services and support for their customers’ operations.
It was founded in 1911 as a merger between Södertälje based Vabis and Malmö- based Maskinfabriks-aktiebolaget Scania. Volkswagen Group acquired 100%
stake in the beginning of 2015. Scania had a revenue of 152.419 million SEK and has 51,278 employees as of 2019 according to the Scania Annual and Sus- tainability Report 2019 [4]. It operates all across the world and has a total of 10 factories worldwide.
The Smart Factory Lab at Scania Södertälje is an experimental test environ- ment that explores, assesses and pilots new technologies for adoption in Sca- nia’s production processes across all factories. The multinational team con- sists of 15 engineers, thesis workers and trainees, working together to carry out trials.
This thesis is part of an ongoing project at Scania’s Smart Factory Lab.
1.2 Problem analysis
Modern industries are growing increasingly complex trying to meet the ever- growing customer demands and maintain a competitive edge in the market.
The more complex a system gets, the higher the probability and the ways it can fail.
Corrective maintenance is usually performed by human technician. Human in-
volvement is necessary to handle complexity of mechatronic components and take rapid and correct decisions [5]. The process involves the human techni- cian analyzing the fault, ascribing the fault to a cause and then carrying out the corrective action based on the ascription [6][7]. In case of distributed Cyber Physical Production Systems (CPPS), the scenario is complex, wherein mul- tiple factors affect the operation of a component [8]. Thus, a failure can be of multiple types: mechanical failure, electrical failure, software failure, service interface failure or a combination of these. Thus a failure is ’Mechatronic’
in nature. The human technician needs to analyze complex systems, apply- ing multi disciplinary aspects in order to perform the corrective action [7][6].
Thus, the chances of human error occurrences during the corrective mainte- nance increases. And any human error elongates the production downtime and leads to production losses [7]. Thus, there is a need to provide assistance that can fully support corrective maintenance of mechatronic systems in order to minimise human errors and increase the technician’s efficiency.
Now, reflecting on the state of corrective maintenance at Scania: Scania has adopted the following process for corrective maintenance upon failure of a machine:
1. The line operator handling the affected machine would contact the Main- tenance Centre and report the fault from his/her own understanding.
2. The Maintenance Centre would then convey this information via a pager to the designated maintenance technician. The information includes the machine ID number and a brief description of the reported fault.
3. The maintenance technician upon visiting the affected machine, would be briefed by the operator about the time and reason for the occurrence of the fault. The operator also often expresses his/her opinion on fault identification.
4. The technician would then proceed with troubleshooting the problem solely based on his/her own skills and experience.
This process suffers from the following problems:
• Since the line operator lacks deep technical knowledge about the system, the description of fault as explained to the Maintenance Center usually happens to be incorrect and/or misinterpreted. Due to which, the Main- tenance Center may not be able to assign a technician with the right skills for solving the problem. Also often, the line operator while expressing his/her diagnosis of the problem to the maintenance technician, ends up
misdirecting the technician.
• The maintenance technician attempts to identify the fault and find its cause without any systematic guidance or standard procedure. For in- stance, by following instincts from experience and/or trial error tech- nique to solve the problem. These approaches often lead to incorrect solution tryouts and loss of time which is critical in case of corrective maintenance.
In summary, the current approach suffers from poor maintenance manage- ment, and the lack of systematic data-driven approach to fully support cor- rective maintenance of mechatronic systems, often resulting in incorrect and time consuming troubleshooting.
The literature presents three broad categories of solution to this problem namely:
AI based solution, Standard maintenance solutions and the Digital Twin (DT) based solution, which are discussed in detail in Chapter 3. The AI based solu- tion and the Standard maintenance solutions do not handle the complexity of mechatronic systems effectively to solve the corrective maintenance problem.
Whereas the DT-based solution has shown to be effective but lacks practical implementation and evaluation for its use in corrective maintenance.
Nowadays, industries are growing rapidly. In order to thrive in this increas- ingly competitive market, companies are aiming for higher production effi- ciency and better utilization of resources. This calls for making informed de- cisions based on solid understanding of the production system state through the real-time data it generates. This requires the useful data to be identified, processed and visualized, facilitating monitoring and control of the physical systems, which is most effectively achieved using a digital mirror of the phys- ical world – “A Digital Twin” [9].
A Digital Twin is a virtual representation of a physical product or process. It helps manufacturers see, understand, and improve the manufacturing process.
The main point of Industry 4.0 is improving operational efficiency in manu- facturing. This in fact can be achieved through digitalization and exploitation of key data (KPIs). Production quality would improve exponentially with the implementation of digital twin [10]. When digital twin is coupled with main- tenance, it can positively impact not only its management but also its efficiency and accuracy. This thesis attempts to implement and evaluate the significance of employing a digital twin to assist in a corrective maintenance task through its user-centric visualizations.
1.3 Purpose
The thesis aims:
• to develop a digital twin for a complex CPPS involving different physical devices in a distributed network.
• to design user-centric visualizations building onto the developed DT, acting as a graphical user interface.
• to evaluate the significance of user-centric data visualizations through a digital twin, for a corrective maintenance application.
1.4 Research Question
Based on the purpose and the problem statement, the following research ques- tion has been proposed:
For Corrective Maintenance in a complex CPPS, how effective (in terms of correctness and efficiency) is the use of user-centric
data visualizations through a digital twin, in aiding the maintenance technician leverage their multi-disciplinary skills to
solve complex maintenance tasks?
The above research question was formulated after carrying out literature re- views on previous studies. Some authors have presented different approaches to solve the corrective maintenance problem. Among which, this thesis at- tempts to evaluate the Digital Twin solution considering the tremendous po- tential of digital twin for smart manufacturing.
1.5 Requirements
A set of requirements were stated by the stakeholders in order to limit the thesis and to define the expected outcome of it. These were divided into must-haves and nice-to-haves to ensure that the scope of the thesis was maintained during the whole period.
The following requirements were the must-haves in the project:
1. Study should include interviews with the identified user groups, to better understand the problem. At least 1 interview should be conducted with each of the target user groups.
2. The end user groups must belong to distinct functional levels, i.e. Line Operator, Maintenance Engineer and Production Manager.
3. The visualizations to be developed must be based on the requirements from the identified end user groups.
4. The digital twin must contain at least two physical devices to be consid- ered "built for complex CPPS".
5. The digital twin must use the existing factory layout model.
6. The digital twin must have bidirectional communication with its physi- cal devices.
7. The operation of the physical devices shall always be performed in a safe manner for everyone involved.
8. The virtual model of the physical devices shall be built on one of the most popular real-time development platform for 2D and 3D interactive experiences, which is called ’Unity’.
9. The experimental validation of the built solution must include at least 10 participants, preferably with maintenance background.
The following requirements were the nice-to-haves in the project and were supposed to be included if time and resources are available:
10. The digital twin shall include more than 2 physical devices.
11. The digital twin shall follow its real counterpart accurately and in real time.
12. The experimental validation shall include as many participants as pos- sible (20-30 would be optimal).
1.6 Delimitation
The project was bound to certain limitations due to a number of factors such as time, people and resources.
For the development of digital twin concept, the use case scenario at Smart Factory Lab involved a network of Cyber Physical Systems (CPS). From which, the physical devices that were considered in this research were the 6-axis in- dustrial robots: ABB IRB 1200 robot and Yaskawa HC20 robot, which repre- sented the Logistics and Assembly workstations respectively. If time and re-
sources permit, it is desired to include other physical devices: MiR Automated Guided Vehicle (AGV), Radio Frequency Identification (RF-ID) Reader, Real Time Locating System (RTLS) devices and other Internet of Things (IoT) sen- sors.
Due to the prevailing regulations for Covid19 at Scania, the interviews to be conducted to determine the user requirements for visualizations were limited both in number and in manner of conduction. The interviews shall be con- ducted online with at least 1 interview for each user group.
To be able to use this digital twin solution in the future, the design shall be made modular by using a middle-ware integration system ’Node-Red’ between the physical and virtual models. This ensures that it can be scaled to include the entire shop floor or a production plant for future expansion.
1.7 Reader’s Guide
The following chapter provides the background knowledge that is necessary to understand the technical concepts discussed in the thesis. Chapter 3 discusses the literature review on different solutions that exist to the corrective main- tenance problem. In Chapter 4, the methodology is explained. This chapter describes the organization of the team in order to meet the requirements, and the workflow that was designed to achieve the goal of this thesis. Chapter 5 entails a detailed description of the implementation describing the interview process, the digital twin development process and the design of the controlled experiment, along with their verification and validation processes. The results from the implementation process are then presented and discussed in Chapter 6. The final chapter states the conclusions and ends with proposition of future work. In the appendix, there is the interview guide and the Post-experiment Questionnaire that were designed.
Chapter 2 Background
This chapter describes some relevant concepts that are necessary to understand the discussions that follow in the subsequent chapters.
2.1 Cyber Physical System and Digital Twin
CPS integrates a physical object in real world with its digital model (cyber twin or digital twin) in virtual world through sensors, network and computing devices.[11] DT serves as one of the core technologies to realize CPS, which by definition is an exact copy of a physical object including its properties and environment[12].
Thus, CPS can be defined as a holistic system that includes the physical object, communication interfaces, computational hardware, software applications and the digital model. [12]. Likewise, a Digital Twin can be defined as an inter- connected digital model that represents the physical object.
“A digital twin is a computerized model of a physical device or system that represents all functional features and links with the working elements.”[13].
A DT is constantly in sync with its physical system through continuous data exchange and provides real-time status of the system through ultra-realistic simulations [14]. Besides monitoring, a DT also provides direct control over the physical system.
Pertaining to production, the following definition of digital twin exists: "The DT consists of a virtual representation of a production system that is able to run on different simulation disciplines that is characterized by the synchroniza-
8
tion between the virtual and real system, thanks to sensed data and connected smart devices, mathematical models and real time data elaboration. The topi- cal role within Industry 4.0 manufacturing systems is to exploit these features to forecast and optimize the behaviour of the production system at each life cycle phase in real time." [15]
2.1.1 Levels of Integration
Due to multiple definitions and concepts of DT, there is an obscure understand- ing of this DT concept [16] [17] [18]. In order to get a better understanding of digital twin, the concept is further classified based on its level of integration as discussed below [19]. Here, manual data flow refers to communication that only happens when triggered by the user while automatic data flow refers to automated communication without continuous trigger from the user.
Digital Model Digital Model is a digital representation of a physical system without any form of automated data exchange between the physical and digital systems.
Figure 2.1: Digital Model
Digital Shadow Digital Shadow is a digital representation of a physical sys- tem with automated one way data flow from the physical system to the digital system but not vice versa.
Figure 2.2: Digital Shadow
Digital Twin Digital Twin is a digital representation of a physical system with automated bi-directional data flow between the physical and digital sys- tems.
Figure 2.3: Digital Twin
Chapter 3
Literature Review
This chapter introduces some most relevant works that address the corrective maintenance problem, providing insights into potential solutions.
3.1 Corrective Maintenance
After conducting a thorough literature review using the search engines: Google Scholar, Science Direct, Scopus and IEEE Xplore, the solutions to the correc- tive maintenance problem can be broadly classified into three categories: the AI/Software based solution, the DT based solution and the standard mainte- nance solutions such as Failure Mode And Effects Analysis (FMEA) and Root Cause Analysis (RCA).
3.1.1 AI Approach
AI or Software based approach employs AI techniques to optimize mainte- nance based on the system log data and/or the sensor data that provide aware- ness about the condition and performance of the system[20]. The AI approach can be further sub-classified into 3 groups: Expert Agents, Machine Learning (ML) techniques and Fuzzy logic.
Expert Agents are designed based on failure symptoms and causes to find the root cause to a problem. They are built on expert’s knowledge, which is never lost unlike the human knowledge/experience which is lost when the human becomes unavailable or dies. Further, expert systems does not deteriorate with time and produces consistent results unlike human experts that deteriorate due
11
to lack of practice. [20] presents an expert system built using before-and-after analysis to optimize maintenance of pumps in petroleum storage industry.
ML approach employs ML techniques such as artificial neural networks, linear regression, clustering, pattern identification and classification, to develop a model that optimizes maintenance. The model is developed by training it with huge volumes of data. This data can be either system log data, sensor data or both. [21], [22] and [23] use log data to train the model, while [24], [25] and [26] use sensor data and [27] uses both.
The third approach uses probabilistic fuzzy logic technique to develop a soft- ware model that can optimize maintenance. Alternatively, a combination of these approaches can be employed to achieve cumulative better results. [28]
applies ML technique (Artificial Neural Networks) and Fuzzy logic technique to develop a model for fault detection and simulation.
These AI methods are efficient in identifying the commonly identified prob- lems and also the unidentified complex problems to some extent. But in case of complex mechatronic systems, the occurrence of unidentified combined faults is more likely and this warrants human involvement as explained earlier. Be- sides the general concerns with the AI approach requiring large volumes of data to train a model, its time intensiveness and its high susceptibility to er- rors [29], it should be noted that it does not consider the human involvement to jointly solve complex maintenance problems [30]. Human aspect plays a crucial role in improving the the efficiency and effectiveness of maintenance activities[31]. Failure with computer integrated systems had been widely re- ported due to neglect of organisational and human factors[32]. The mainte- nance teams have a lot of knowledge about the systems that cannot be com- pletely substituted by software data analysis of sensors[33]. A significant part of most maintenance processes is relatively straightforward to model; however, human cognitive involvement with more diverse and complex tasks is almost impossible to model [34]. As discussed earlier, in case of complex CPPS, the faults are highly complex due to their mechatronic nature. Potential problems could arise with software approach neglecting human factor, including failure to recognise and solve completely new and unidentified faults, failure to di- agnose correctly when two different faults produce the same symptoms, etc.
Research on Human-AI collaboration to accomplish complex goals is under- way. [30][34]
3.1.2 Standard maintenance solutions
The second category of maintenance solution include FMEA and RCA. These techniques help identify the root cause to a corrective maintenance problem.
The FMEA analysis data should be already prepared and available prior to the occurrence of the problem since its preparation time is considerably high. The analysis involves a study of all possible causes to the problems an equipment experiences.
[35] suggests FMEA as a solution to a corrective maintenance problem. FMEA is an engineering technique that supports maintenance by identifying potential failure modes of a system and measuring its associated Risk Priority Number (RPN) [36][37]. In this approach, the RPN values are ranked in order of de- creasing values. These ranked RPN values facilitate fault identification and isolation. The method helps the technician to identify the components with the highest RPN value, where the probability of locating the failure is high.
For example, the table below shows ranked RPN values for a simple pneu- matic valve. The highest RPN value is associated with spring component D.
The technician would first attend to this component and then proceed to the subsequent components based on their RPN rank: O-ring, C Seal and so on.
Component RPN Rank
D Spring 450 1
O-ring 320 2
C Seal 114 3
Body 40 4
Table 3.1: Component RPN ranking
Further, the author demonstrates a case study to prove the proposed FMEA approach. However, the dependency of the approach purely on the FMEA data of the system could result in its failure when an unidentified combined fault occurs, which is very likely in case of complex mechatronic systems. Also, since no quantitative evaluation has been presented by the paper, it is difficult to understand the true significance of this approach. Regardless, performing an FMEA of a system is costly, because it requires expert working hours; is time consuming and error prone, because of the complexity of mechatronic systems and human involvement. The other maintenance technique: RCA [38]
also suffers from such limitations. Considering these complications, due to the
limitation of resources, it was deemed infeasible to use this solution approach in our thesis.
3.1.3 DT based Approach
The third category of corrective maintenance solution employs a cyber-physical model of the system taking into consideration the human aspect in mainte- nance, providing necessary contextual data to help the maintenance technician leverage his/her skills to solve the problem[39][7][8].
DT based approach can improve quality, effectiveness and management of the maintenance activity[40].
Due to the complex nature of mechatronic component/system, human involve- ment is necessary to make rapid and correct decisions [6], and the system envisioned needs to be modular[7] to provide flexibility, adaptability and re- configurability to the system. [6] and [7] propose Model-based Engineering as a solution to handle this complexity of the mechatronic system. The model- based approach requires representing multidisciplinary information and their interactions and interdependence within the system. Since the function of component/system remains the same for all disciplines involved, a functional- oriented development with skill-based approach is recommended by the au- thors of both the papers. Skill-based approaches are found to reduce the com- plexity of modeling a system. In these approaches, skills represent capabil- ities of a mechatronic system, which are used as interfaces to the system, as described in the figure 3.1 below.
Figure 3.1: Skill-based Approach
Figure 3.1 shows a mechatronic system model constituting the interlinked phys- ical model, kinematic model and electrical model. Skills represent its control inputs while the system states represent its observable control outputs. Trig- gering a skill triggers a transition in the state of the system. Any fault occurring can be detected by triggering a skill input and then comparing the expected behavior of a component(state) with the real behavior influenced by sensor values. The identified fault is then ascribed to its cause(s) based on the chain of elements in the mechatronic model that were responsible for the expected behaviour.
Based on this skill-based approach, [6] conceptualizes a cyber-physical sim- ulation model that envisages these skill-interfaces and the behaviour of the component as shown in the figure 3.2 below. Here, the control inputs(skills) are triggered by the user from the cyber model causing the actuators in the physical model to behave accordingly, while the control outputs are obtained from the sensors in the physical model and monitored in the cyber model.
Figure 3.2: Implementation Approach 1
In addition, the author in [6] asserts using a visual interface and providing fault-debugging recipes like manuals, standard operating procedures, fault clas- sification/ascription techniques and engineering data, to help the human to solve the problems. It should be noted that this solution presented by the au- thor is purely conceptual, there is no practical implementation of the same.
Unlike the cyber-physical model presented by [6], a more recent study [7] pro- poses a solution concept which is exactly same in principle but differs in the implementation methodology. [7] approach employs a digital twin built using the same model based functional approach. And in addition to skill interfaces and real-time engineering data, the digital twin also incorporates fault ascrip- tion techniques through a GUI to assist the maintenance technician. Any fault occurring can be detected by the digital twin model and then ascribed based on some fault ascription logic. While the solution presented by the former research [6] was purely conceptual, this research [7] has taken a step forward by implementing a virtual digital twin model of a real component and having tested the concept in simulation. However, neither of the papers have prac- tically implemented the solution concept on a real system and evaluated its significance. This thesis proposes to take the next step towards implementing the proposed solution on real CPPS using a Digital Twin which provides both real-time engineering data and the fault ascription techniques, and evaluate its significance in aiding human technician to leverage his mechatronic skills.
3.2 Digital Twin
Although there have been a lot of studies on the concepts of digital twin in the industry, there have been few actual implementations to demonstrate their uses. There is an increasing trend of employing digital twin in industries at var- ious stages of Product Life Cycle. Digital twin was listed in the’Gartner Top 10 Strategic Technology Trends for 2018’ [41]. The report ’IDC FutureScape:
Worldwide IoT Predictions’ depicted that by 2020, 30% of G2000 companies will be using data from digital twins to significantly improve productivity [9].
Despite a surge in technological advancements and industry-wide discussions about CPS and DT, and the businesses’ interest, the technical adaptation and implementation of DTs are still under exploration[42]. According to the cate- gorical literature review conducted by Kritzinger et al. [19] in 2018, majority of the publications were classified as Digital Shadows (35%) and Digital Mod- els (28%). Although they used the term Digital Twin, only 18% of them were actually illustrating a digital twin with bidirectional communication. There was only one case-study which practically implemented and analyzed an ac- tual digital twin [43]. Over the following years, research works [44][45] con- tinue to use the term Digital Twin without actually illustrating one in reality.
And there are only a few case studies[46][47] on smaller parts focusing on specific aspects of DT development. With none of the literature covering all aspects of DT [48], let alone its implementation in CPPS.
As rightly pointed out by various authors, digital twin is greatly advantageous for applications in industry, but there is still a lack of case-studies which apply the concepts in practice [49][50][51][52][53]
"Research on Digital Twins is still at the beginning, there is a need for future research works on relevant industrial applications to investigate and demon- strate the wide range of applications and benefits where the DT could express their potential."[15]
Thus, there exists a tremendous need to practically implement and exploit the significance of digital twin for real production system.
3.3 User-Centric Visualizations
In an industry, there are different stakeholders present at different hierarchical levels, whose needs vary based on the work they perform. Thus, the data that would help them make informed decisions also varies, both in type and in de- tail. Therefore, there is a need to provide different visualizations to different
stakeholders in order to improve their understanding of the system and support their decision-making process [54]. Previous works do not focus on this par- ticular user-centric aspect of data visualization. There has been one work [54]
which has motivated the need to develop different visualizations based on dif- ferent stakeholder requirements for complex logistics operations of CPS. This thesis aims to extend this user-centric visualization concept to the proposed CPPS Digital Twin case study.
Chapter 4
Methodology
The methods, tools and techniques that were used during the planning and the realization phase of the thesis are described in the following chapter.
4.1 Project Management
During the start of the thesis, a planning report was prepared to guide us through. The report described the problem statement, the purpose, the time- plan and the potential risks involved in the thesis. This was essential in order to maintain track and avoid over-exceeding the scope of the thesis.
Through the project, multiple management strategies have been applied with various purposes. In order to plan the work, a Gantt chart was implemented as a means to control which tasks to prioritize and to divide the workload during the timeline of the project. Further, a waterfall methodology was im- plemented to provide an illustrative understanding of the sequential workflow, cross-system couplings and the gradual developments in the thesis.
4.2 Project Organisation
A project organization structure was created to facilitate the planning, coordi- nation and implementation of all project activities for a successful realization of the project. The organization was created in such a way that it generated an inclusive environment for the team members to easily communicate and contribute with minimum disruption.
19
Digital Twin development constitutes the major workload of the thesis in terms of both time and effort. To divide this workload, the thesis was split into two different subgroup tasks called the back-end tasks and the front-end tasks. The back-end sub tasks were performed by Aamir, which consisted of communi- cation with the physical model, data collection, integration and processing.
While the front-end sub tasks were performed by Sidhvin, which included communication with the virtual model, data visualization and simulation. All other tasks including literature study, designing and conducting interviews and experiments along with results analysis were performed jointly.
The methodology has been designed to answer the research question men- tioned in section 1.4. The workflow diagram in figure 4.1 explains the signif- icant sequential steps/events involved.
It consists of 3 main tasks performed in 2 separate stages described as follows:
1. Digital Twin creation (see 4.2.1) 2. Conducting Interviews (see 4.2.2)
3. Conducting a controlled experiment (see 4.2.3)
Figure 4.1: Methodology workflow diagram
4.2.1 Digital Twin Creation
Given the physical devices, in general, there are 6 steps that make up a digital twin, as explained below [55].
1. Connection : the connection between the physical and virtual environ- ment is established and tested for reliable communication. The network framework for connecting all the physical devices and the virtual model should be established. This involves investigation and testing of com- patible communication protocols across physical devices from various manufacturers and integrating them into the common virtual environ- ment.
2. Collection : data from the physical devices are tapped into virtual envi- ronment and tested.
3. Virtual model : virtual representations of the physical devices are cre- ated through 3D CAD design and modeling.
4. Integration and Processing : the data collected from different physical devices is integrated and processed for visualization.
5. Geometric simulation : the physical devices are simulated in virtual environment based on geometric motion data received from the physical devices.
6. Control : establish a two way real-time communication between the physical and virtual models forming a closed loop digital twin.
4.2.2 Interviews
For the purpose of identifying contextual data for user-centric visualizations and for designing an experimental scenario of a corrective maintenance task, semi-structured interviews [56] are used as a qualitative inquiry method. Semi structured interviews are conducted when there is sufficient objective knowl- edge but the subjective knowledge is lacking [57]. In this research, the outline of the interview, i.e. topics/issues to be covered are known while the detailed specifics of it are unknown, thus semi structured interviews are considered suitable. The interview questions would be aimed to understand the routine work of the identified stakeholders (Line Operator, Production Manager and Maintenance Engineer) and identify their specific needs for the purpose of de- veloping user-centric visualizations and probable maintenance scenarios. The interviews would essentially enlist the data that the stakeholders would like it
visualized. The identified data would then be extracted by the DT and pro- cessed to develop user-centric visualizations. The interviews would also pro- vide an insight into the activities of stakeholders which are then used to create scenarios for the controlled experiment.
Upon identifying the the stakeholder specific contextual data, the user-centric visualizations are developed based on a real time digital twin of the CPPS.
4.2.3 Controlled Experiment
Once the user-centric visualizations are developed based on the digital twin, the visualization specific to the corrective maintenance engineer is evaluated for its effectiveness in a corrective maintenance task. Controlled experiment strategy [58] can be used to evaluate this human centered task with respect to correctness and effort. The experiment scenario would be designed based on the interviews with corrective maintenance stakeholders. At least 10 partic- ipants are expected to take part in the experiment. The participants are ran- domized into two groups. One that performs the maintenance task with the visualization assistance provided by the DT and the other group that performs the maintenance task without the visualization assistance, but rather by using the conventional methods. The experiment would yield a correctness score and the time taken for each of the tasks. This data would then be subjected to ANOVA[59] to establish that the results are statistically significant. Fur- ther, a post experimental questionnaire will be conducted to collect empirical data through qualitative questions to further support the statistical results. The experimental results would ultimately lead to answering the research question.
Reliability and Validity of Study
Reliability refers to the consistency of the results produced. Inter-rater statisti- cal method would be used to estimate the reliability of this study. This method qualitatively determines the consistency of results when different users per- form the same experiment. Correctness scores which different users obtain during the course of the experiment would be used in assessing the consis- tency among them.
The validity of a study refers to how accurate and trustworthy are the results of the study. It includes two types: internal and external validity.
Internal validity refers to the extent to which the results of a study represent the truth within the context of study. It relates to how well a study is conducted.
The following measures would be adopted to ensure high internal validity of the study:
• Randomization: Randomly assigning participants to the two control groups to ensure there is no systematic bias between the groups.
• Study Protocol: Following a specific procedure that defines the extent of help, the extent of interaction with the participants, the scoring and tim- ing criteria and the manner in which the experiment is to be carried out.
This is to ensure that there is no differing treatment towards different participants.
External validity refers to the extent to which the results from a study can be generalized to other settings. In other words, how well do the findings apply to other people, population representation and research environment.
The following measures would be adopted to ensure high external validity of the study:
• Study environment: The DT based solution is to be designed to assist in the corrective maintenance of any mechatronic system. And the test subject selected would represent a typical mechatronic system. This ensures that the study could be extended to any mechatronic system from any industry.
• Population representation: Technical competence, maintenance approach, standard procedure and practices followed by the maintenance engineers could differ from one industry to another. This could produce varying results by the 2 control groups. However, since these differences do not exist between the participants of the 2 control groups from the same in- dustry, the net result (accuracy and efficiency change) which depends on the relative difference between the two groups would essentially remain the same.
Chapter 5
Implementation
In this chapter, the design and implementation of the system is explained and motivated. The aim of this chapter is to provide a comprehensive understand- ing of how different subsystems were designed and structured to build the user- centric visualization through a DT and evaluate its significance to ultimately answer the research question.
5.1 Digital Twin Creation
This section describes the DT use case comprising: the physical system, its virtual counterpart and their communication structure; followed by a detailed illustration of the steps involved in creating the DT.
5.1.1 Physical Model
The physical model consists of various industrial resources, which can be sum- marized as “human/machine/material/environment” The following physical devices are involved in the model:
25
ABB IRB-1200 robot
Figure 5.1: ABB IRB 1200 [60]
A 6 axis industrial robot with payload carrying capacity of 5 kg. It has a 3D camera and an electric 2 jaw parallel gripper attached to it. It is employed for Logistics application: to detect, pick and place objects.
Programming language: RAPID, for scripting custom programs for user de- sired applications.
Communication: RWS (REST), TCP (Ethernet IP), DeviceNet, Profinet, Mod- bus and OPC-UA.
Yaskawa HC20 robot
Figure 5.2: Yaskawa HC20
A 6 axis industrial collaborative robot with payload carrying capacity of 20 kg. It is attached with a pneumatic 2 jaw parallel gripper. It is employed for Assembly application: to pick and assembly objects.
Programming language: MotoPlus, for scripting custom programs for user desired applications.
Communication: TCP(Ethernet IP), ROS, Profinet, Modbus and OPC-UA.
5.1.2 Virtual Model
The virtual representation of the physical devices was built in Unity Game En- gine. It is a popular real-time development platform for 2D and 3D interactive experiences. It has its own built-in physics engine that manages physical sim- ulations of 3D models and other interactive development tools for real-time simulations and visualizations. It simplifies the development of an interactive platform by the ability to attach programmable C# scripts to 2D and 3D ’Game Objects’ by simple drag and drop. The Game Objects’ behaviour is thus di- rectly programmable through the C# scripts. It also has a wide variety of UI creation tools that makes creating interfaces a simple process.
5.1.3 Communication between Physical and Virtual Mod- els
Following are some of the most popular industrial communication protocols that were used to implement the communication structure of the digital twin.
• Ethernet IP: EtherNet/IP is an application-layer protocol built on top of TCP/IP. It uses Ethernet as standard physical layer and TCP/IP as trans- port and data link layer.
Since EtherNet/IP can use the standard Ethernet with switches, it can provide one-to-many connections allowing an unlimited number of nodes in a system, i.e. a single network across many different end points in a factory floor. It is widely used in Automation, Process and Control in- dustry.
• HTTP: Hypertext Transfer Protocol is an application layer protocol for transmitting hypermedia messages such as HTML. It provides half-duplex communication over a single TCP connection. HTTP handshake be- tween the client and the server closes the connection immediately after the request has been served. The most common HTTP request methods include GET (to retrieve data) and POST (to send data to the server).
Applications include accessing web pages on the internet.
• REST: Representational State Transfer (REST) is simply an architec- tural style for developing web based machine-to-machine communica- tion services. Any information that can be transported over HTTP pro- tocol is a resource. This can be documents, images or any other infor- mation. Resources are identified by uniform resource identifiers URIs.
REST is a client server architecture where the server and client evolve independently. The client only needs to know the URIs. It is these URIs that enable communication between the client and the server over the web. Applications are similar to HTTP.
• WebSocket: WebSocket is a computer communications protocol that provides a full duplex communication over a single TCP connection.
Unlike HTTP, the WebSocket handshake between the client and server opens a connection permanently until closed by any of the two. Thus fa- cilitating real time data transfer to and from the server. Real-time appli- cations include Social media feed, Web gaming, Collaborative content editing such as Overleaf, media streaming such as sports updates.
5.1.4 Process of DT Creation
As explained in the previous chapter: section 4.2.1, there are 6 distinct steps involved in the development of a Digital Twin.
1. Connection : the connection between the physical and virtual envi- ronments was established via Ethernet/IP protocol and tested for reli- able and stable communication. Both the physical system (network of physical devices) and the virtual system (hosting virtual models) were connected physically via standard Ethernet to the same factory network (see figure 5.3). They were assigned static IP addresses on the network to maintain steady network configuration facilitating automation. The connection was verified by a ping test between the systems.
Figure 5.3: Connections between physical and virtual environments
2. Collection : data from the physical devices was tapped into virtual en- vironment and tested. Programming scripts were written and run on the physical devices as background tasks, continuously extracting the re- quired data. These scripts send/receive data via Ethernet/IP built over TCP/IP protocol. The scripts were written in RAPID1 and MotoPlus2 application languages as supported by the ABB and Yaskawa robots re- spectively. The transmitted data was essentially the user-identified data
1RAPID is a high-level programming language used to control ABB industrial robots
2MotoPlus is a professional programming language used to control Motoman industrial robots
from the interviews which translated to Joint Position, Cartesian Posi- tion, Joint Velocity, Joint Torque, Robot Up time, Cycle time, Alarm Statuses, etc. The received data was regulated as desired by reworking the scripts and verified using TCP socket client/server applications such as SocketTest and Hercules. The scripts were then subjected to debug- ging until the desired data was displayed.
3. Virtual model : virtual representation of the physical devices was built in Unity. Based on the requirement 5, the digital twin was built upon the existing factory layout model. Two physical devices: ABB robot and Yaskawa robot were included in the virtual model, thus satisfying the requirement 4.
Figure 5.4: An sample of the Smart Factory Lab environment in Unity
In Unity, all entities are defined by fundamental objects called GameOb- jects. The GameObjects consist of components that define its function- ality. The developed virtual space is mainly composed of the following types of GameObjects :
• Environment : The Smart Factory building layout consisting of static 3D objects like walls, floor, lights, chairs, tables etc. forming the base structure of the virtual space. This can be seen in figure 5.4.
• Robot : The two robots (ABB IRB 1200 and Yaskawa HC20) and their mounts along with their work tables. The CAD files of the two robots in ’fbx’ format were imported into Unity such that each joint/link of the imported robot geometry behaves like a sub- GameObject whose behaviour can be controlled in 3D by attaching programmable C# script component. A closer look of the Yaskawa HC20 in Unity’s environment can be seen in figure 5.5.
• Interface : 2D Interface elements such as buttons, text boxes, im- ages, checkboxes, etc. are used to create an interactable and intu- itive UI. These elements are driven by the digital twin data and help constitute the front-end UI features for user-centric visualizations.
The subsection 5.2.1 explains this user interface in more detail.
• Camera : these elements provide view-points in 3 dimensional space such as differing factory viewpoints and individual worksta- tion views. The scripts attached to them control switching between the views as desired.
Figure 5.5: Yaskawa Motoman HC20
4. Integration and Processing : the extracted data from different physical devices was integrated and processed for visualization.
In order to make the whole system modular, a middle-ware integration
application ’Node-RED’ was employed between the physical devices and the virtual system. Node-RED3is an effective tool for managing/in- terfacing heterogeneous hardware and software environments. It uses Ethernet/IP and REST protocol to communicate with physical devices and WebSockets to communicate with the virtual system Unity (see fig- ure 5.6).
The transmission rate of the data sent to Unity was optimized by con- trolling the sampling rate at the physical device and by filtering in Node- RED. Too low transmission rate would result in a non real-time data transfer, which would consequently delay the behaviour of the robot in physical/virtual space. Too high transmission rate would increase com- putational load on the receiving system to execute predefined functions for each of those samples at a faster rate and would thus result in sig- nificant packet loss. Therefore, an optimal transmission rate was de- termined through iterative testing, which would facilitate real-time data transfer without significant packet loss.
Figure 5.6: Integration and Processing in middleware Node-RED
5. Geometric simulation : the simulation of physical devices in virtual space wherein the virtual model follows the physical model is accom- plished, i.e. one-way communication.
3Node-RED is an open source low level programming tool: https://nodered.org/
The joint data of the respective physical devices is transmitted to Unity (see figure 5.7). It is then translated as Euler angle rotations of the in- dividual joints (GameObjects) of the virtual robots via the attached C#
script components, producing geometric simulations in 3 dimensional virtual space. In addition to geometric simulation, the operating state, alarm state, error state and other motion properties of the physical robots is simulated.
Figure 5.7: One-way Physical->Virtual
6. Control : a two way real-time communication between the physical and virtual models was established, forming a closed loop digital twin.
The one-way communication from the physical to the virtual model was established in the previous step. For communication in the other di- rection (see figure 5.8), the user inputs from the interaction with the user-centric visualizations are translated into various system state and motion commands including angular rotations of the individual joints of the physical robots.
The bidirectional communication between the physical devices and the virtual system was tested to ensure reliable, error-free transmission with- out any data losses.
Figure 5.8: Other-way Virtual->Physical
The developed digital twin along with the user-centric visualizations were ver- ified that it satisfied the requirements listed out in section 1.5. In effect, the timing response in bidirectional communication between the real and virtual system was different by a mere 1 second.
5.2 Interviews
A total of 8 interviews and around 4-5 shop-floor visits were held in order to identify contextual data for visualizations and to get insights for designing a Controlled Experiment scenario.
First, a rough draft with rather broad interview questions was designed based on the purpose and aim of the interview. After a couple of loosely semi- structured interviews, the draft of interview questions was redefined to be more structured and specific based on the gathered inputs. At least 1 interview was conducted with each of the target user-group, i.e. Line Operator, Maintenance Engineer and Production Manager, thus satisfying the requirements 1 and 2 (see section 1.5).
The interviewees were made aware of their rights, the purpose and usage of the information that will be collected. The interviews and the shop-floor visits were audio recorded with due consent from the interviewees, then later sum- marized to derive key notes. After about 5-6 interviews, user-centric visualiza- tions were created and subjected to continuous improvements thereafter with the subsequent interviews. The interview sheet can be seen in the AppendixA.
The information gathered from any interview was cross-verified and validated with the information from all the other interviews. The accuracy of the consol-
idated findings were further confirmed by the relevant experienced personnel at Scania.
5.2.1 User-Centric Visualizations
The results from the interviews are presented in the next chapter. Based on these results, the maintenance technicians desired to visualize certain features.
These features can be seen in figure 5.9 which describes each feature in detail.
Figure 5.9: User Interface of the Maintenance Technician
The figure 5.9 is a representation of the maintenance technician’s UI. Various UI elements are numbered 1 through 12 and they are explained as follows :
1. Error Panel : displays the identified fault based on the fault classifi- cation performed on the list of error logs generated by the machine. It contains the error code and a short description of the error.
2. Ascription Panel : inspired from the fault-ascription concept from the DT-based approach explained in section 3.1.3, this panel enlists possible causes for the identified error and the corresponding corrective steps for each in the Action division. The Action division also includes an ad- ditional ’help’ button that directs the user to a detailed troubleshooting guidance from the documentation. Further, the frequency of occurrence of the error due to the individual causes is derived from the past mainte- nance history and displayed against each cause. This would essentially
provide some insights to the user about the frequently responsible cause and the corresponding maintenance performed in the past.
3. Up-Time Panel : the number of hours the machine has been up and running. It could provide data for scheduled maintenance operations.
4. Stop-Time Panel : the amount of time for which the production line has been stopped. Besides measuring mean time to repair, it could also help in the decision to elevate the situation and employ more technicians if necessary.
5. Navigation buttons : the ’Overview’ button shows a top down view of the factory as shown in figure 5.10. And the home button navigates the user to the user group selection menu.
6. Emergency Stop : this button immediately stops all tasks the machine is currently running.
7. Controller Status Panel : shows the current state of the machine’s con- troller.
8. Task Status Panel : displays the current task, its status (active or not) and the latest recorded cycle time of that task. The cycle times are stored in the back-end database.
9. Joint Data Panel : shows the real-time position, velocity and torque data of each of the 6 joints of the robot. The data can be viewed either as numbers or as bar graphs. The data is bounded within the operating limits of the respective robot. The window below it will indicate if the data for any of the joints exceeds its operating limits.
10. Jog Control Panel : this panel allows for jogging4each of the individual joints manually to a specific position.
11. Task Select Buttons : These represent skills that invoke changes in the state of the robot following the skill-based functional approach. The
’Test Run’ skill performs motion tests on all the joints of the robot, while the ’Production Task’ runs the real production task the robot performs which in case of ABB robot involves identification, picking and placing the objects. Upon triggering these skills, any deviations in the expected output state will result in error.
4the action of moving a machine part in small incremental steps
12. Documentation : this button opens a range of documentations and other resources pertaining to this robot.
Figure 5.10: A top down view of Smart Factory Lab
5.3 Controlled Experiment
A controlled experiment was designed to test the effect of using the Digital Twin based visualization solution in solving a corrective maintenance task.
The experimental scenario was inspired from some of the past corrective main- tenance experiences of the maintenance technicians who were interviewed.
5.3.1 Experiment Design
Since the solution was aimed at assisting corrective maintenance of mecha- tronic systems, the ABB industrial robot described earlier (5.1.1) was used as test subject in the experiment. Industrial robots are considered mecha- tronic systems since they combine the aspects of mechanics, electronics and software to perform desired functions [61][62]. The controlled experiment was designed to be reproducible consistently to ensure reliability. This was made possible through continuous improvements from conducting a number of dummy experiments with the engineers at Smart Factory Lab. For the real experiments, the candidates (test objects) were randomly split into two groups : one group would perform the corrective maintenance task with the help of
