MASTERTHESIS Adaptive sensor
Exploring the use of dynamic role allocation based on interestingness to detect blood and tumours in a smart pill
Can Yang
Embedded and Intelligent Systems, 30 credits
Halmstad University, August 27, 2018
s u p e r v i s o r s : Martin Cooney Håkan Pettersson
e x a m i n e r s : Antanas Verikas Slawomir Nowaczyk
l o c at i o n : Halmstad, Sweden
For intelligent systems, the ability to adapt a sensor’s sensing capa- bilities offers promise for reducing numbers, weight, and volume of sensors required. This basic idea is in line with a recent assertion by the well-known roboticist Rodney Brooks, that versatile robots could be used to perform various tasks instead of requiring a large number of specialized robots. In the current work, we consider the concept of a "smart" sensor which could dynamically adapt itself to replace multiple static sensors–within the application area of ingestible smart pills, where small sensors might be required to detect problems such as bleeding or tumours.
Simulations were used to evaluate some basic strategies for how to adapt the sensor and their effectiveness was compared; as well, a hardware prototype using LEDs to indicate system switching was prepared.
v
of neurological signal, directing us to fruitful areas of inquiry.
— B. F. Skinner[1]
A C K N O W L E D G E M E N T S
I would like to thank my thesis supervisors Martin Cooney and Håkan Pettersson for many fancy brainstorming and suggestions, providing me chance to bold imagination. Also, another thank you to Tommy Salomonsson who gave me ideas for solving problem in a clear way.
vii
1 i n t r o d u c t i o n 1 1.1 Background 1
1.2 Problem Definition 2 1.2.1 Challenges 2
1.2.2 Research questions 3 1.3 Motivation 3
1.4 Research Goals 4 1.5 Contribution 5 1.6 Thesis Outline 6 2 r e l at e d w o r k 7
2.0.1 Smart device 7 2.0.2 Adaptive system 9 2.0.3 Transition system 10 3 m e t h o d o l o g y 15
3.1 Theory 15
3.1.1 Definition of Interestingness 15
3.1.2 Reaching Interestingness - Electric Switching 16 3.1.3 Reaching Interestingness - Mechanical Switch-
ing 17 3.2 Hardware 19
3.2.1 Conductivity Calculation 19 3.2.2 Colour detection 21
3.3 Software 23
3.3.1 Data source 23
3.3.2 Basic math models 29 3.3.3 Model Applications 30 3.3.4 Evaluation Methods 30 4 e x p e r i m e n t 33
4.1 Hardware 33
4.1.1 Sensor Selection 33 4.1.2 Prototype of Switching 47 4.2 Software 53
4.2.1 Data Source 53
4.2.2 First System Switching With A Sequence 53 4.2.3 Second System With Single State 57
4.2.4 Third System With Flexible Switching 58 5 c o n c l u s i o n a n d d i s c u s s i o n 65
5.1 Conclusion 65 5.2 Future Work 67 b i b l i o g r a p h y 69
ix
Figure 1 System Overview. Both hardware and software are being explored. The self-made sensor is a sensor measuring conductivity which is con- structed by the author. 4
Figure 2 Contribution 5
Figure 3 sensor modular transforming 13 Figure 4 Sensor Family 17
Figure 5 Navigation Robots 18
Figure 6 Conversion among touch, tilt, range sensors 18 Figure 7 Figure 7 shows the skeleton of a device which
applies two electrical currents to double elec- trodes for measuring voltage. 20
Figure 8 Different markers represent blood in different solutions. In addition, due to the similarity of the colour of the solution, the green star mark overlaps the red star mark 21
Figure 9 Data Plotting in Colour space 23
Figure 10 The Human Digestive Tract pH Range Diagram 25 Figure 11 Potassium and PH Data set 25
Figure 12 10Different Patients Blood Detection 27 Figure 13 Special Samples 28
Figure 14 Outputs From FSR 29 Figure 15 Touch Sensor 34 Figure 16 Switch control LED 35 Figure 17 Transistor control LED 35 Figure 18 Conductance Measurement 37 Figure 19 Data Simple Analysis 38 Figure 20 Conductivity Measurement 40
Figure 21 Figure 21ais the distribution of current under different frequencies with various solutions.Fig- ure 21b is conductivity of these solution I cal- culated from current. These solutions contain different amounts of blood, lemon juice and vinegar 43
Figure 22 94HZ Conductivity with different blood amounts.
The x-axis is the blood density (percentage), and the y-axis is the conductivity (ms/cm.) 44 Figure 23 Colour Detecting 45
Figure 24 RGB Colour in RGB Mode Chart 46
Figure 25 Equipment 48
Figure 26 Mixture 49
x
Figure 27 Equipment 49
Figure 28 Results of Blood In Water 50 Figure 29 Results of Blood In Milk 51
Figure 30 Results of Blood In Calcium Tablets 52 Figure 31 The upper side of Figure 31ais current sensor
state, The lower side of Figure 31a is interest- ingness chart that system predicted. The right of Figure 31b is the raw output from system.
Same meaning with the other four more fig- ures 55
Figure 32 Simple System with Different Switching points.
Different colours present different changing speeds and x axis is different noise length or switching delay for each curve 56
Figure 33 The upper part of Figure 33a is sensor state and lower is interestingness chart. Figure 33a is the raw date detected from this system. And Figure 33bis same. 58
Figure 34 Sensor output and STD values 59 Figure 35 Smart System Evaluation 60
Figure 36 Accuracy and Matthews Coefficient 63 Figure 37 Accuracy and Matthews Coefficient 63
Table 1 Possitua Value 24 Table 2 Nacl Conductivity 41
Table 3 Calibration solution Conductivity 41 Table 4 Lemon Juice and Vinegar Current 41 Table 5 Current of blood measurement 42 Table 6 Blood Colour Measurement 47 Table 8 Evaluation Chart 2 54
Table 7 Evaluation Chart 1 54
Table 9 System evaluation starts from blood sensor with different changing duration. 61
Table 10 System evaluation starts from tumour sensor with different changing duration. 61
Table 11 Smart Sensor Accuracy of 20 Different Cases 62 Table 12 Simple Sensor Accuracy of 20 Different Cases 62
xii
xiii
STD Standard deviation
AMP Unit of applied voltage
xiv
1
I N T R O D U C T I O N
1.1 b a c k g r o u n d
Cancer is a major public-health problem worldwide causing high morbidity and mortality. More than 30% of all people will develop some form of cancer during their lifetime [2]. Cancer is the second leading cause of death in the United States, and is expected to sur- pass heart diseases as the leading cause of death in the next few years [3].
Diagnosis is the first and most important step in cancer therapy.
The symptoms and signs of cancer are currently detected by tedious and energy-consuming health checkups, such as imaging with fluo- rescence and ultravioletâvisible spectroscopy (UV-vi) absorption meth- ods. While the search for new strategies for cancer therapy continues, it becomes increasingly possible that employing advanced healthcare technologies could help early detection of cancer and precise localisa- tion of cancer.
Healthcare intelligence has led to ambitious visions of how an in- telligent system can serve doctors and patients. An example of an in- telligent system is a machine with an embedded, internet-connected computer which has the capacity to gather and analyse data and com- municate with other systems.
Incorporating smart sensor technology is one way in which health- care techniques could be revolutionised. One example is a sensor sys- tem that can perform different reactions by perceiving changes or un- usual signals in the environment, such as a smart pill. In this thesis it is proposed that smart pills can be used to detect cancer symptoms such as internal bleeding and tumour growths.
The term smart sensor has been used in different ways in prior works. One kind of smart sensor is a sensor which also has on-board processing, to extract and efficiently send low noise-rate signals from large amounts of data [4]. Another definition of a smart sensor is a component of the Internet of Things, which can be a single node of a tremendously large network using wireless communication [5]. This aims to monitor and control a system for household electrical appli- ances.
1
To obtain accurate outputs from smart sensors the data needs to be gathered, integrated, and assessed using decision-making methods in intelligent systems such as rules used in expert systems. An expert system is a computer system that emulates the ability of a human expert to solve complex problems with knowledge-based facts and relevant rules.
Smart pills using such intelligent methods could improve the speed and accuracy of a doctor’s diagnosis.
In this thesis the term "smart" is aligned with the definition in Smart Sensor Technologies and Signal Processing [6]: “structures which are able to sense and respond or adapt to changes in their environment are often referred to as smart”.
Furthermore, in this work, "interestingness" is defined as a measure intended for selecting and ranking sensor signals to diagnose cancer based on how informative they might be.
1.2 p r o b l e m d e f i n i t i o n 1.2.1 Challenges
Traditional smart sensor systems have several problems. As a result of following an expert systems approach, recognition for smart sys- tems can be rigid and not good at generalisation. For example, some characteristics can be detected if the system has a specific definition or knowledge base, but in this case, some other general features of the problem cannot be recognised, which leads the system to have bias.
Moreover, in order to obtain accurate output, multiple devices or sensors might be required. These might be high-cost and too many sensors can lead to problems including too much space being occu- pied, a higher chance of a component breaking, and integration of the data.
In this project we investigate the use of one adaptive sensor system as a solution intended to be used in place of several traditional single- functional sensors. Adaptive or flexible sensor systems with the capa- bility of activity recognition offer opportunities for smart pills. Being
"adaptive" means here that a sensor can perform different reactions by perceiving changes or unusual signal in the environment (human body). This would reduce the numbers, weight, and volume of sen- sors required for ingestible smart pills. At the same time, it might
save cost compared to using multiple static sensors.
More specifically, we are considering a flexible sensor with two functionalities, one which can detect bleeding, and one which can detect bulges linked to tumours. The adaptive sensor would switch states, turning into one sensor, then turning into another, as needed.
The main focus in this research is to explore the theoretical per- formance of such a sensor system in various cases. First, the degree to which a signal is interesting is judged in terms of definitions for interestingness. Afterwards, a decision-making theory about when to switch and where to switch is presented. Those systems can be eval- uated by detection accuracy given some ground truth.
Additional challenges associated with smart pills which are not the focus of this thesis are: whether they are small enough to allow ingestion; are there risks to retention in body for a long time; and looking for unmet clinical need associated with the technological so- lution. This could be addressed in future work.
1.2.2 Research questions
Based on the challenges of the current work, the research question is:
how could an adaptive sensor use dynamic role allocation based on interestingness to detect various stimuli, in a healthcare application involving a smart pill for detecting cancer symptoms?
1.3 m o t i vat i o n
Sensor technology is becoming more mature and intelligent systems are developing at rapid rates as well. However, there are not many joint studies in these two areas. Current sensors are "static", detecting just one thing, such as light, sound, or touch. Because of this, in com- plex systems such as vehicles or homes, many sensors are typically required. A "smart" sensor which could dynamically adapt itself (to detect different signals) could replace multiple static sensors, leading to less space taken, less cost, less work for installations, a greater abil- ity to operate when changes occur, and possibly even easier repairs (self-healing). Alternatively, an ensemble of dynamic sensors could acquire more information than the same number of static sensors.
Moreover, being adaptability will provide more possibilities for ap- plications. First, an object can be measured in different ways with sensor fusion, more results can make detection more reliable and ro- bust. As a second benefit, one sensor configuration can be used as an
alternative when another is broken to extend the life cycle of the sen- sor, especially for some fragile devices or robots intended for single use in a dangerous environment that will be discarded after perform- ing a task. Also, when the sensor is used as a pill inside a human body, quality assurance and robustness are particularly important, so adaptive attributes can be considered to be fail-safe contingencies.
Furthermore, if such sensors can be made small and lightweight, various benefits would emerge: e.g., making arbitrary surfaces em- bedded with sensors for robots such as intelligent skin, and interest- ing tasks at the micro-scale could potentially be facilitated, such as monitoring cells inside of a person.
1.4 r e s e a r c h g oa l s
This research aims to explore a small “smart sensor system” which can adapt itself and monitor its environment, based on measured interestingness and dynamic role allocation, for a healthcare appli- cation of a smart pill to recognise biometrics. There is uncertainty about the theory, software, and hardware. Therefore these aspects are investigated.
Figure 1: System Overview. Both hardware and software are being explored.
The self-made sensor is a sensor measuring conductivity which is constructed by the author.
Theory
A core step is acquiring a definition of interestingness. From a range of different understandings of what makes a signal interesting such as salience and relevance, standard deviation and information entropy are used for analysis. Besides, some design transitions with morphol- ogy changing are considered for future work.
Software
A series of simulation data is generated. Starting from a simple data set, more cases are then added in order to indicate the system’s ro- bustness. Furthermore, three system solutions are proposed to detect an interesting signal with various data sets. An evaluation and con- clusion are conducted at the end.
Hardware
Four sensors are tested to understand the working principle, which are shown in the left side of Figure 1. This work focuses on the fea- tures to detect blood and tumours. A switching strategy is shown by a demo with LEDs.
1.5 c o n t r i b u t i o n
Most of the previous work in this area focuses on either the integra- tion of different sensors or big data analysis after data are obtained.
This project combines the areas of intelligent algorithms and sensor systems. The contribution of this thesis is exploring the concept of a smart sensor which can autonomously change its state to seek to achieve good performance toward some target in the presence of some uncertainty, in a healthcare application with a smart pill, from the perspectives of theory, software, and hardware.
Figure 2: Contribution
Based on the research questions which is exploring an adaptive sensor using dynamic role allocation with interestingness to detect various stimulus and applying for detecting cancer symptoms as a smart pill, the main contributions (Figure 2) are:
1. Theory: A general concept of interestingness is defined, along with features that can be used to track an interesting signal. In
order to observe the signal, a data selection methods is using to decide the usage data size. A dynamic moving window is collecting the series data, which is also adjusting the data gen- erating speed. The standard deviation is calculated for the data within this window. The values of the threshold of interesting- ness is proposing to set the class of interesting or uninteresting.
2. Theory: A role allocation methods which is also called a switch- ing algorithm is proposed, whose performance is evaluated com- pared to another simple algorithm. By employing various switch- ing rules based on the defined interestingness to gather as more symptoms as possible for tracking the patients inner environ- ments as a smart pill in this medical applications. Different in- ner environments from different patients are prepared to verify the performance of the role allocation methods.
3. Software: The concept of being adaptive to the stimulus are proving by the implementation in the software. It is discussed that, instead of staying in a static mode or integrating tons of sensors, switching among various sensor categories has more potential. More flexible states, real-time judgement with selec- tion data can provide a new simple way to deal with multiple features surrounding.
4. Hardware: A solution of blood detection is investigated in this work. A mechanism for detecting blood is presented. Putting a self-made sensor system in a black box to measure conductivity in different simulation solutions, which are milk, water and acid solutions, is for detecting blood. In addition, with those same solution, a simple colour recognition with a flashing light has been implemented and the data is processed in MATLAB to find blood.
5. Hardware: A prototype of blood detection is presented to show functionality by LED lights. According to the implementation of blood detection with colour, a red lights shows positive which is with blood and green is for no blood in the solution.
1.6 t h e s i s o u t l i n e
Chapter 2 describes some previous work that has been done related to both sensor systems and measuring interestingness. Chapter 3 de- scribes the methodology that has been applied during the whole project. An experiment and the results are discussed in Chapter 4.
Chapter 5 presents conclusions and discussions about future work.
2
R E L AT E D W O R K
The development of smart medical devices is surging in the health- care industry. Because of advances in miniaturisation, wireless con- nectivity and so on, smart services are being taken into consideration for a range of products. Being adaptive and flexible can reduce re- source requirements and create a customizable and integrated system to meet requirements. Sensors which are not needed can temporarily hibernate; i.e., a device can switch among different types of sensors.
This thesis is mainly in the area of intelligent systems, and relates to multiple other disciplines: health technology, smart technology, dy- namic changing and interestingness measurement.
2.0.1 Smart device
Smart devices are used in diagnostics, monitoring, and therapeutic treatments in order to change and save human lives. Improving sen- sor technologies can provide people with more comprehensive and specific information. Some work on sensors seeks to improve the out- put signal quality by reducing noise. Building a network with wire- less techniques to communicate between different nodes is another way to apply smart sensor techniques.
There are some efforts to bring in smart medical devices with sen- sor techniques in place of traditional medical technologies. Some high-cost medical devices are being introduced into the health care arena in recent years, where new techniques pose less risk to dam- aged tissues and human organs. Furthermore, combining outputs from different conventional sensors at the same time is another idea which has been proposed: this is referred to as a multi-functional sen- sor system.
Also, Ambient-Assisted Living (AAL) is one technology which seeks to use wearable sensors and ambient intelligence to provide reminders or monitor for human health. An essential aspect of AAL system de- sign should be to consider a device’s battery size without compro- mising on time and human body compatibility [7]. Human activity recognition (HAR) has become one important application of wearable sensors in AAL. Related to medical, military and security areas, this is a ubiquitous way to monitor people’s body status. At the same time, one aim of the current work in wearable sensors is on reliable and
7
multi-functional systems. In order to improve the comfort of people for long-time wearing, another challenge is how to develop minimally obtrusive technology [8], which can be done by optimising shape and ensuring light weight. Nevertheless, for wearable sensors, it is not easy for people to wear them every day and utility of accessories is limited. One way to enhance convenience and practicality involves embedding sensors in fabric by reducing the size of the sensors.
Many public and private health systems face pressure to reduce costs, in addition to aiming to achieve high-value personal care. For instance, in the early years when the concept of the “Smart Hospital”
was introduced, scientists spent approximately USD $300 million to generate a human genome sequence [9]. Also, da Vinci surgery robots cost $2.3 million for the Division of Pediatric Urology in the Depart- ment of Surgery [10].
Recently some work has begun to explore multi-functional sensors, for achieving high-efficiency sensing compared to traditional single- functional sensors. For instance, one system [11] can simultaneously detect strain and pressure, which utilizes the characteristic of high conductivity to monitor finger and knee motions. In another study, two types of multi-function ceramic sensors, humidity-gas ceramic sensors and temperature-humidity ceramic sensors, have been devel- oped, which detect the changes of parameters rapidly and simulta- neously in the environment [12]. Thus multifunction sensors, which gather all data which can be obtained and elect some optimised fea- tures, have some weaknesses: wasted energy costs, time-consumption for processing and extra data to store. A better solution is maybe to select what data to process and store using some criteria for “interest- ingness”.
Therefore, based on the various definitions of smart sensor systems, and considering the price of operating equipment and multi-sensor integrated systems, we hope to suggest the benefits of adaptive, low cost and power saving for medical technology. Smart here not only means that it can communicate with other nodes but also that a sen- sor can identify the environment. Moreover, this does not mean that high prices and advanced sensors must always be used. The bene- fit of a multi-sensor system is that we can collect different sources of information at the same time, but for simple tasks or at specific times throughout the process, some information may be redundant and may waste some energy in this way. The current work seeks to extend approaches to obtaining multiple data sets to increase the ver- satility of sensors; also, if one sensor can be adapted instead of using a big one that can measure two things, then a solution can be smaller
and lighter.
2.0.2 Adaptive system
Being adaptive and perceptive is another way to define a smart de- vice. Prediction can involve using methods such as Cognitive Analyt- ics, decision-making theory and interestingness measurements such as information entropy; for example, in regard to how predictable data are. Low entropy data are predictable and contain little new in- formation, so they might not be useful; high entropy data contain more new events. This can help the system decide what to do next.
In order to identify the surrounding environment, some decision- making methods have been developed in the medical diagnostic field.
Heart disease in Tohid Hospital in Sanandaj is predicted using a rule- based fuzzy system for increasing forecast accuracy [13]. Decision making is based on knowledge, sophisticated systems, and consul- tation with the medical team advisor. After data extraction and pre- processing, there are seven risk factors involved in heart disease as input for the fuzzy system, which results in an output of “yes” or
“no”. Other smart algorithms are related to the field of artificial intel- ligence. For example, one automatic diagnosis system for detecting breast cancer uses association rules and a neural network [14]. Fea- ture extraction is the first stage of processing in this study, which plays a vital role in the whole project. These decision algorithms pro- vide a robust system with high precision results. But for classification tasks, prior and specific knowledge is needed to identify the target class. Knowledge acquisition is one of the common shortcomings, and a large amount of basic knowledge or experience of people or experts to encode is a challenge. Also, firm reliance on selected fea- tures can be another risk.
One work making use of objective measures looks at the automatic detection of computer mouse activity in online videos [15], using the visual features that can be extracted from the online video to compute real-valued interestingness scores and binarize the real-valued scores to label activity as interesting or not. A semantic measure is another kind of interestingness measure; utility and actionability are exam- ples of semantic measures. One example of a robust method from previous work[16] involves combining three cues, emotion, complex- ity and novelty, with a fourth cue, which is the predicted interest- ingness result from applying a learning technique. Support vector regression is used to generate a model of interestingness scores that is normalised with mean and variance once these are obtained. Then a simple linear model is used to combine these cues. Cues are less
in this study, and it is better to consider semantic cues and context relation as well. Except for a single content-based method above, an- other study works out interestingness measures by association rules.
Twenty-two objective interestingness measures were assessed, ignor- ing the non-interesting branches[17]. The challenge with this work is that the result is not ideal when dealing with low-level pixels and complex objects.
Information theory as one kind of interestingness measure is also more focused on images. Information entropy is a mathematical the- ory provided by Shannon[18] to analyse the complexity of an even- t/signal from its uncertainty with this equation as proved by [18].
H(X) = X∞ i=1
pilogpi
For instance,the importance of feature metrics is measured by com- puting the entropy weighting of pattern recognition information[19].
Thus, much of the research in interestingness measures focuses on im- age processing, and not on other signals, whereas the current work will attempt to explore other signals with more sparse data, like light, temperature, and PH.
Much work has been done with role allocation for agents. Dempster- Shafer is one related approach, which is an extension of Bayesian the- ory that gathers uncertainty from a variety of sensor sources, to sup- port decision-making. Compared to the Bayes approach that works on a singlelevel of abstraction, the result can be enhanced by fusing the output from multiple sensors[20].
As a result, previous work has not explored how to build an adap- tive sensor which shifts its configuration for acquiring data based on the estimated interestingness of a signal, toward reducing cost and weight and unneeded data. Therefore, the contribution of this thesis is to present a design for such an adjustable sensor integration system.
Signal analysis, which can focus on a binary signal, uses a weighted method based on information entropy. When a signal becomes less interesting, decision-making based on Dempster-Shafer theory will trigger a dynamic transaction to another type of sensor.
2.0.3 Transition system 2.0.3.1 Chemical Solution
This thesis considers the case of a smart pill. One factor which should be considered for a smart pill is the material used, as such pills usu- ally pass through the human digestive system. Such biocompatibility
is a challenge for systems like Pillcam. Pillcam is one instance of a minature video camera which transmits pictures to a data recorder which a person wears on a belt [21] Swallowable Wireless Capsule is another similar application, which uses a low image resolution [22].
If the image is not the only data to be the inputs, other functions to collect data will be a way to compensate.
Edible circuits have the same disadvantage. These sensors are usu- ally static and can only be used for one thing. One work described how to make an edible pH sensor by taking into account capaci- tance changing between Au and ZnO electrodes[23]. Another work described how to make some edible wireless sensors which can be used to sense mechanical strain wirelessly [24].
Another factor to consider is how a sensor could morphologically adapt. Some related theoretical proposals have been made using the terms "programmable materials", "utility fog", and "claytronics". Pro- grammable materials are a kind of smart dynamic materials that re- spond and adapt to environments in form and function. For example, self-transforming carbon fiber, printed wood grain, or custom textile composites [25]. Programmable materials give insight into the com- bination of materials and machines. Incorporating with robotics or other devices into materials can provide a way to program the physi- cal world to assemble itself and transform on its own. In this way, we can acquire another solution for adaptive morphology, by program- ming qualities like shape, stiffness, optical characteristics, acoustic characteristics, and viscosity. So far, this technique has been applied to robot self-reconfiguration projects. For instance, a serpentine robot which can cross a tunnel, and then a rugged area by reconfiguring as a six-legged robot [26]. Moreover, Claytronics is another example of programmable matter which scales to numerous micro-scale ele- ments like "catoms", intended to provide a modular theoretical basis for robotic systems and realize long-distance communication ("pario") and "synthetic reality" [27].
The downside is that such theoretical proposals are currently chal- lenging to put into practice. Also, due to the need for specialised programming to achieve reasonable theoretical performance, not all applications can be explored [28].
2.0.3.2 Mechanical Solution
In order to achieve system transitions, mechanical motion is one solu- tion besides considering chemical aspects. Some very few papers have also started to explore sensors whose morphology can be changed dy- namically. An example of an adjustable sensor configuration is a ther- moplastic adhesive material that uses a Hot Melt Adhesive (HMA).
This mixed material has different states depending on various tem- peratures so that the shape can be reconfigured according to the soft- ening point and the melting point[29]. Another solution to trigger state changing is suggested by an experiment developed by the Mi- crosoft Research Institute in Cambridge named SenseCam which con- tains some different electronic sensors, like lightintensity and light- colour sensors, a passive infrared detector, a temperature sensor, and a multiple-axis accelerometer. The signal that triggers phototaking is the change of body heat or light-level [30]. However, this method is sequential in one direction and irreversible. Only when the transition between sensors is speedy, this flaw can be ignored.
Utilising single function sensors in a different way is another way to make a simple system functionally efficient. Besides the basic func- tion of the sensor, some derived functions are also good to consider.
For instance, touch can be sensed not just with a touch sensor. The touch sensor shown in Figure3a is designed to detect the amount of light reflected from a surface at varying distance by a photo-interrupter to measure the change of force or velocity [31]. Usually, a Tilt sensor can detect the changing of orientation or inclination. But shown in Figure 3b is another example [32], of a magnetic core with perma- nent magnets on both ends covered with a layer of magnetic fluid.
The output sensing signal is proportionally changed by the distance between two repulsion magnets with tilting. Moreover, a range sen- sor normally measures the depth to the nearest surfaces, e.g., using time of flight. There is a work using 2D laser range sensor shown in Figure 3c measured by amplitude modulation of light waves and difference of phases between transmitted and the received one [33].
A common feature of these three sensors is that calibration is needed.
But discovering more ways to use single function sensors for other purposes is another mindset which could allow function transitions.
(a) Touch Sensor[31]
(b) Tilt Sensor[32] (c) Range Sensor[33] Figure 3: sensor modular transforming
In summary, to build a dynamic transition system, motion can be a way to trigger the transformation. Various studies were discussed, including a method to adjust sensor states by shape changing due to different melting points and a one direction switching device with light-intensity and temperature. Three standard sensors and exam- ples of how to use to them in a different way for detection were dis- cussed.
3
M E T H O D O L O G Y
To explore the concept of the smart sensor, the approach taken in this work was to explore three aspects: theory, hardware and software. For hardware, this involved using a simple sensor to detect blood with a switching strategy. For theory, a definition of what is an interesting signal is introduced. For software, a simulation experiment was done in Python, which was detected the interestingness of a signal on three systems.
3.1 t h e o r y
This research explores a concept which has not yet been realized in practice. Therefore, theory is one of the essential and fundamental parts of the entire project. In particular the part about interestingness is a cornerstone of the study. First, the definition of interestingness is proposed. Some mathematical models will be used later based on this motivation. Second, in order to reach or maintain an interesting mode, electrical switching and mechanical switching are the two solutions to this goal. The analysis of the electrical switch is given, and some mechanical switch designs are illustrated.
3.1.1 Definition of Interestingness
Interestingness is a key to this research. One problem in studying in- terestingness is how to define interestingness. To explain this word,
"interesting" can be understood as seductive or memorable content, so it can refer to a moment that is able to capture attention in a short time or a moment that can be remembered despite the presence of a huge amount of other information.
Similarly, the Cambridge dictionary states that “someone or some- thing that is interesting keeps attention because of he, she, or it is unusual, exciting, or has a lot of ideas” [34].
Therefore it can be regarded as an indicator of where to give atten- tion, or how to choose or pick information. So an interesting signal could be during a time frame which has some changing which is not easy to explain by people, or a point where an experimental result is not the same as the expected result. Compared to classification or identification, general interestingness could be useful to explore as it does not require subjective decisions from a human about what
15
classes to use.
1. Variance. Changes can indicate something is happening or ac- tive. So when a pattern does not change, it is regarded as a dull signal.
2. Motifs and Anomalies. If a part of a signal appears multiple times, the system can detect such "motifs". Moreover, anoma- lies, it also can be surprisingness or novelty, which means that suddenly there are some strangely different patterns.
3. Peculiarity. Some signals are clear and do not contain multi- ple meanings or possibilities. The difference compared to the anomaly is that this feature has less ambiguity. The distance to other patterns are big, so it is easy to make an accurate decision.
4. Conciseness. Monotonous high mode. In other words, it con- tains fewer categories or topics. This type of feature will be eas- ier to remember or classify. Conversely, documents containing too many different types of signals are tedious.
3.1.2 Reaching Interestingness - Electric Switching
This sensor system is supposed to adapt to an uncertain environment with switching. Tracking interesting signals is the main task in this whole project. Identifying interestingness and tracking expected sig- nals are two sub-goals. Switching on and off the mode of the electrical circuit is one way to control the sensor system. The advantages and disadvantages of electricity switching are analysed here.
Electricity Circuit Disadvantages
1. The speed of electrical signals is slower than the speed of light, so using optical connections will be a little faster.
2. Changing current will generate a magnetic field. Some magnetic fields are bad for people’s health, especially inside the human body.
3. There is a problem with environmental pollution. Some forms of electrical energy production are dirty.
4. Many cables are required, which occupied a lot of space in the circuit.
Electricity Advantages
1. Electricity is more stable than mechanical systems since the com- ponents don’t move, so there is less need to worry about hard- ware connections.
2. Low-cost
3. Hundreds of states will not be a problem.
3.1.3 Reaching Interestingness - Mechanical Switching
In addition to electrical switches, mechanical switches are another option for this system. The goal here is to attempt to avoid the dis- advantages of mechanical conversion and explore the feasibility of Mechanical Switching. For this, several prototypes were designed for a potential application. It brings more specific areas to the subject for in-depth research.
Sensor Family
Figure 4: Sensor Family
A fascinating consideration in this project is the possibility to inte- grate thousands of sensors into a small size system. In order to re- duce unnecessary and erroneous operations, several sensor categories could be created based on functional similarity (Figure 4). Then, the transformation can be achieved by checking the table list or the root category.
If it is known which kind of sensor configuration yields dull infor- mation, in the next stage the sensor can selected from another sensor configuration family or group. Furthermore, the ranking sequence de- pends on the similarity between the current group and target group.
This is one way to implement the switching operation.
Navigation Robots
Figure 5: Navigation Robots
Another way to change system functionality is to treat each circuit component as a separate robot. An approach could involve moving the component in an electric circuit to recreate a new circuit which can provide other capabilities (Figure 5). For instance, the distance be- tween the two units of the capacitor can be varied to substitute other components equivalently. Thus, several navigation robots which act as the electric components of a circuit could be used to form differ- ent sensors; i.e., a merging of circuit theory and robot path-planning provides a method for sensor transformation.
Conversion among Specific Sensors
Figure 6: Conversion among touch, tilt, range sensors
Some specific sensors have a common structure and moving a part of them can finish the conversion. One example is considered. Tilt sen- sors, touch sensors and cameras are ubiquitous and powerful sensors.
The measurements from these three sensors could fit some require- ments. Figure 6 shows the transition ideas. A "baffle" can be moved flexibly. When the baffle is free to move over the entire area, the cur- rent position of the baffle can report the angle of inclination of the
sensor. Second, when the shutter is fixed to one side, the sensor can be used to detect touch. In addition, if the "bezel" is removed, the camera can work in this case when one side is open.
Thus, in terms of structural formation, morphological changes are a way of switching between sensor states.
Bionics: Jellyfish prototype
Bionics is becoming more and more popular, and the morphological characteristics of animals could also be used in certain projects. An- other example is considered. The body of a jellyfish could provide a large space for different sensors, where the jellyfish’s manubrium could be attached to a touch sensor to detect tumours. Some soft robotics are inspired [35] by jellyfish. So based on their efforts, such designs could be adapted for smart medicine.
3.2 h a r d wa r e
Regarding the specific application which this thesis considers, blood is one of the targets to be detected, since it could be useful in cancer detection. Two main features that are common to distinguish blood are colour and presence of metallic elements like iron. Detecting con- ductivity and red colour are the goals of this hardware aspect of the thesis.
3.2.1 Conductivity Calculation
In order to design a simple device to measure liquid conductivity, it is necessary to understand the method to compute conductivity. Con- ductivity measurement is a low-cost judgement of ionic strength in any solution, which can be used to detect the presence of blood cells because biological blood contains different metal elements.
Figure 7:Figure 7shows the skeleton of a device which applies two electri- cal currents to double electrodes for measuring voltage.
Resistance Calculation
The impedance of a test object (such as an amount of blood) can be measured by applying a voltage (V) and measuring the current (I) through the object. The impedance (Z) is then obtained from Ohm’s law V = Z ∗ I, where V is the voltage in Volts and I the current in Amperes. By taking into account the geometrical shape of the test object, the real part of impedance, which is a pure material constant, can be inferred. Measuring the conductivity of an ionic solution in a test cell as shown inFigure 7is more complicated, and cell correction factors need to be taken into account.
Conductance Calculation
Conductance(G) is the reciprocal of the impedance(Z) (Equation1).
G = Z1 (1)
Cell constant Calculation
The effective area of two electrodes and distance between two elec- trodes are also important factors (Equation2).
Kcc = ad (2)
a is the effective area of two electrodes.
d is the distance between two electrodes.
K_cc is a parameter which is cell constant.
Conductivity Calculation
Conductivity is the capability of a solution to pass electron ions [36] (Equation3).
K = G∗ Kcc (3)
K = conductivity (S/cm) G = conductance (S) K_cc = cell constant (cm-1)
Calibration
The goal here was to find a matched cell constant with the current situation. A standard solution is used for calibration. So a conductiv- ity table can be checked under different temperatures. In this way, a precise cell constant(K_cc) value is obtained. With these parameters an alternating current with different frequency was measured. There- fore, it could be checked if conductivity is a proper way to detect blood.
3.2.2 Colour detection
Another way to identify blood is colour segmentation. The inside of a human body will be dark. Therefore the sensor should have an LED that provides light. In this way different intensities of red could be measured, and the results could activate system actions. R, G, B (red, green, blue) values of RGB colour space can be obtained from a colour sensor. Red has long wavelengths which is 564-580 nm [37] [38]. The three dimensions are compressed into two dimensions. For this reason, there are only three elements, two of which can represent the total. Moreover, these two can be picked randomly. Red and blue are chosen and shown in 2d dimension. Two methods are designed and tested, which is Classify colour regions method and Fitting line method.
Figure 8: Different markers represent blood in different solutions. In addi- tion, due to the similarity of the colour of the solution, the green star mark overlaps the red star mark
Classify colour regions
In order to detect blood, different cases are considered, so intensities of red are plotted inFigure 8. Different categories of data can be easily divided by lines. Therefore, the blue line serves as a threshold for determining the presence or absence of blood. This way is quite easy and fast reactive to provide a prototype with the later steps. As shown in theFigure 8, the upper part of the line is the case containing blood.
The lower part of the line is the opposite, meaning that no blood is present. Therefore, this method of colour recognition is implemented with LED to show system prototype.
Fitting Line
Curve fitting is a tool for fitting all scatter points in a predicted plane or line. By the curvature of the plane, the boundary between the inner and outer regions can be drawn. Any position outside the carpet can be recognised as outliers or failure. Then, a linear polynomial model is obtained in equation 4:
fitresult(x, y) = p00 + p10 ∗ x + p01 ∗ y + p20 ∗ x2+ p11∗ x ∗ y + p02 ∗ y2 (4) Coefficients (with 95% confidence bounds): p00 = 538.4, p10 = 0.1644, p01 = 0.2562, p20 = -1.292e-05, p11 = 4.771e-05, p02 = -2.281e-05.
Figure 9ashows the distribution of blood colour in pure water, milk and Calcium tablets in 3 dimensions.The stars represent the absence of blood, and the circle the presence of blood. And all round blue dots are specific on the plane and the stars are outside the plane(Figure 9b).
(a) Data distribution in RGB dimensions
(b) Fitting Curve
Figure 9: Data Plotting in Colour space
3.3 s o f t wa r e 3.3.1 Data source
Before the experiment procedure, a data source is demonstrated. There are two data sets, one from the experiment results obtained in this work from the hardware section and the other from the experimental record. From the research paper, some data were used to build the algorithm environments simulated as different patients.
3.3.1.1 Potassium and PH
The application of this smart medicine is to detect blood and tumours.
Potassium and PH values are chosen according to research as two fea- tures which can use to assess problems.
Potassium: Blood
There are electrolytes in the human body, and the balance of elec- trolytes is vital to our health. Na+, K+, Cl- are important electrolytes in our body. The data shown in Table 1 explained that Na+, Cl-, K+
have a different amount in blood from the other fluids of the human body. In order to get an electrolyte as a feature, there is a compari- son of these electrolytes. For the current work, potassium in blood is an desirable feature to be observed in comparison to the amount of blood in other solutions like saliva, gastric juice, and so on. Moreover, potassium ion is the main electrolyte of intracellular fluid, account- ing for 98% of the total amount of internal liquid cations [? ]. From the potassium amount in the blood, acute or chronic kidney failure and other diseases can be indicated. Thus, doctors can get more in- formation from a blood sample with potassium level for diagnosing or inferring disease as well. So potassium is selected to detect blood.
NA+ K+ CL-
Blood 140 3.5 104
Saliva 10- 40 26 10- 30
Gastric Juice 20 10 - 20 150 Pancreatic Juice 140 5 40 Intestinal fluid 140 5- 15 50- 110
Table 1: Possitua Value
PH: Tumour
PH value measurement is a proper way to detect tumours. In the metabolic environment a potential difference in PH between solid tumours and surrounding normal tissue is revealed. Many tumours prevail under acidic conditions. Therefore measurement of pH has been conducted for therapy of tumours.
In order to simulate an environment like a patient’s digestive sys- tem, a normal level and an exceptional level of potassium and PH are combined to create the data set. Potassium amount is obtained from Table 1 and PH values of different positions in the digestive system are displayed in Figure 10[39].
Figure 10: The Human Digestive Tract pH Range Diagram
One Sample
Two results are generated which are from blood potassium sensor and touch PH sensor. Figure 11 is one case which can present pa- tient data. The red line in Figure 11 shows the different amount of potassium in the human digestive system, and the blue line shows PH values in the digest system when the medicine travels through the human body. Moreover, the yellow line displays interesting time points. Each data is generated by seconds which is the axis. Later on, a switching manipulation between sensor states with detecting inter- estingness is evaluated by the yellow line. Since signal processing is not included in this project, that is the reason the data keeps clear and tidy.
Figure 11: Potassium and PH Data set
Twenty Samples
Because only one sample would be too few to show adaptive per- formance, the same methods will be used to 20 different situations, which reveal that the proposed system can deal with problems in dif- ferent positions. A random seed was used to generate twenty cases (??) of bleeding positions and tumour positions as inputs to the smart system.
Special Samples
(a) 20 Bleeding patients
(b) 20 Tumour patients
Figure 12: 10 Different Patients Blood Detection
(a) 6 Bleeding Samples
(b) 6 Tumour Samples Figure 13: Special Samples
InFigure 13, there are 6 special cases of different positions of bleeding or tumour points.
3.3.1.2 Pressure Colour Pressure Sensor
Figure 14: Outputs From FSR
For tumour detection, a pressure sensor can be passed along a surface, where the sensor is depressed when there is a bump. So a simulation pressure provided to a pressure sensor in order to collect the signal.
Single tumour and multiple tumours are two extreme cases, as shown inFigure 14.
3.3.2 Basic math models
Math can be used to understand data properties. Therefore, a few math models are mentioned in this section.
Mean
Mean is a statistic which is the average of a data set. It is calculated by the total sum amount divided by the total count of the numbers added. Mean can indicate the centre position of a data distribution and also it can smooth signals that contain noise.
Standard Deviation
Standard Deviation can be used to quantify the variation or disper- sion of data set. Sudden changes can be detected by Standard Devia- tion. So this model is used frequently in one algorithm in this thesis.
