A Flexible, Low-cost Approach to Slippage Detection using Pyroelectricity

V¨

aster˚

as, Sweden

Thesis for the Degree of Master of Science in Engineering - Robotics

30.0 credits

A FLEXIBLE, LOW-COST APPROACH

TO SLIPPAGE DETECTION USING

PYROELECTRICITY

Anders Robin Larsson

rln14003@student.mdh.se

Examiner: Mikael Ekstr¨

om

M¨

alardalen University, V¨

aster˚

as, Sweden

Supervisor: Fredrik Ekstrand

M¨

alardalen University, V¨

aster˚

as, Sweden

Company supervisor: Coen Lauwerijssen,

2M Engineering, Valkenswaard, The Netherlands

Abstract

Myoelectric prosthesis on the market today are mostly very expensive and rarely allow the amputee any feedback, leaving the users separated from their own robotic arm. Integrating sensory systems into an arm which needs to be replaced several times during a lifetime may also not be cost efficient. With a sensory system fitted in a removable and re-sizeable glove, the sensory system will not require a replacement unless broken. Using a flexible, durable, low-cost material, sensitive to both change in pressure and temperature, this may be achieved. Using the pyroelectric properties of thin-film Polyvinylidene Fluoride, a sensor able to detect the incipient of slippage and its initial direction is achieved.

Table of Contents

1. Introduction 4 1.1. Problem Formulation . . . 4 1.2. Hypothesis . . . 4 1.3. Research Questions . . . 4 2. Background 6 2.1. Prosthetic devices . . . 6 2.2. User feedback . . . 6 2.3. Current Market . . . 6 2.3.1 Prosthetic hands . . . 7 2.4. Pyroelectricity . . . 9 2.4.1 Theory . . . 9 2.4.2 Applications . . . 11 2.4.3 Materials . . . 11 2.4.4 Figure of Merit . . . 11 2.4.5 Signal processing . . . 12 2.5. Comparison method . . . 13

2.5.1 Force Sensitive Resistor . . . 13

2.5.2 Discrete Wavelet Transformation . . . 13

3. Related Work 15 4. Method 16 4.1. Sensor Module . . . 16 4.1.1 Pyroelectric material . . . 16 4.1.2 Electrode Attachment . . . 17 4.1.3 Heating element . . . 17 4.1.4 Initial design . . . 18 4.1.5 Final Design . . . 19 4.2. Signal processing . . . 19

4.2.1 Equivalent Circuit Model . . . 19

4.2.2 Wiring . . . 20

4.2.3 Charge Amplifier . . . 20

4.2.4 Filtering . . . 21

4.2.5 Instrumentation & Verification . . . 22

4.3. Test Setup . . . 23

4.3.1 Characterization test . . . 23

4.3.2 Gripper test . . . 26

4.4. Graphical User Interface . . . 27

4.4.1 Data Structure . . . 27

4.4.2 Experimental . . . 27

4.4.3 Analysis . . . 29

5. Ethical and Societal Considerations 30 6. Results 31 6.1. Characterization . . . 32 6.2. Gripper . . . 35 7. Discussion 37 8. Conclusion 38 9. Future Work 39 References 43

Appendix A Appendix A 44 1.1. Results of tests with Gripper used for averaging . . . 44 1.2. Average comparison between different sensing elements . . . 47 1.3. Slippage Direction Detection compared with mechanical disturbance . . . 48

List of Figures

1 I-Limb by TouchBionics (Retrieved from [1]). . . 7

2 Advanced Bionic Hand by Psyonic (Retrieved from [2]). . . 7

3 Sensor Hand Speed by Ottobock (Retrieved from [3]). . . 8

4 PriMa by Florida Institute of Technology (Retrieved from [4]). . . 8

5 Hannes by INAIL and Italian Institute of Technology (Retrieved from [5]). . . 8

6 All Ferroelectrics are Pyroelectric, and all Pyroelectrics are Piezeoelectrics, but not vice versa. . . 9

7 The black dot representing the centre of a cell with surrounding atoms. Symmetrical (left) and non-symmetrical (right) atomic structure (Image taken from PowerPoint of S. Stuijk [6]). . . 10

8 FSR for pressure (left) and for bending (right). . . 13

9 Common wavelets starting from top left, moving right: Morlet, Daubechies, Coiflet, Biorthogonal, Mexican Hat, and Symlets (Taken from Matlab). . . 14

10 The Haar wavelet presented as a mother wavelet to the left (green). Examples of scaling the wavelet on top to the right (red) and of translation over time on bottom right (blue). . . 14

11 Principle of Fingerprint sensor using pyroelectricity by J. Han et al. (Figure taken from J. Han et al. [7]). . . 16

12 Two initial designs of the sensor module to be developed for detecting slippage. . . 18

13 Explanation of the sensor module’s working principle. . . 18

14 The final version of the sensor design. . . 19

15 Model to describe internal circuit of pyroelectric material.(Retrieved from [8]). . . 19

16 Circuit for converting and amplifying signal of piezo/pyroelectric material. Including equivalent circuit for the 28 um thick PVDF-film (internal resistance not modelled). 21 17 Setup to test (left) and principle overview (right) of compensating element for fil-tering microphonic effect. . . 21

18 Overview of how sensor module is built with compensating sensor element to reduce microphonic effect. . . 22

19 Piezo Film Lab Amplifier (TE Connectivity). . . 23

20 Analog-to-Digital Converter (Adlink Technologies). . . 23

21 Infrared array sensor HTPA80x64d (from Heimann Sensor Datasheet). . . 23

22 Top (left) and back (right) view of the characterization test setup. . . 24

23 The base characterization test setup. . . 24

24 Extension of characterization test setup. . . 24

25 Experimental setup in both directions. . . 25

26 Setup of cart for investigating the pyroelectric response. . . 25

27 Thermal camera modification to center of base plate. . . 25

28 Experimental test setup for slippage detection. . . 26

29 Data structure of a test run saved in an xlsx-file sheet. . . 27

30 The main GUI used as an assisting tool when running experiments. . . 28

31 GUI used for performing offline analysis of collected data. . . 29

32 Maximum heat allowed for heating element. . . 32

33 Thermal images of heat profile in slipping object. . . 32

34 Pyroelectric response for moving heating element attached to stick on cart (no phys-ical contact). Moved past sensor material twice in opposite directions (Axis Units (x,y): [ms], [∝ V ]). . . 33

35 Pyroelectric response for moving heating element attached to stick on cart (no phys-ical contact). Stopped above sensor material and then moved back (Axis Units (x,y): [ms], [∝ V ]). . . 33

36 Comparison between long and short wires to read output of PVDF (Axis Units (x,y): [ms], [∝ V ]). . . 34

37 Average test results for blank 2x3 mm sensing element placed before heat (Axis Units (x,y): [ms], [∝ V ]). . . 35

38 Average test results for blank 2x3 mm sensing element placed after heat (Axis Units (x,y): [ms], [∝ V ]). . . 35 39 Comparison of average results with 4 second heat profile on object before slippage

(Axis Units (x,y): [ms], [∝ V ]). . . 36 40 Slippage of object imposed in both directions, with heat (∆t=4s) and without(Axis

Units (x,y): [ms], [∝ V ]). . . 36 41 Five runs with heat element after sensing element, with no heat on(Axis Units

(x,y): [ms], [∝ V ]). . . 44 42 Five runs with heat element after sensing element, with 2 seconds of heat on before

slippage(Axis Units (x,y): [ms], [∝ V ]). . . 44 43 Five runs with heat element after sensing element, with 4 seconds of heat on before

slippage (Axis Units (x,y): [ms], [∝ V ]). . . 45 44 Five runs with heat element before sensing element, with no heat on (Axis Units

(x,y): [ms], [∝ V ]). . . 45 45 Five runs with heat element before sensing element, with 2 seconds of heat on before

slippage. three runs to the right for simpler visualization (Axis Units (x,y): [ms], [∝ V ]). . . 46 46 Five runs with heat element before sensing element, with 4 seconds of heat on before

slippage (Axis Units (x,y): [ms], [∝ V ]). . . 46 47 Comparing average of five runs with no heat (Axis Units (x,y): [ms], [∝ V ]). . . 47 48 Comparing average of five runs with 2 seconds of heat on before slippage (Axis

Units (x,y): [ms], [∝ V ]). . . 47 49 Comparing slippage direction response with mechanical disturbance of setup (Axis

Units (x,y): [ms], [∝ V ]). . . 48

List of Tables

1 Seven different myoelectric prosthetic hands released on the market in the past decade. One (PriMa) most likely never reached market as it was a university project and no information can be found after 2016. . . 7 2 Comparison of pyroelectrics, data taken from manual of Measurement Specialities

Inc.[9]. . . 12 3 List of world-wide companies selling PVDF film along with countries of origin. . . 17 4 Parameters for calculating capacitance of PVDF. . . 20 5 Capacitance values of PVDF films. . . 20 6 List of selected parameters for Piezo Film Lab Amplifier. . . 31

Acknowledgement

I would like to express my great appreciation to Fredrik Ekstrand for supervising and supporting me during my work. I would also like to acknowledge Coen Lauwerijssen and everyone at 2M Engineering for the help and support I’ve received. Lastly I would like to give a special thanks to my family and girlfriend for continuous support during the darkest hours of this work.

1.

Introduction

Pyroelectricity is a phenomenon which have been known for over 2000 years[10], but received its daily used name in 1824[11]. It is today applied in many different sensor and actuator solutions. The focus of this work is to investigate whether the pyroelectric effect can be utilized to detect slippage of objects grasped by robotic grippers, more specifically a prosthesis. A myoelectric pros-thetic hand which does not have any sensing feedback to the user will instead put strain on other sensory systems. Without proprioceptive feedback or touch, users are forced to rely on visual feedback instead of intuition. As mentioned by D. Joshi et al. [12]:

”Intuitive myoelectric prosthesis control is difficult to achieve due to the absence of proprioceptive feedback, which forces the user to monitor grip pressure by visual information leading to fatigue and handling errors.”

Without any information from the hand, this will be replaced by the visual and auditory systems instead. Eg looking as an object is being grasped or hearing when a grasped object has already slipped out of the hand and dropped. Today there are several solutions for how to solve the specific sensing problem for slippage using optics[13] and vibration[14] among others, but they are all currently integrated into their respective prosthetic devices. Since a younger user must replace their prosthetic device frequently to be proportional with their body, there is an unnecessary cost for the sensory system if it’s integrated to the device. On top of this, most prosthetic devices doesn’t include the user with the sensory information, but is instead only used by the controller. This type of solution frees the user from cognitive burden, but at the same time it excludes the user from the interactive information given by eg touch or temperature. P.H. Chappell [15] makes a good point on this, as the dexterity of the prosthetic devices increases, so does the need for higher sensing capabilities, also for the cases where only the controller is closed into the loop. With a low-cost glove solution for slippage detection of grasped objects, which can easily be adapted to different prosthetic hand devices, the end goal is to give any prosthesis user the option to include sensory feedback for slippage to their device. This section will present the work performed in three different forms; problem formulation, hypothesis and research questions.

1.1.

Problem Formulation

The challenges of this work is highly related to the selection and characterization of a material possessing certain key properties, the main one being pyroelectricity. The full sensor requires flexibility, durability and must be low in cost to produce in order to achieve an integratable and cheap sensor glove solution.

1.2.

Hypothesis

Based on the limitations and related work from B. Yang et al.[16], the following hypothesis was concluded:

By the use of pyroelectric properties, a flexible, low-cost slippage detection sensor, integratable into a glove, can be developed.

1.3.

Research Questions

On the basis of the hypothesis and the limitations mentioned by S.B. Lang [17], the following research questions were raised:

1. What pyroelectric material should be used?

• Which materials will be affected the least by elements other than heat? 2. Can slippage be detected using the pyroelectric properties of a material?

• What type of algorithm and signal processing can be used?

3. What figure of merits can be concluded for thermal sensing applications using pyroelectrics?

• Which material parameters will affect the results the most?

2.

Background

In this section a basic explanation of the key concepts related to this work will be presented. To start, a short description of the type of prosthetic device used as reference will be discussed. This will be followed by a reasoning for why the development of a cheap and adaptable sensory solution is necessary, specifically for slippage detection. Also included will be the current market for this type of application, and the principles of pyroelectrics as well as the material to be used. Lastly the signal processing followed by a description of the comparison method will be discussed.

2.1.

Prosthetic devices

A prosthetic device can be defined and categorised based on the working principle of the device, and the missing limb being replaced. Starting with the latter, there are four different categories describing which limb is being replaced, as described in an article by C. Woodford [18]:

• Below the knee Transtribial (BK) A prosthetic lower leg attached to an intact upper leg. • Above the knee Transfemoral (AK) A prosthetic lower and upper leg, including a prosthetic

knee

• Below the elbow Transradial (BE) A prosthetic forearm

• Above the elbow Transhumeral (AE) A prosthetic lower and upper arm, including a pros-thetic elbow

The working principles can be defined as three different prosthetic types, as mentioned by A. Chadwell et al.[19]:

• Cosmetic A static hand used for visual purpose mostly

• Body Empowered Functional and mechanically controlled by the amputees body, by e.g. harness or cables

• Electrically powered (Myoelectric) Functional and controlled by electromyographic (EMG) signals read from the amputee’s body

As a body empowered prosthetic hand makes use of body movements to control the opening and closing of the device, a feedback is still given to the user. This is because the body becomes mechanically connected to the device gripper, unlike myoelectric prosthesis which removes this mechanical feedback to the user. In return for the loss of feedback however, more dexterity can be gained as more commands become available, for less movement, to control the actuation of the gripper. The work of this thesis will focus mainly on developing feedback for myoelectric, transradial devices. Do note however, this does not limit the applicable field to only this type and category of prosthesis, neither will it be limited to the sole application of prosthetic devices.

2.2.

User feedback

A supporting factor for development in this topic is found in the high rejection rates of prosthetic devices. Abandonment of prosthesis by users reach approximately 30% due to association with a foreign object, mentioned by L. Zollo in E. Waltz’s article (2019) [20] ”About 30 percent of prosthesis users abandon them out of frustration with operating what feels like a foreign body..”.

2.3.

Current Market

The current market for pyroelectric solutions border more than one field, and is often found in different types of detectors[21][22][23][24]. To keep the focus narrow only one application field will be taken into account for the work. As mentioned previously, the focus is on enabling transradial, prosthetic device users a sensory upgrade, making prosthesis the field of interest. It’s worth noting another potential market to consider as it directly relates to the solution itself, which is sensory glove solutions as the sensor is to be integrated into a thin glove.

2.3.1 Prosthetic hands

The leading developers and most recent products of prosthetic hands are very diverse. From highly expensive and sophisticated to cheaper devices with only basic functionality. Depending on where in the world, different prosthesis are available for different people. In this section, eight devices found in table 1 are compared. The five devices with sensory feedback will be shortly described, where as three of them does not include the user.

Product Name

Hero

Arm Bebionic I-limb

Advanced Bionic Hand

SensorHand

Speed Michelangelo PriMA Hannes

Sensing Capabilities

No sensors

No

sensors Slippage Touch Slippage

No external

sensors Pressure

Pressure Position Slippage

Feedback to - - Controller User Controller - User Controller

Table 1: Seven different myoelectric prosthetic hands released on the market in the past decade. One (PriMa) most likely never reached market as it was a university project and no information can be found after 2016.

Figure 1: I-Limb by TouchBionics (Retrieved from [1]).

I-Limb Developed by TouchBionics[25], at a market cost between 60’000-120’000 USD, the I-Limb has several adap-tive grips but no sensory feedback. It is however able to detect when an object is touched by measuring the current input to the motors controlling the gripper.

Advanced Bionic Hand Out of the eight prosthetic de-vices presented in the table above, this is the only product on the market which delivers sensory feedback to the user. It is developed by Psyonic [2] and has low-cost pressure

sen-sors embedded into the fingertips which is presented to the user through electrical stimulation. Mechanically stretching the skin is another feedback they offer in order to give some sense of pro-prioception. It’s planned to reach the market this year (2019) with a production cost of 550 USD. Research prototypes are already available for 5’000 USD .

Figure 3: Sensor Hand Speed by Ottobock (Retrieved from [3]).

Sensor Hand Speed A three-fingered gripper by Ottobock [26] seen in fig. 3 to the right. The device costs about 10’000 euros. With a sensory feedback for slip-page back to the controller, the device is able to detect and prevent slippage of grasped objects without the user needing to intervene.

PriMA A project for a 3D printable prosthetic hand and arm by Florida In-stitute of Technology[4], seen in fig. 4 be-low. The device is said to cost 1’500 USD and has a sensory feedback for pressure to the user. No further information was found after 2016 and it is believed that the project is no longer active.

Figure 4: PriMa by Florida Institute of Technology (Retrieved from [4]).

Figure 5: Hannes by INAIL and Italian Institute of Tech-nology (Retrieved from [5]).

Hannes A prosthetic hand devel-oped in a collaboration between IN-AIL [5] and the Italian Institute of Technology, see fig. 5 to the right. The device costs approxi-mately 10’000 euros and contains sensors for pressure, position and slippage, with all information going to the controller only.

2.4.

Pyroelectricity

Pyroelectricity is a material property which causes an electrical charge to be elicited upon temporal change in temperature. Depending on the application this charge can be read as either a signal, as done in sensors[27], or be stored/re-used as energy, as done for energy-harvesting [28]. This section will dig deeper into the different aspects of pyroelectricy necessary for this work. First, some general theory behind the pyroelectric property will be presented, including a small part of piezoelectricity as well. This will be followed by current applications utilizing the property of pyroelectricity, and lastly a closer look at what parameters are of importance when choosing the material for the specific application of this sensor module.

2.4.1 Theory

All pyroelectrics also possess piezoelectric properties, therefore the output from the material will be divided and discussed as three different parts; the piezoelectric effect, the primary pyroelectric effect and the secondary pyroelectric effect. The last one is a combination of the other two as the material expands due to thermal in-flux, thereby inducing new stress upon the material. Each of these can induce internal polarization, meaning all three will also interfere with each other.

Pyroelectric and ferroelectric materials are both piezoelectrics, along with some additional properties, making them a sub-group of piezoelectrics as seen in fig. 6. For a material to be piezoelectric, it must have the property of transforming electrical energy to mechanical energy and vice versa. Pyroelectric materials also have the property of converting thermal energy to electric energy, and this is true as well for ferroelectrics. What differs this final group of materials is that they also require their spontaneous polarization to be reversible. So, theoretically ferroelectrics are also pyroelectric, but not all of them have useful pyroelectric properties in real world applications.

Figure 6: All Ferroelectrics are Pyroelectric, and all Pyroelectrics are Piezeoelectrics, but not vice versa.

Piezoelectric effect Piezoelectrics are non-centrosymmetric materials as seen in fig. 7, and can be considered as capacitors when used as sensors. Due to this atomic structure the material can operate as a transducer, both from mechanical to electrical and vice versa. As the material is introduced to mechanical stress, the internal structure will shift and the magnitude of the material’s internal polarity will change causing a charge to elicit. The polarity may even become negative to its previous state. When an external electrical field is applied, this will also causes a shift in the internal structure and thereby physically change the size of the material. Depending on the direction of the applied polarization and the internal polarization, the material will expand or shrink.

In situations where the piezoelectric properties are unwanted and its effect is considered noise for the application, the word microphony or microphonic effect can be used as described by

In-Figure 7: The black dot representing the centre of a cell with surrounding atoms. Symmetrical (left) and non-symmetrical (right) atomic structure (Image taken from PowerPoint of S. Stuijk [6]).

fraTec[29]. This is the case for Passive Infrared (PIR)-sensors which uses pyroelectric material to detect the movement of eg humans through Infrared Radiation (IR). This type of noise is a problem for pyroelectrics and most likely actions must be taken in order to reduce it. Such actions focus on either separating the sensing material from any environment prone to vibration or by heavily dampening the surrounding vibrations in the environment, as mentioned in the previous stated manual of InfraTec[29]. Measurments Specialities argues that the electrical charge generated by mechanical stress is proportional to the volume of the film under stress (p. 6 [9]). Meaning thinner film gives rise to lower charge build-up

Primary pyroelectric effect Pyroelectrics as already mentioned possess the same properties as piezoelectrics along with the ability to convert energy between a thermal and electrical form. This is done mainly by the primary pyroelectric effect, which can be defined both under unloaded or constant strain/stress. From here on this report will focus only on the condition of material under constant stress, as it may significantly differ to free conditions. As temperature changes, two phenomenon occurs within the material. The first one is that the material’s internal dipoles will have a change in distance between its opposite charges, and the second one being a change in the orientation of individual dipoles. The latter one is also known as thermal agitation according to S. Stuijk [6]. Both of these will affect the polarity of the material and thereby the charges which are built up on each side of the material.

Secondary pyroelectric effect The secondary effect is defined by Bhalla et al. [30] as the thermal deformation of the crystal, or by X. Li et al. [31] as the thermal expansion. As mentioned earlier, the temperature change of the material will cause it to change size and thereby inflicting new stress upon the material which in turn will have an additional piezoelectric effect. For appli-cations such as Passive Infrared (PIR)-sensors, this is an undesired noise which is filtered out by the use of a second sensor element as described by S. Stuijk [6] and ”Application of a Pyroelectric - Infrared detection” [32].

Formulas For pyroelectrics, the rate at which the temperature of the material changes (∆T ) is proportional to the current output as described by Measurement Specialities( p.[9]), and is dependent on the pyroelectric coefficient (ρ), as can be seen in eq. (1 & 2) below.

Q = ρ∆T A (1)

V = ρ∆T

 (2)

Following formula eq. (3) relates capacitance C with the dielectric permittivity , where A is the area of the capacitor and d is the distance.

C = A

 can in turn be described with the relative permittivity r and the absolute permittivity 0(a constant of 8.854 x 10−12 F/m) as seen below in eq. (4)

 = r0 (4)

2.4.2 Applications

Pyroelectric materials are used in a wide range of sensor applications such as intruder and fire alarms, pollution monitoring, gas analysis, thermal imaging, IR detection etc.[21][22][23][24][17], but also in other applications such as energy harvesting[33]. As pointed out by C.C. Hsiao et al. [34], pyroelectric materials also have a lot of desired properties on top of their pyroelectricity; they are fast with a wide spectral response, highly sensitive, low cost, and due to its low limitation in size, it can easily be scaled and integrated into circuitry. A company by the name Pyreos[35] uses a thin film material with pyroelectric properties known as Lead Zirconate Titanate (PZT) for their gas/flame detectors and analysers among other products. Another well-known and used pyroelectric is Polyvinylidene Fluoride (PVDF) [36].

2.4.3 Materials

Today there’s a large variety of different pyroelectric materials such as Lead Zirconate Titanate (PZT), Lithium Tantalate (LiTaO3), Strontium Barium Niobate (SBN) and Triglycinesulphate (TGS)[37][38]. Countless more are discussed by M Srinivasan[39]. These are used in several applications already discussed above in section 2.4.2. The choice of material depends on the application and selection is partly based on a defined Figure of Merit (FoM), usually done by the author for the specific application. The pyroelectric property differs between materials and so does the piezoelectric property. Besides these, other factors may differ between materials as well. Piezoelectrics exist as natural materials such as Quartz which is a very known material used in clocks, but may also be synthetically produced as both solids, such as PZT, and films like PVDF. Each have their own advantages and disadvantages which must be considered for the desired properties.

2.4.4 Figure of Merit

According to an article from 2012 by X. Li et al. [31], there are three re-occurring FoM used for assessing pyroelectrics and its output. Two of them are the FoM voltage (FV) and current response(FI). The third one is specified as critical for detectors, and is related to the Signal to Noise Ratio (SNR) (FD). All three are related to the pyroelectric constant p, which can be seen in eq. (5-7) below. Where ’ and ” are the real and imaginary parts of the dielectric permittivity, respectively. FI = p (5) FV = p/0 (6) FD= p/ √ 00 ₍₇₎

Laser Components LLC [40], a company manufacturing pyroelectric detectors, states four dif-ferent parameters when defining a FoM for comparison of three materials used for pyro-detectors. Two of these parameters are the already mentioned pyroelectric coefficient and dielectric permittiv-ity. What will also be an important factor in this specific application, is the piezoelectric property as the sensor will be prone to the following three:

• Vibration from object sensor is attached to • Vibration from grasped object

All three may cause noise in the raw signal, but the two latter may also have a favourable effect on the resulting output of the sensing material. Therefore, the piezoelectric coefficient is also to be considered. This leaves three important parameters which will be considered for the choice of material in this specific application:

• Pyroelectric Coeffient (p) Ability to produce electric charge from IR radiation. The bigger the better

• Dielectric Permittivity () Determines the capacitance, and thereby affect the noise. The larger the capacitance ( eq. (3)) the lower the noise (for voltage operation)

• Piezoelectric Coefficient (d) Ability to produce electric charge from mechanical stress. Preferably smaller for the sake of Signal to Noise Ratio (SNR)

This is also further strengthened by two out of the five parameters used in Measurments Spe-cialities Inc. manual[9] to compare different pyroelectrics, see table 2. Where pQ is defined as the pyroelectric charge coefficient,  the dielectric constant, α the thermal diffusivity, L the thermal Difffusion Depth at 1Hz, PV the pyroelectric voltage coefficient and lastly Ml as the FoM. The only notation made for the last parameter is the unit of the presented value, which can be seen in eq. (8).

ρQ [C ∗ ],

V ∗ mm2

J (8)

Material TGS LiTaO3 BaTiO3 PZT PbTiO3 PVDF VF2VF3

ρQ 350 200 400 420 230 30 50 /O 30 45 1000 1600 200 10.7 8.0 α .16 1.31 1.00 .44 .67 .06 .06 L 225 646 564 374 461 138 138 Pv 1.32 .50 .05 .03 .10 .47 .71 Ml .53 .16 .02 .01 .03 .20 .31

Table 2: Comparison of pyroelectrics, data taken from manual of Measurement Specialities Inc.[9].

2.4.5 Signal processing

The electric charge given by piezo/pyroelectric materials is very small and will therefore require to be preprocessed before it being analyzed. When R. H. Brown [41] performed some different experiments with PVDF film of size 15x30 cm, a peak output of 68 mV (actually measured charge: 8.8 nC) was observed as a response from the Infrared Radiation (IR) radiation of an adult human, at a distance of 1.7m away. For analyzing the signal, the film was connected to a preamplifier. This way of pre-processing the output signal has been performed by others as well[42], and is used within commercial piezoelectric products. There are two commonly used methods for pre-processing the signal from piezoelectric materials: Voltage mode and Current/Charge mode [43][40]. Voltage mode reads the signal from the sensor as if it was connected to an open circuit. The charge builds up on the two electrodes of the sensor which creates a potential difference that can be read as a signal. The current mode however operates in a short circuit manner, where the built up charge is removed from the sensor resulting in a current output and zero potential difference between the two electrodes. This current can then be proportionally converted to a voltage that can be considered the signal output. S. Stuijk also discuss the two different types of circuits which are used for processing the raw output signal from the sensing material of a pyroelectric sensor. He refers to the two as voltage follower and current-to-voltage converter. The former having a longer response time than the latter ( p.23-25 [44]).

In addition to the conversion and amplification of the signal, it will also require some filtering. According to InfraTec( p.48[45]), the pyroelectric excitation is within a single frequency or a slim frequency band. With the use of a bandpass filter, a lot of noise introduced from the cables and

connections, due to the very small raw signal, can be filtered away. Other sources of noise is also introduced, some of the more important ones such as the piezoelectic effect, but also smaller from the circuits themselves. As Hamatsu Photonics characteristics document mentions, for charge amplifiers there are three main sources of noise (p.5 [46]). As this work will not focus on a final product, the minor-impact noises will not be discussed further. Only the main sources of noise will be considered, such as the microphonic effect, electromagnetic interference, and others which may have a considerable effect on the result of this specific application.

2.5.

Comparison method

To compare the outcome of this work, an already mentioned method has been chosen as reference and will be presented in this section. B. Yang et al. [16] found that using a cheap, commercially available pressure sensor sampled at 1kHz along with Discrete Wavelet Transform (DWT), the occurrence of slippage can be detected. As a grasped object starts slipping, the object and grasping finger will experience a rapid drop in pressure from each other. The speed at which this decrease in force occurs is unique compared to both grasping and ungrasping of the object. With the use of DWT and a threshold, this occurrence can be distinguished and directly related to the incipient of slippage from a grasped object.

2.5.1 Force Sensitive Resistor

Also known as force-sensing resistor, sensors consists of a conductive substance, such as carbon and changes its resistance proportionally to the pressure or stress applied to it. They can easily be printed with resistive-ink (eg carbon based) on thin substrate, making them very cheap to produce and flexible. The principle can be used to create both bend-and pressure sensors, see fig. 8. They are however not very accurate and can drift in value for a longer time if under pressure. Once drifted it can also take up to several hours for it to be fully recovered depending on the exposed pressure and exposure time.

Figure 8: FSR for pressure (left) and for bending (right).

2.5.2 Discrete Wavelet Transformation

Fourier Transform (FT) is able to show the frequency components of a signal, but without any information on when in time these components occur. This extra information can however be found with the use of DWT. This method uses something called wavelets, which can be constructed in an infinite number of varieties, to find the location in time at which a certain frequency occurs. Some common wavelets that have been predefined in Matlab can be seen in fig. 9. What might not be directly obvious at a first glance is that all wavelets have a total area summing up to 0, this equilibrium is so the energy will be equally distributed. Mathematically this can be seen in eq. (9), where ψ(t) is the wavelet function.

Z ∞ −∞

Figure 9: Common wavelets starting from top left, moving right: Morlet, Daubechies, Coiflet, Biorthogonal, Mexican Hat, and Symlets (Taken from Matlab).

A mother wavelet is chosen based on the application, or the component of interest within a signal more precisely. This mother wavelet is then translated and scaled over time and frequency in order to find certain frequency components at specific locations in time, see fig. 10. Here the Haar wavelet is used as a mother wavelet. Looking at the top figure to the right, scaling of the mother wavelet can be observed. This is necessary to match the the mother wavelet with the desired feature of a signal which may vary in frequency at different points in time. This leads to the bottom-right figure which shows how the mother wavelet is translated over time to find a match in the original signal. By scaling the wavelet, the same features at different frequencies can be obtained and by comparing it with the signal at different time steps, frequencies can be matched with their corresponding locations in time.

Figure 10: The Haar wavelet presented as a mother wavelet to the left (green). Examples of scaling the wavelet on top to the right (red) and of translation over time on bottom right (blue).

Mathematically this can be defined as seen in eq. (10), where x(t) is the signal to be analyzed, ψ_a,b∗ (t) the analysing function (ie the mother wavelet), and X(a, b) is a matrix represented by the two coefficients scale a, and translation b.

X(a, b) = Z ∞

−∞

3.

Related Work

As pyroelectric applications has already been discussed, this section will focus more on related work within slippage detection. This has become an increasingly larger area along with the development of robotics today. Robotic grippers are becoming increasingly complex in order to execute tasks requiring more dexterity and precision. In case an object slips, this may cause an undesired orientation or position of the object. Scenarios like these can be very important to detect for industrial robots, but also for prosthetic grippers where the user might not be aware that a gripped object slips and/or drops.

Several different methods have been proposed for detecting slippage in robotics grippers. First one which has already been discussed, as it will be used for comparison, is by the use of an FSR sensor to detect a transient drop in pressure upon slippage [16]. Similar work has been performed by C Pasluosta et al. [47] where a neural network is used in addition to adapt the grip force. This method is only able to detect the point in time at which slippage begins, but not direction nor when it stops. Another method proposed by H N Sani et al. [13] for prosthesis is the use of an optical solution. By the use of a laser sensor found in modern computer mice, accurate results for slippage detection were obtained. The problem with this type of implementation is however that it looses accuracy on highly reflective and transparent surfaces, as is also commonly known for normal computer mice. Similar work have been performed by R Anderberg et al. [48] with their Intelligent Pre-touch Adaptive (IPA) Gripper and following work by J Johansson et al. [49].

Proposals of methods utilizing the onset of vibrations as a result from slippage have also been made. One method, discussed by M Ostberg et al. [50], tried to use piezoelectric properties of PVDF to detect slippage from vibration. Another method by D Dornfeld et al. [14] takes advantage of the acoustic emission to detect slippage. A problem which arise when using vibration is that robotic grippers are very prone to vibration from the external environment which may cause false positives.

4.

Method

By the use of a flexible material with pyroelectric properties, combined with a heat-emitting frame around this material, a small enough sensor module to fit on the fingertip of a glove is developed to detect slippage of grasped objects. The heat-emitting frame, surrounding the pyroelectric material, is to create a temperature profile on the grasped object. As the object starts to slip, so will the position of the heat-signature in relation to the sensing material. This change in temperature will be picked up by the pyroelectric material and thereby elicit a change in potential which can be detected as a slippage. A sensor with similar working principle at a much smaller scale has been done for fingerprint sensing as presented by J. Han et al. [51][7], see fig. 11. This sensor heats up an array of micro elements, and when a finger is pressed against the sensor some of the elements will make contact with the ridges of the fingerprint, while others will end up between the ridges (in valleys). The ridges of the fingerprint acts as heat-sinks, creating a temperature difference between the elements at a ridge and those in the valleys.

Figure 11: Principle of Fingerprint sensor using pyroelectricity by J. Han et al. (Figure taken from J. Han et al. [7]).

The developed sensor module is to be compared with the approach of B. Yang et al. [16] by using an Force Sensitive Resistor (FSR) and Discrete Wavelet Transform (DWT) to detect a unique drop in pressure as slippage occurs.

4.1.

Sensor Module

In this section all choices regarding the sensor module will be presented. First the choice of material will be reviewed followed by electrode attachments. These two are very important factors as the choice of material and how it is attached may have a large impact on the sensors output. The heating element will be briefly discussed and lastly the sensor module design will be presented. During the work performed, more than one design approach was considered. The design changed over time as new knowledge was obtained from both empirical results and theoretical reviews. Each design will be presented including a motivation for why it was chosen and possible problems encountered with the design.

4.1.1 Pyroelectric material

Materials which possess the necessary qualities for this work make a very short list. It should pos-sess a strong pyroelectric effect, be flexibly integrable, and also be mass-producible for a low cost of the final sensor. A well-used material which possesses all of these is PVDF, a thin-film, pyroelectric polymer. To say it has a strong pyroelectric effect is debatable as many other materials have a much larger pyroelectric constant than PVDF, but it’s strong enough to be used in pyroelectric applications. This material is also rather unique in todays’ industry for pyroelectrics, and that has been well exploited. PVDF is used to mix with different materials to improve or obtain new properties while still preserving its durance and the flexibility of a film. One of these mixtures is P(VDF/TrFE) which shows promising results regarding its pyroelectric applications[42]. For the

reason that PVDF is one of the only pyroelectric thin-films that is commercially available and can be purchased for a relatively low price, it was chosen for this work.

A compilation of six companies selling PVDF and co-polymers can be found in table 3 below. After reviewing each one of them, a final choice was made based on delivery time, coating, available size of sheets and cost. Certain mixtures of P(VDF/TrFE) does have better pyroelectric properties than normal PVDF, but was ruled out for this work due to its price. The Student pvdf Starter kit from Precision Acoustic was ordered. The kit includes three 5x5(cm) sheets of poled PVDF with different thicknesses; 28, 52, and 110µm. Each sheet also contain a coating on both sides with a layer of about 250nm Gold on top of a 40nm layer of Chrome for improved connectivity with electrodes.

Company Country Material

Precision Acoustics[52] UK PVDF

Good Fellow[53] UK(France, Germany) PVDF

Polyflon[54] UK PVDF

PiezoTech[55] France PVDF

P(VDF/TrFE)

PolyK Technologies[56] USA PVDF

P(VDF/TrFE)

Mouser[57] USA PVDF

(Te Connectivity[58]) Switzerland

Table 3: List of world-wide companies selling PVDF film along with countries of origin.

4.1.2 Electrode Attachment

In order to use the pyroelectric material for sensing, electrodes must be attached to each side of the film. Due to the low melting temperature and even lower degradation temperature of PVDF film, soldering is not a possibility for closing the circuit between the electrodes and the film. As men-tioned in the Technical data sheet by Precision Acoustics[59], two possibilities exist to attach the electrodes: mechanical contact and conductive adhesion. Considering that the application requires the flexibility of the film, connecting the two electrodes with conductive adhesion is considered a more suitable choice. Some very important factors of the glue, besides being conductive, is a good adhesive strength, durability, and a low cure temperature. In the data sheet from Precision Acoustics they mention that most conductive adhesives have a low durability, but recommend two products to use with their PVDF film from Henkel [60]: Henkel Loctite Abelstik 56C [61] and 64C [62]. They are also known as Hysol Eccobond 56C and 64C. However, the price for these products are quite high for a single use application (56C costing between 600-1000 USD depending on vendor) and there is already conductive epoxy, CW2400 [63], from Chemtronics[64] available. The CW2400 fulfills the requirements for having a low enough cure temperature, good adhesive strength and flexibility, and can therefore be used for closing the circuit between the sensor mate-rial and the electrodes. Before a final design choice is made and some valuable proof of concept can be confirmed, the material will be tested with mechanical attachment.

An important thing to note is that the difference between using an adhesive compared to mechanical contact can also have an impact on the output of the sensing material. The glued area will be placed under stress as the material changes from eg thermal expansion, while the non-glued area will be under free stress at all times.

4.1.3 Heating element

The heating element is allowed to take on a temperature of maximum 40 degrees, as this is just below the human pain threshold, as stated by Eugene Ungar et al.[65]. This is also within the operational temperature of the pyroelectric material. Integrating the heating element also requires that it is isolated as much as possible from the sensing material in order to get as strong signal as possible. As discussed by J Han et al. [7], in order to isolate from thermal radiation the underlying

substrate should have as small heat conductivity as possible. In addition, thermal insulation should be placed under the heating element to reduce the transfer of heat to the substrate. This is because the substrate will hold both the sensing material and the heating element, thereby connecting the two. So the more heat being transferred via the substrate, the less difference will there be between the heat frame of the object and the sensing material. This is the opposite of what is desired since the signal is directly affected by the rate of temperature change of the pyroelectric material. 4.1.4 Initial design

Two concepts were originally designed for comparison of the sensors working principles, as can be seen in fig. 12. Firstly, the front side of a finger tip is represented by the light blue ellipse. On top of the finger is the sensor module, consisting of a (red) heating element and a (gray) pyroelectric material. On the left-hand side is the initial idea of creating a full heat-frame around the sensing material. The aim of the latter design was to make the module more efficient in terms of energy consumption by reducing the necessary heat produced.

Figure 12: Two initial designs of the sensor module to be developed for detecting slippage. Using the first mentioned design, the working principle of the sensor is presented in fig. 13. Looking at the finger from the front, an object (transparent green) is placed to touch the sensor in the left most image. As the object makes contact with the sensor, a heat-frame is created on its surface as seen in the middle image. Lastly, when slippage occurs, the heat-frame gets displaced and a change in temperature occurs over the pyroelectric material. This shift in temperature will elicit a change in the polarity of the material and thereby output a temporary variation in the voltage.

Figure 13: Explanation of the sensor module’s working principle.

A test module was created for the latter design with a material size of 1.5x1.5(cm) and two Surface Mounted Device (SMD) resistors connected in parallel as heating element. To keep the sensors thin and flexible, copper tape was used as electrodes for the sensing material and for the resistors. The sensor was put together with the use of electric tape and a thin, flexible, plastic substrate.

4.1.5 Final Design

After testing the initially designed sensor module, and characterizing the material with the test setup discussed later in section 4.3.1, further changes were made based on these empirical results. A basic overview of the new design can be seen in fig. 14, where the sensing material is divided into four parts with the heat emitter remaining in the middle.

Figure 14: The final version of the sensor design.

The purpose of this design is to detect the direction of the slippage by comparing the output of the four divided sections. Included in this sensor module is also the filtering of the signals, but this will be discussed in the upcoming section covering signal processing.

4.2.

Signal processing

As discussed in the theory, for this application a fast response time is necessary and therefore a current-to-voltage converter is used for reading the raw signal from the sensing material( p.23-25[44])( p.27-28[6]). In other words, the sensor material will operate in current mode. As the output produced by piezo/pyroelectric materials such as PVDF film is of a very low charge (as low as tens to hundreds of picocoulombs), after conversion the signal needs to be amplified. The signal is also very prone to various types of noise, making this material hard to control and implement correctly, so the signal processing therefore requires a lot of attention. Therefore, in order to analyze and utilize the signal received from the sensor, there is a necessity of pre-processing in terms of amplification and filtering. A large part of this is performed with a verification instrument in order to ensure that the developed sensor module works as intended before creating a custom circuit, both the instrument and custom circuit will be discussed below. This simplifies a lot of the signal processing work, and will help to select proper parameters for the designed pre-processing circuit without the need to ensure that the circuit works correctly. This is further discussed in section 4.2.5 below. Furthermore, in order to better understand the output of the material, an equivalent circuit of the material will first be introduced.

4.2.1 Equivalent Circuit Model

Before creating any custom pre-processing circuit for the sensing material, an equivalent circuit for the material is to be modelled. Piezo/pyroelectrics have similar behaviour to capacitors: the charges build up on each side with a dielectric in-between. The full internal circuitry of PVDF film can be considered as the equivalence of a capacitor and resistor in parallel[8], this can be observed in fig. 15. Ideally, this circuit would have an infinite resistor value [66].

Figure 15: Model to describe internal circuit of pyroelectric material.(Retrieved from [8]). This was also chosen as the design for modelling the equivalent circuit of the PVDF. The value of the capacitor was calculated using equations (3 & 4) with the following parameter values seen

in table 4. 0 8.85 ∗ 10−12 r 10 12 A 2.25 ∗ 10−4 d 110 ∗ 10−6 52 ∗ 10−6 28 ∗ 10−6

Table 4: Parameters for calculating capacitance of PVDF.

ris given by the vendor and defined as a value between 10-12. d is given as three different values as the ordered student pack by [52] includes three different films of these given thicknesses. Using each thickness and the max and min value of the relative permittivity, six different capacitance were calculated, two for each film, as can be found in table 5 below.

d (um) epsilon0 C (pF) 110 10 1.81 110 12 2.17 52 10 3.83 52 12 4.60 28 10 7.11 28 12 8.53

Table 5: Capacitance values of PVDF films.

4.2.2 Wiring

Due to the very small charges elicited by the material, the wires the signal must travel before reaching the amplifier may also affect the signal and the performance of the sensor. For this reason, wires unprotected from external noise between the sensor and amplifier, is kept as short as possible.

4.2.3 Charge Amplifier

Circuit for the current-to-voltage converter and signal amplification was designed based upon circuit notes from Analog Devices Inc. [67]. Component selection was also mainly based upon their notes with minor modifications for this specific application. As the charge amplifier will only give out a response in the region of tens of millivolts (mV), an additional amplification of the signal needs to be performed. Therefore a non-inverted amplifier was added subsequent to the charge amplifier to get the signal within arduinos 0-5 volt analog input range as can be seen in fig. 16. For further protection, a zener diode was added to restrict the input voltage to the arduino between 0-5V, but is not shown in the design. The chosen OPamp AD8606 [68], used for both the conversion and amplification, was selected as it fits the single supply from the arduino 5V output and ground, using a 2.5V virtual ground. As the AD8606 is a surface mounted component, a through-hole OPamp ( OPA2340 [69]) was also chosen due to its similar specifications. The purpose of selecting a through-hole component was to easily assemble an initial circuit to test each part of the design before creating a proper Printed circuit board (PCB). The gain was calculated using the formula found in eq. (11). Where AV is the gain, Vin and Vout are the in-and output of the OPamp, and Rf and Rinare the feedback and input resistors to the inverted input, respectively.

AV =Vout Vin = 1 +

Rf

Figure 16: Circuit for converting and amplifying signal of piezo/pyroelectric material. Including equivalent circuit for the 28 um thick PVDF-film (internal resistance not modelled).

4.2.4 Filtering

In order to isolate and extract the pyroelectric effect, filtering is performed at two stages: when the raw signal is acquired by the sensing element and when pre-processed by the instrumentation, both will be presented here.

Sensing element

To remove the mircophonic effect discussed earlier, a reference (compensating) element is used[27]. This can be observed to the right in fig. 17, where two identical sensing elements are connected. One which absorbs the heat while the other reflects it. As they are placed into the same environment, under the same mechanical conditions, the vibrations and forces acting upon the two elements will be similar. The heat reflecting element can be considered as a reference element which is meant to cancel out the signals acquired from mechanical stress. The charge build up from the pyroelectric effect however, will only be present in the absorbing element. Using this principle, parts of the piezoelectric effect can be filtered out.

What is interesting to note here, and will be further shown in the results, is that an excitation present in only one of the elements showing a positive response, will yield a negative (inverted) response when instead occurring only in the other element.

Figure 17: Setup to test (left) and principle overview (right) of compensating element for filtering microphonic effect.

Using the gripper setup, this principle was put to a test using the left figure in fig. 17. Two similar sensing elements were placed on either side of a heat emitter, both oriented in the same way and connected in series. Firstly tests were made without any absorbing or reflective surfaces, just a thin tape to keep everything in place. As the object slips downwards in the shown figure, the heat frame will affect the bottom element while moving away from the top element. At the same time, both elements should be affected by similar mechanical stress. Tests for these were done both with and without heat, and in both directions. To this setup, a reflective and absorbing surface was added on top of respective element and the same tests were done again.

Next step involved creating a sensor element applying this principle, this can be seen in fig. 18 where the left side shows how the element was built and the right side shows a front view of the setup. The build-up of the sensor element is very similar to the one created for testing the principle, with the difference that both elements are on the same side of the heating element now. The same tests done to the setup in the previous figure was performed here as well.

Figure 18: Overview of how sensor module is built with compensating sensor element to reduce microphonic effect.

Instrumentation

The instrumentation filtering was done using the Piezo Film Lab Amplifier which will be discussed in the next section. It filters the continuous signal using a high-and low-pass filter, also known as a band-pass filter. Looking at fig. 19, both these filters can be set to five different values each, spanning from 0.1Hz to 1KHz for the high-pass filter and 10Hz to 100KHz for the low-pass filter. The cut-off frequencies were chosen as 0.1Hz and 10 Hz respectively.

In the beginning longer wires (1 m) were used to read the data from the PVDF into the amplifier. When it was realised this was creating too much noise and masking parts of the target signal, the wires were shortened (1.5 dm). Tests for the microphonic effect and electromagnetic interference were performed respectively by tapping repeatedly in the table with a finger, and waving the arm near the electrodes of the wires.

4.2.5 Instrumentation & Verification

All tests regarding measurements of the pyroelectric material were done using the following two commercial instruments: Measurement Specialities Piezo Film Lab Amplifier [70] for pre-processing and Adlink Technologies[71] Data Acquisition (DAQ) module[72] as an Analog-to-Digital Converter (ADC), see fig. 19 & 20.

The amplifier has several built-in features allowing the user to eg set the gain, choose capac-itance of the feedback capacitor of the charge amplifier, filter out high and low frequencies and choose between charge and voltage mode[43]. The ADC uses a Universal Serial Bus (USB) inter-face enabling the data to be communicated directly to a laptop or Personal Computer (PC). The continuous data reading was implemented using a c# script which relayed the data to the main software in Matlab for analyzing, see section 4.4..

This setup had already been used for experiments with piezoelectric material earlier, so with some minor adjustments to the script and settings, it could be used for this work. To read the data from the FSR, an Arduino Duo was used for easy implementation as this saved time and complexity to set up compared to if the Adlink would’ve been used. The Arduino was also used

Figure 19: Piezo Film Lab Amplifier (TE Connectivity).

Figure 20: Analog-to-Digital Converter (Adlink Technologies).

Figure 21: Infrared array sensor HTPA80x64d (from Heimann Sensor Datasheet).

in parts of the characterization of the pyroelectric material. A digital switch using the Arduino General Purpose Input Output (GPIO) was set up to start and stop a stopwatch in the arduino. These were used to measure the start and stop time indices of object slippage, so the continuously read sensor data from the Adlink could be marked out with an approximation of the inception and halt of slippage. The synchronization of data between the two instruments was performed in the central Matlab application by calculating the round time delay of the message communication. To communicate a start and stop of the slippage to the arduino, the cart used during these experi-ments was set up with cupper tape which would close and open the circuit connected to the GPIO as the cart would start sliding and when stopped.

To confirm the concept of the sliding heat profile, tests were also performed using a thermal cam-era by Heimann Sensor GmbH [73]. More specifically, their infrared array sensor HTPA80x64d [74], along with their camera GUI ArraySoft V 1.25 was used for this task, see fig. 21. The setup and tests performed with this camera will be described in the next section.

4.3.

Test Setup

During the time of this work, two test rigs were developed and improved. The initial setup was made for testing the characteristics of the sensing material in order to get a better understanding and intuition about how the material acts under different circumstances. This allowed for a more controlled environment, less prone to noise from eg vibrations and movements. A second setup was then made to replicate a more real-life scenario of two gripping fingers.

4.3.1 Characterization test

To get an understanding of the material, a characterization setup was designed and used for initial testing of the three different PVDF-sheets (t: 25, 52, 110 µm). Every part of the setup was designed in CAD, laser cut as Poly(methyl methacrylate) (PMMA) (also know as Plexiglas), and assembled with screws and bolts. The tests performed using this setup will be discussed later in this section, after an understanding of the setup has been introduced. Firstly we’ll discuss the base plate where the experiment will take place as can be seen in fig. 22.

A small strip (yellow) of 50x5 mm from each sheet was cut out and carefully attached to the plate using double-sided tape. For each sensing strip, a heating element (red) is also placed before it to heat up the slipping object during the experiments. To control the direction of the slippage,

Figure 22: Top (left) and back (right) view of the characterization test setup.

the object (brown) is attached to a sliding cart (blue) which can move backwards and forwards along two railings on each side of the base-plate as seen in the back view figure above. Since all material is made of PMMA and the touching surfaces between the cart and base plate are relatively small, only a low friction will be present allowing the cart to move smoothly. The aim with this was to minimize the microphonic effect, such as vibrations in the base plate arising as the cart slides, to not interfere with the target signal. As this is for understanding the sensing material, the target signal includes both the piezo-and pyroelectric response.

Figure 23: The base characterization test setup. To make the cart move at a relatively

similar manner repeatedly, the base plate connects to two protractors (brown), one on each side, via two handles (green), as can be seen in fig. 23. Assuming an angle of 0 de-grees when the base plate is horizontal, the handles can be turned along the protractor rotating the test setup to an angle between [-70, 70] degrees, with a 1-degree accuracy. The greater the slope, the greater the accel-eration of the slipping object.

Looking at back the Top View of the setup in fig. 22, four square holes (green) were added to the initial design in case of necessary alterations/extensions of the setup. These were utilized by adding a

two-dimensional arm meant to imitate a closing grip on the object as seen in fig. 24. As the cart starts moving, the slipping object will induce a charge in the sensing material, once it reaches the arm, the object will be squeezed between the arm and the sensing material. This will cause the cart and attached object to decelerate until it reaches a stop, at the same time the pressure upon the sensing material will increase.

Using the above setup, several experiments were performed. Mainly the difference between slippage with heat and no heat was investigated. This was done by placing the object at a start position touching the heat element and releasing it with the base plate at an angle. With different angles of the base plate, also slippage at different accelerations was tested. A difference in the heat intensity was also considered by performing the experiments with no heat, and with heat turned on between 0-6 seconds before the cart starts sliding. With these parameters, tests were also performed in reverse, as seen in fig. 25, to get a better understanding of how the temperature change affects the piezoelectric output and the difference between a heated object and non-heated object.

Figure 25: Experimental setup in both directions.

To investigate the pyroelectric effect deeper, the cart was set up without a slipping object, but instead with a longer stick was attached. A heating element was attached to this stick enforcing a stable 1 dimensional movement (forward and backward) as seen in fig. 26. This way the sensing material can be isolated from the vibrations and movements of the cart. To further ensure the material is isolated from the mechanical forces created by the cart, the sensor material and the cart setup were placed on separate tables.

Figure 26: Setup of cart for investigating the pyroelectric response.

For investigating the principle of the sliding heat profile, additional modification was performed to the base plate of this setup. A hole was drilled in the middle of the plate where the thermal camera was placed to observe the sliding object from below, without interfering with any other experimental setups. An easy to remove heating element was placed on top of the casing edge surrounding the lens to see the direct heat signature as the object passes the heating element, but also to observe the heat propagation inside the object when it’s not sliding, as can be seen in fig. 27. The sliding object attached to the cart was then placed over the camera. Slippage was induced in both directions while the thermal images were observed and recorded in the ArraySoft GUI.

Tests were also performed to observe how the temperature increases as the heating element from the point of activation, and at what voltage/current the acceptable max temperature of 40 degree Celsius is reached (pain threshold for humans).

4.3.2 Gripper test

Figure 28: Experimental test setup for slip-page detection.

The final sensor solution will be tested by the use of a modeled hand wearing the sensor on its fingertip. By grasping objects of different shapes and sizes with this hand, with a grip holding the objects firmly enough not to slip, a force will be applied upon the object from above to induce a slip between the object and the glove, as similarly performed by B. Yang et al. [16]. This force will be applied both as an impulse as well as in a continuous manner. The tests will be performed both with the heating element on, as de-scribed in the above section, but also when it’s turned off. Since pyroelectric materials also posses piezoelec-tric properties, vibrational noise and other forces from both the object and the prosthesis itself will need to be investigated as well. From these experiments quan-titative data will be extracted, analyzed and then used as a basis for making a qualitative comparison. The experiments will be performed in an iterative process where empirical adjustments will be made to the post-processing of the output signals from the sensors.

4.4.

Graphical User Interface

In order to save time for running the experiments and analysing the data, some supporting tools were needed. Therefore, two separate Graphical User Interfaces (GUI) were developed. One which allowed for easier control and management of the experiments, and another for the analysis of the experimental data. Both were developed using Matlab’s built-in App designer, an environment for GUI-development. Both will be presented and discussed in this section. To better understand the connection between the two GUIs, an overview of how the data is saved will be shown.

4.4.1 Data Structure

With the easy to use support for handling xlsx -files (Excel ) in Matlab, and also human readability of the files, this became the choice for saving all the data. Each test run is saved as a sheet in an xlsx-file, so in one file a set of tests can be saved. Each run/sheet contains three different types of information as seen in fig. 29. Firstly, and most importantly, the interval of each data section is presented in cells A1-A3. This was created to easily make change to the data within the structure if needed. The data (blue) is saved vertically along with a time index in seconds, starting from zero. Lastly, there is the header (red) and the info (green). Both of these include information for the user to easily distinguish between different test runs and to also know what were the applied conditions/parameters for this specific run. These are separated as the initial data structure only contained the header along with the data, and later on the additional information was added to the structure.

Figure 29: Data structure of a test run saved in an xlsx-file sheet.

4.4.2 Experimental

The experimental GUI has two main functionalities: To read and display data of the recent test run, and to save it for later analysis. Along with these, many other secondary features are included as can be seen in fig. 30. When starting the application, the user is first prompted with a dialog box (blue) for selecting the a DAQ-module. If non is connected, the user can also choose to simply load and quickly analyse some previously saved data. The analysis part was implemented at the beginning before the analysis tool was developed.

Once a module for data acquisition is selected, the main (black) is displayed. At the left, seven buttons are displayed allowing the user to perform the following actions:

• Start Test Start reading data

• FFT Perform Fast Fourier Transform (FFT) on data acquired from test run • DWT Analysis Never implemented

• Analysis (Deprecated) • Save Data Save data to file

• Load File Select file to load test run from (Actually loaded with Load sheet

• Load Sheet Select sheet (single test run) to be loaded, and displayed, from selected file Run Test activates a separate program which continuously reads data acquired from the measuring instrumentation and relays it to the Matlab script. Once a test run is over, the data is displayed on the graph to the right. When FFT is performed, the graph will switch tab to the FFT -tab to show the corresponding graph. Additional features includes the User Log, loaded information, and Note.The first one gives feedback to the user to ensure the executed actions actually happens, and the two latter is for the user to easily see what type of experiment was loaded.

Figure 30: The main GUI used as an assisting tool when running experiments.

The loaded information is filled in by the user before saving the acquired data from a test run. This information is applied in separate tabs to the buttons (red and green). First the configurations may be changed under the config tab. These will affect the experiment by allowing the user to define the time interval of a test run, if the data should be displayed live, if the arduino and the Adlink instrument needs to be synchronized for an experiment etc. This part also includes some parts which does not make a difference for the actual experiment, but will be saved along with all other information so the user may easily understand what type of experiment was performed when eg analysing the data at a later time. The AmpParam tab allows the user to set further information which will be saved. All settings in this tab should match with the settings used on the Piezo Amplifier during a test run, see fig. 19. Once a set of test runs have been performed and saved, the experimental GUI may be closed and the analysis can begin.

4.4.3 Analysis

To analyze the saved data, a GUI allowing the user to load, display and select a data range was developed. Looking at fig. 31, a file must first be selected and then one-by-one, or all, the data can be loaded as series (data from one test run). The loaded series will be displayed in the Series selection where one or multiple series can be selected and then plotted, removed or saved. When several series are selected, operations such as plotting the average of all selected series, subtracting one series from another, or adding them together may also be performed. The Hold on checkbox allows the user to plot normal series together with series that have had operations performed on them.

At the bottom of the GUI, series may be selected and handled individually. With a default of showing the whole original series, a start-and-stop index may be selected to only show parts of the data. With this series can be aligned to simplify comparison between different test runs. There is also a multiplier which is applied to the entire series.

Figure 31: GUI used for performing offline analysis of collected data.

The Data Info tab shows all information discussed earlier so the user easily can identify what type of test was performed to acquire a series of data. Lastly, the white space above the Data Handling/Info tabs is a user log.

5.

Ethical and Societal Considerations

No humans, animals or other living beings were included in this work nor affected by it. Therefore no ethical or societal considerations are presented.