imaging and machine learning

Applications of hyperspectral

imaging and machine learning

methods for real-time

classification of waste stream

components

Martin Sevcik1 Jan Skvaril1

Elena Tomas Aparicio2

1_{Future Energy Center, Mälardalen University, Västerås, Sweden}

2_{Mälarenergi AB, Västerås, Sweden}

17th _{September 2019}

19th _{International Council of Near Spectroscopy Conference, Gold Coast, Australia}

Would you like to discover the inner soul of the materials and processes through exciting real-time analytical techniques?

JOIN OUR NEW ONLINE COURSE AT MÄLARDALEN

UNIVERSITY!

Applied Spectroscopy for Future Energy and Environmental Systems https://www.mdh.se/utbildning/livslangtla rande/futuree/applied-spectroscopy-for- future-energy-and-environmental-systems-1.118583?l=en_UK Information (www)

Introduction

● Combined heat and power production

● Municipal solid waste → Refuse-derived fuel (RDF) ● Fuel is highly variable: → Process instabilities

● Increase formation emissions

● Lower efficiency CONTROL SYSTEM Steam to the turbine Flue gas

for further treatment

Ash

used bed material

Steam boiler

Circulating fluidized bed

Refuse derived fuel

Composition Moisture content, Ash content, Heating value, Other properties Paper and carbord 28% Others 9% Textiles 9% Cellulose 6%

Introduction

for further treatment

Ash

used bed material

Steam boiler Circulating fluidized bed Single-point NIR sensor Real-time measurement of fuel properties

Refuse derived fuel

Composition Moisture content, Ash content, Heating value, Other properties

● Previously installed single-point NIR sensor → Provide good conditions for feed-forward control

CONTROL SYSTEM

Steam

to the turbine

Flue gas

for further treatment

Ash

used bed material

Steam boiler Circulating fluidized bed Single-point NIR sensor Automatic pre-sorting? Hyperspectral imaging NIR sensor? Real-time identification of components and measurement of fuel properties

Refuse derived fuel

Composition Moisture content, Ash content, Heating value, Other properties ← Further upstream

Introduction

Spectroscopy workflow

(whole image) False colour images Model performance evaluation Internal validation set SENSITIVITY 1 - SPECIFICITY Overall accuracy Model performance evaluation external validation set SENSITIVITY 1 - SPECIFICITY Overall accuracy Prediction on internal validation data set Computational time Selected pre-treatment Data pre-treatment (various methods) Training set - 1000 pixels

per material category

Internal validation set – 1000 pixels per material category

Sampling and sample praparation

Category Number of

sample objects Examples of the samples

1. Paper 22

Carboard boxes, paper tissues, napkins, printing paper, notebooks, recycled paper, newspapers, magazines

2. Incombustibles 12 Ceramics, cups, stones, glass bottles, metals 3. Food 7 Dried fruits, pasta, rice

4. PE 12 HDPE – shampoo bottle, bottle caps_{LDPE – plastic bags, food packages} 5. PET 4 Plastic bottles, food containers 6. PP 5 Food containers, straws 7. PS 5 Expanded foam, plastic cutlery

8. PVC 2 Tubes

9. Textile 4 Old clothes, curtains

10. Wood 6 Wood chips

11. Background - Carbon black styrene-butadiene rubber

● Sampling from the RDF and other sources…

PET HDPE Textile PS PP PVC Paper LDPE

Glass Paper hygienic

Ceramics Cardboard

Wood Print recycled

Print white Food

● Samples identified by FT-IR Spectrometer in ATR mode, followed by library search:

● Common materials

● Polymers, Polymer Additives & Plasticizers

● NIR-HSI Spectral data push-broom camera

● 900 - 1700 nm, InGaAs detector,

● 224 spectral bands, 640 spatial pixels

● Framerate 300 fps, 5 ms exposure time

● Approx. 1200 frames per image

● Two linear halogen light sources

● Laboratory scanning table 20 cm x 40 cm

● 90 mm/s, translational movement Scanner table translational movement Sensing area width approx. 18 cm Hyperspectral imaging camera (900 – 1700 nm) Illumination source 2 Illumination source 1 NIR-HSI DATA ACQUISTION Samples

Data acquisition

Data processing and chemometrics

● Decomposition of the hyperspectral cube, calculations, pre-processing and modelling were done using scripts programmed in MATLAB

● Removing bands based on signal-to-noise ratio → 192 bands

● The training data set and internal validation set included different randomly selected data points (i.e. pixels), 1000 per material category

● Relative absorbance - converted to relative absorbance based on raw reflectance image from the sample, dark reference image and white reference image.

● Spectral data pre-processing

● Mean centering (MC),

● Standard normal variate (SNV),

● Savitzky-Golay smoothening (SGS, 2nd _{order polynomial, 11 smoothing points)}

Chemometrics and model evaluation

● Classification algorithms used:

● Partial least-squares discriminant analysis (PLS-DA),

● Support vector machine (SVM),

● Radial-basis neural network (RBNN),

● Model performance evaluation:

● Sensitivity (true positive rate)

● 1- specificity (false positive rate)

● Parameters are based on whether prediction is a true positive (TP), true negative (TN), false positive (FP), or false negative (FN).

● Calculated for each class separately. Overall classification accuracy

of the model is then expressed as a mean value of sensitivities for all classes.

Predicted class A B T ru e cl ass A TP (true positive) FN (false negative) B FP (false positive) TN (true negative)

Results and discussion

● FT-IR analyses reveals that samples in each group contained various amounts of plasticizers and

other additives making the identification challenging.

● Data pre-processing has significant impact on resulting overall accuracy of classification models.

Pre-processing method PLS-DA SVM RBNN

No pre-processing 64% - 90.6%

SGS 64.3% - 90.4%

SGS + mean centering 74.7% 92.4% 89.9%

SGS + SNV 88% 92.6% 89.6%

SGS + 1st_{derivative 85.1% 93.8% 90.7%}

SGS + 1st_{derivative + mean centering 85.8% 94% 90.3%}

SGS + 1st_{derivative + SNV 83.2% 91.5% 88.3%}

SGS + 2nd_{derivative 84.4% 91.1% 88.8%}

SGS + 2nd_{derivative + mean centering 84.9% 91.5% 88.6%}

SGS + 2nd_{derivative + SNV 82.9% 87.2% 81.8%}

4 best pre-treatments used in prediction on external validation set

Results and discussion

Class

PLS-DA SVM RBNN

Sensitivity 1 - Specificity Most confused _with Sensitivity 1 - Specificity Most confused _with Sensitivity 1 - Specificity Most confused _with Backgr. 88.3% 4.0% incombustibles 97.8% 2.2% - 98.9% 4.0%

-Paper 74.3% 0.3% textile 97.5% 0.5% - 95.0% 1.3%

-Incomb. 60.9% 2.5% background 78.7% 3.0% background 64.4% 1.8% background

Food 98.4% 1.6% - 98.7% 0.0% - 96.4% 0.0%

-PE 89.5% 0.2% - 86.0% 0.2% incombustibles 85.3% 0.1%

-PET 97.3% 0.1% - 94.5% 0.0% - 91.6% 0.0% background

PP 90.4% 0.5% incombustibles 95.1% 0.1% - 92.2% 0.2%

-PS 73.7% 0.4% incombustibles 85.0% 1.0% incombustibles 86.0% 2.6% background

PVC 99.2% 0.1% - 97.9% 0.0% - 95.2% 0.0%

-Textile 97.8% 2.1% - 97.8% 0.1% - 94.7% 0.3%

-Wood 98.2% 1.3% - 98.2% 0.1% - 97.9% 0.0%

-● Detailed classification results for best preprocessing for each of the classification methods -internal validation set

Wood

Results and discussion

● Prediction of classes by PLS-DA model (external validation set)

Class

PLS-DA

Sensitivity 1 - Specificity Most confused with Backgr. 86.0% 7.1% -Paper 78.7% 4.2% textile Incomb. 34.8% 1.6% paper Food 98.0% 0.2% -PE 75.6% 0.3% background PET 66.0% 0.1% background PP 88.5% 0.3% background PS 91.9% 3.5% -PVC 74.9% 0.1% background Textile 97.7% 5.3%

-Pre-processing method Accuracy Classification _time

SGS + SNV 77.3% 1.9 s SGS + 1st_{derivative 81% 2.1 s} SGS + 1st_{derivative + MC 80.9% 1.9 s} SGS + 1st_{derivative + SNV 75.7% 1.9 s} Background Paper Incombustibles Food PE PET PP PS PVC Textile PET HDPE Textile PS PP PVC Paper LDPE

Glass Paper hygienic

Ceramics Cardboard

Wood Print recycled

Print white Food Metal

Results and discussion

● Prediction of classes by SVM model (external validation set)

Class

SVM

Sensitivity 1 - Specificity Most confused with Backgr. 28.7% 0.8% PS Paper 98.6% 2.5% -Incomb. 78.9% 8.8% PS Food 99.0% 0.2% -PE 86.4% 1.6% incombustibles PET 80.1% 0.2% PS PP 93.7% 0.6% -PS 94.5% 17.6% -PVC 86.1% 0.2% incombustibles PET HDPE Textile PS PP PVC Paper LDPE

Glass Paper hygienic

Ceramics Cardboard

Wood Print recycled

Print white Food Metal Background Paper Incombustibles Wood PET PP PS

Pre-processing method Accuracy Classification time

SGS + SNV 83.7% 228 s

SGS + 1st_{derivative 84.9% 219.8 s}

SGS + 1st_{derivative + MC 85.1% 202.7 s}

15

Results and discussion

PET HDPE Textile PS PP PVC Paper LDPE

Glass Paper hygienic

Ceramics Cardboard

Wood Print recycled

Print white Food Metal

● Prediction of classes by RBNN model (external validation set)

Class

RBNN

Sensitivity 1 - Specificity Most confused with Backgr. 79.0% 4.5% PS Paper 95.0% 3.0% -Incomb. 58.3% 3.0% PS Food 96.4% 0.1% -PE 82.5% 0.3% incombustibles PET 77.9% 0.3% background PP 89.1% 0.2% background PS 97.1% 7.8% -PVC 77.3% 0.1% background Textile 82.4% 0.4% paper Background Paper Incombustibles Food Wood PE PET PP PS PVC Textile

Pre-processing method Accuracy Classification time

SGS + SNV 75.4% 14.9 s

SGS + 1st_{derivative 83.5% 17.7 s}

SGS + 1st_{derivative + MC 84.6% 18.4 s}

0,10 0,08 0,06 0,04 0,02 0 -0,02 -0,04 -0,06 -0,08 0,12 PE PET PP PS PVC R egre ss ion coeffi ci ent s (-) 0,04 0,02 0 -0,02 -0,04 Background Paper Incombustibles Food Textile Wood

Most influential bands for classification

● Regression coefficients in PLS-DA model

● Pre-processing: SGS + SNV

● Most influential range for synthetic organic

polymers: ● 1100 -1250 nm

● 1350 -1450 nm.

● Most influential regions for classification of

food, textile, paper and wood: ● 1450 -1490 nm. R egres sio n coeffi cie nt s (-)

Conclusions

● NIR-HSI shows good potential for development of a classification model that can recognize most common components of MSW→RDF.

● Models based machine learning methods can successfully classify NIR-HSI pixels belonging to textile, wood, food, paper, and most common types of plastics with reasonable accuracy.

● Classification of materials such as metals, glass or ceramics i.e. not NIR-active materials is more challenging.

● SVM based model - accuracies of up to 94% for internal val., 85% for external val. and was the best in

identifying incombustible materials, but highly demanded on computational time.

● RBNN based model - accuracy 91% for internal val. and 85% for external val., 10 times faster than SVM ● PLS-DA based model - accuracy 88% for internal val. and 81% for external val., 100 times faster than SVM. ● Applicability of the developed models can be further enhanced by including new samples of varying composition

Thank you

17th _{September 2019}

19th _{International Council of Near}

Spectroscopy Conference, Gold Coast, Australia

Jan Skvaril

Email: jan.skvaril@mdh.se

Mobile: +46-73-6620977 Future Energy Center

Mälardalens University, Västerås

Would you like to discover the inner

soul of the materials and processes

through exciting real-time analytical

techniques?

JOIN OUR NEW ONLINE

COURSE AT MÄLARDALEN

UNIVERSITY!

Applied Spectroscopy

for Future Energy and

Environmental Systems