Anomaly Detection and Revenue Loss Estimation in Accounting Data

IN

DEGREE PROJECT ENGINEERING PHYSICS, SECOND CYCLE, 30 CREDITS

STOCKHOLM SWEDEN 2020,

Anomaly Detection and Revenue Loss Estimation in Accounting Data

GUSTAV EDHOLM

KTH ROYAL INSTITUTE OF TECHNOLOGY

SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE

d R R^d

x

Anomaly Normal

!_!

!_"

[0, 1] [1, 0]

(x, y) y

x x

i x

y x_i y_i

j x^j

R

x ∈ R^d y ∈ R^k

y y x

ˆ

y y

y x y

t |y − ˆy| |y − ˆy| > t y

ˆ y > y

LR(x) =

Ax + b A k × d b ∈ R^k

mse(y, ˆy) =

!k

i=1(yi− ˆyi)²

k .

0 2 4 6 8 10 x

0 1 2 3 4 5 6 7 8 9 10

y

x ∈ R y ∈ R

(x, y) y = bxˆ

x yˆ f (x) = ˆy

x ˆ

y x

x yˆ

N {(x^j, y^j)}^Nj=1

xi

y σy {(x^j, y^j)}^kj=1

!_!

!_" !_#

!"

!"

!"

!"

!

!_! < c_"

!_! ≥ %_"

!_# < %_$

!_# ≥ %_$

!_% ≥ %_& !_% < %_&

x ci

ˆ yⁱ

{(x^j, y^j)}^Nj=k+1

ˆ y^l

l {y^j}ⁿj=m

ˆ y^l=

!n j=my^j

n− m + 1 = µ^l_y.

σ^l_y/µ^l_y t

K

{fk}^Kk=1 = φ yˆ

{ˆy^j}^Kj=1

L

ˆ

y y

(x, f (x))

∂f

∂x η

x^t+1= x^t− η∂f

∂x(x^t)

"K k=1

Ω(fk)

Ω(fk) = γTk+ 1

2λ||w^k||²,

L(φ) = l(y, ˆy) +

"K k=1

Ω(fk)

l y yˆ Tk

k wk γ

λ

t ft

L^(t) =

"N i=1

l(yⁱ, ˆy^i,(t−1)+ f_t(xⁱ)) + Ω(f_t)

ˆ

y^i,(t−1) yⁱ

t− 1

L^(t) $

"N i=1

[l(yⁱ, ˆy^i,(t−1)) + gift(xⁱ) + 1

2hif_t²(xⁱ)] + Ω(ft)

gi hi l

gi = ∂_yˆi,(t−1)l(yⁱ, ˆy^i,(t−1)),

hi = ∂_y²_ˆi,(t−1)l(yⁱ, ˆy^i,(t−1)).

t

L^(t) $

"N i=1

[g_if_t(xⁱ) + 1

2h_if_t²(xⁱ)] + Ω(f_t).

η < 1

x

F (x, θ) = ˆy hⁱ

ˆ y

Hⁱ θⁱ

Mⁱ Aⁱ

Hⁱ hⁱ x

h¹ = H¹(x),

h² = H²(h¹), ...

ˆ

y = H^l+1(h^l).

l

hⁱ = Hⁱ(hⁱ⁻¹) = Aⁱ(Mⁱhⁱ⁻¹)

Mⁱ Aⁱ

x L(ˆy)

ˆ y

y x

y yˆ

Hidden layers Input layer

Output layer

Weights

!

" #$

{xⁱ}ⁿi=1 n

{F (xⁱ, θ)}ⁿi=1 {ˆyⁱ}ⁿi=1

{L(ˆyⁱ)}ⁿi=1

θ

∂L

∂θⁱ

η

θⁱ ← θⁱ − η_∂θ^∂Li

argminθ[L(F (x, θ))]

x y

h

ReLU (hⁱ) = max(0, hⁱ)

h^l

sigmoid(h^l) = e^hl e^hl+ 1. k

sof tmax(h^l) = e^hl

!k j=1e^hlj,

CE(ˆy, y) =−

"k i=1

yilog(ˆyi).

ˆ y = h^l

θ

x

k k

k

{mⁱ}^ki x

mⁱ mⁱ

mⁱ

Sⁱ ={x^p;||x^p− mⁱ||² ≤ ||x^p− m^j||²,∀j, 1 ≤ j ≤ k},

mⁱ

mⁱ = 1

|Sⁱ|

"

x^j∈Sⁱ

x^j.

k = 2

k

P (x) =

z d

k

k

Z = XW

Z W k X^!X

X x

P^#(z) = ˆx

x ˆ

x = P^#(P (x))

1 d

"d i=1

(xi− ˆxi)².

z k

Encoder Decoder

#"

"

%

z = E(x) x = D(z)ˆ

x

x z x xˆ

T

T P R = T P T P + F N.

F P R = F P F P + T N.

P recision = T P T P + F P.

N

F P R = 0 N F P R = 1 N

T P R > F P R

T P R ≈ F P R

0.5

> 0.5

k

z

k

α

k

k

k

y ˆ

y

k

k

Data

Acquisition Data

Pre-processing Model

Selection Model

Training Model Testing

t

t

p_c t

pt

T pt− pc

p_t > T

T

pt

R_c Rt

Rc =

"N i=1

pⁱ_ctⁱ

Rt=

"N i=1

pⁱ_ttⁱ.

Rt− R^c = 8.5

pc > pt

d = 1473 N = 9813

F (x) = ˆy

[0, 1]

pc pt

n

ˆ

y pc

ˆ y > pc

|pc− ˆy| > T T

ˆ y < pc

max(F P R)× max(T P R)

Rp Rp−Rc

ˆ y t

Rp =

"N i=1

ˆ yⁱtⁱ.

ˆ

y t

µyˆ =

!N i=1yˆ N

σyˆ =

 !N

i=1(ˆyi− µyˆ)²

N .

t yˆ t

p = ˆyt

µp = µyˆµt

ˆ y

t σ_x² = V ar(x) µx = E(x).

σ_p² = V ar(ˆyt)

= E(ˆy²t²)− (E(ˆyt))²

= V ar(ˆy)V ar(t) + V ar(ˆy)(E(t))²+ V ar(t)(E(ˆy))²

= σ_y²_ˆσ_t²+ σ_y²_ˆµ²_t + σ_t²µ²_ˆy

= (σ_y²_ˆ+ µ²_yˆ)(σ_t²+ µ²_t)− µ²yˆµ²_t p

p

µRp = N µp =

"N i=1

pⁱ,

σRp =»

N σ_p² = σp

√N .

n

pt

ˆ

y∈ [0, 1]

pt

pc

pc

λ = 0.1 α = 5000

x d

d d = 1

d = 1

ˆ

ySGBDT yˆN N β γ

ˆ

yens = β ˆySGBDT + γ ˆyN N.

β γ β = 0.3

γ = 0.7

n n =

0.5 0.6 0.7 0.8 0.9 1 AUC score

RLT Ensemble RLT SGBDT regression RLT Random Forest regression RLT NN regression ELT NN regression RLT PCA ELT Binary NN classifier

0 5 10 15 20 25 30 35 40 0

0.05 0.1 0.15 0.2 0.25 0.3 0.35

RLT NN Regression predicted revenue distribution RLT RF and SGBDT predicted revenue

Charged Revenue Target Revenue

t

t

t t

www.kth.se

TRITA-EECS-EX-2020:929