Neural Novelty — How Machine Learning Does Interactive Generative Literature

(1)

(2)

(3)

(4)

C

_C

(5)

(6)

(7)

(8)

(9)

(10)

(11)

(12)

(13)

(14)

(15)

Sets of axiomatic rules with explicit relations of signifier—signified

Categorisation and indexing of rules by the symbolic system

The symbolic system com-putes results by inferring relationships in ruleset

A retained connection between the signifier

and the signified

=

?

!

Raw, unlabelled data. The signified is implicit to the

original to human reader

Formation of neurons by mathematically inferred of patterns of signifiers

The creation of new signifiers through pattern extrapolation

The absolute dis-connect between the signifier and the signified

A state of hyperreality

=

(16)

(17)

(18)

Author _LiteratureWork of Reader → → Meta— Author Generative Literary System User— Reader → ↔

(19)

(20)

(21)

(22)

Machine Learning

Chosen Space of IxD Engagement 4.1 4.2 4.5.1 4.5.2 4.5.3 4.5.4 4.4.1 4.4.2 4.3 4.4.1 4.4.2

(23)

(24)

(25)

(26)

(27)

FIRS T user pr ompt THEN user choic e te xt gener at ion te xt displa ye d to user lo op seq→seq seq→vec input→vec seq→output

(28)

(29)

(30)

corp or a o f religious t exts te xt gener at ion transfer learning

lip syncing _animatr

onic s te xt -t o -sp e e ch sound r ela ye d thorugh sp eak er

(31)

sc enario cr eat ion pr o ce ss te xt gener at ion first round subsequent rounds te xt displa ye d to user te xt pr o ce ssing user pr ompt OR user e dits st ory te xt gener at ion te xt displa ye d to user lo op te xt pr o ce ssing

(32)

corp or a o f religious t exts te xt gener at ion , t one or c ont etn inf orme d b y user sent iment

lip syncing _animatr

onic s re al t ime f acial sent iment analy sis te xt -t o -sp e e ch sound r ela ye d thorugh sp eak er re al-time camer a captur e o f visit or

(33)

(34)

User Prompt

“I was walking”

“the cat”

“by the river”

“somewhere close” “greeting everyone”

“home” “alone.” “It was” “sunny, ” “gazing!” “raining” “my dog” “in the” “forest” “park” “in town,”

(35)

user prompts the network or explores a node te xt gene ra tion te xt displ ay ed to user lo op Implementat ion

Look and feel Role

(36)

(37)

(38)

(39)

user e xplor es a ne w no de te xt gener at ion te xt displa ye d to user asynchronous steps lo op tit le gener at ion st ory t itle up dat e d

(40)

(41)

(42)

(43)

(44)

user e xplor es a ne w no de user sele cts a neur al netw ork te xt gener at ion te xt displa ye d to user lo op tit le gener at ion st ory t itle up dat e d asynchronous steps

(45)

(46)

(47)

Te chnolo gy Aesthetics Interaction Designer Reader Engineer or Data Scientist Conventional Writer or Artist

(48)

(49)

(50)

(51)

(52)

(53)

(54)

(55)

(56)

(57)

(58)

(59)

(60)

(61)

Appendix A

Material for Neural Plumbing

sequence vector input

i→s i→v s→v v→v v→s s→s v→o s→o 1. Cut extremes of post-its to shape using the guides 2. Annotate elements [see: complementary page] 3. Arrange and re-arrange elements to form pipelines

s→s language masking summarization translation text generation text-to-seech question answering s→v named-entity-recognition sentiment analysis text filtering v→v data processing image classification image generation recommendation video object tracking video stabilization v→s image captioning recommendation video stabilization i→s/v api/database query dataset user input s/v→o display further processing sound video output