Support for Emulated 5G-System Bridge in a Time-Sensitive Bridged Network

Support for Emulated 5G-System Bridge in a Time-Sensitive Bridged Network

SHRINISH DONDE

KTH ROYAL INSTITUTE OF TECHNOLOGY

SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE

5G-System Bridge in a Time-Sensitive Bridged Network

SHRINISH DONDE

Master in Communication Systems

School of Electrical Engineering and Computer Science Date: September 23, 2020

Examiner: Markus Hidell

Academic supervisor: Peter Sjödin

Industrial supervisor: Marilet De Andrade Jardim Host company: Ericsson AB

Swedish title: Stöd för ett simulerat system med 5G-brygga i ett

tidskritiska bryggnätverk

Abstract

Time Sensitive Networking (TSN) defined in the IEEE 802.1 working group, is an important enabler for industrial Internet of things, specifically industry 4.0.

3GPP release 16 specifications includes the 5G system as a logical TSN bridge, thus promoting the integration of 5G technology with TSN. This combination provides wireless deterministic communication thus ensuring low, bounded delay and near-zero packet loss. In this thesis, we implement a 5G system in- tegration with TSN using a discrete event network simulator (NS-3). Further, we propose a simplified per egress port scheduling algorithm based on IEEE 802.1Q (scheduled traffic standard) running in the Centralized Network Con- troller (CNC). Average packet delay, average jitter, average throughput and the packet loss is measured for comparing the performance difference when our TSN scheduler is used versus when it is not. The designed system is tested by measuring it’s network impact in terms of average delay and packet loss.

The 5GS logical bridge behavior is simulated by varying the 5G bridge de- lay dynamically. For every frame transmission in the queue, the processing delay of a particular bridge is varied with pre-defined set of values. Two sets of 5GS bridge delay variations are considered, i.e. between 1-10ms and 5- 10ms respectively. On calculating the network impact, we conclude that the overall impact on the network decreases as the variation range for the delay gets smaller. This proves that higher delay variations have a significant impact whereas smaller delay variations have a negligible impact on the network. For the latter case, the system delay is considerably stable and thus can be used for industrial applications in real-life TSN scenarios.

Keywords— Bounded ultra low latency, Deterministic traffic, Egress port, Multi-

queuing, Quality of service, Scheduling, Time Synchronization, Traffic classes, Time

aware system, 5G-System bridge

Sammanfattning

Tidskritiska nätverk (TSN) definierat i IEEE 802.1-arbetsgruppen, är en vik- tig faktor för det industriella Sakernas Internet, särskilt när det gäller Industri 4.0. Specifikationer enligt 3GPP release 16 inkluderar 5G-system som en lo- gisk TSN-brygga, som främjar integrationen av 5G-teknik med TSN. 5G med TSN ger trådlös deterministisk kommunikation som säkerställer låg, begrän- sad fördröjning och nästan noll paketförlust. I denna rapport implementerar vi en 5G-systemintegration med TSN med hjälp av en diskret händelse simu- lator (NS-3). Dessutom föreslår vi en förenklad algoritm för schemaläggning av portar per utgång baserat på IEEE 802.1Q (Scheduled Traffic Standard) som körs i en centraliserad nätverks-controller (CNC). Genomsnittlig paket- fördröjning, genomsnittlig fördröjningsvariation, genomsnittlig genomström- ning och paketförlust mäts för att jämföra prestandaskillnaden när vår TSN- schemaläggare används jämfört med när den inte används. Det utformade sy- stemet testas genom att mäta nätverkets påverkan i termer av genomsnittlig fördröjning och paketförlust. 5GS logiska bryggbeteende simuleras genom att dynamiskt variera 5G-bryggfördröjningen. För varje bildöverföring varieras bryggans bearbetningsfördröjning med en fördefinierad uppsättning värden.

Två fördefinierade uppsättningar av 5GS-fördröjningsvariationer beaktas som ligger mellan 1-10ms respektive 5-10ms. När vi beräknar nätverkspåverkan drar vi slutsatsen att den totala effekten på nätverket minskar när variationen i fördröjningen blir mindre. Detta visar att högre fördröjningsvariationer har en signifikant effekt medan mindre fördröjningsvariationer har en försumbar effekt. I det senare fallet är systemfördröjningen betydligt stabilare och kan användas för tillämpningar i verkliga TSN-scenarier.

Keywords— Begränsad ultra låg latens,Deterministisk trafik, Utgångsport, flera köer,

servicenivå, schemaläggning, Tid Synkronisering, trafikklass, Tidsmedveten system,

systemet med 5G

Acknowledgements

Firstly, I would like to thank my industrial supervisor, Marilet De Andrade Jardim for entrusting me with this innovative and cutting edge technology project at Ericsson AB. Her expertise in the field of TSN was invaluable and the timely feedback provided by her pushed me to sharpen my thinking and brought my work to a higher level. I would also like to thank my Manager Patrick Sellstedt at Ericsson AB for giving me this opportunity of carrying out my thesis smoothly and supporting me in difficult times.

At KTH, I am sincerely thankful to my academic examiner, Professor Markus Hidell and my academic supervisor, Professor Peter Sjödin for helping me in fine tuning the thesis with necessary changes required to make the results bet- ter. Formulating a strong foundation for this thesis would not have been pos- sible without their help. Also, attention to details with academic view point was something that was possible only with their help and encouragement.

I feel grateful to have the company of good friends and colleagues at Er-

icsson AB and at KTH whose constant love and support helped me to achieve

better results in these tough times. Finally, I highly appreciate the support

of my family for helping me morally during each phase of my Master thesis

project and believing in me constantly.

2.1 IEEE 802.1Q Ethernet frame model . . . . 9

2.2 OSI Model . . . . 11

2.3 Fully Centralized model of TSN (adapted for thesis) . . . . 14

2.4 Transmission selection with gates . . . . 16

2.5 System architecture view with 5GS appearing as TSN bridge (Figure 4.4.8.2-1 from [25]) . . . . 20

3.1 Steps for creating 5G TSN System . . . . 27

3.2 Architecture Model . . . . 29

3.3 Scheduling transmission when the frame is about to be dequeued 35 3.4 Start time calculation for TSN scheduler . . . . 41

3.5 Traffic Periodicity and duration for different traffic classes . . . 44

3.6 Basic scheduling example using TSN algorithm . . . . 44

4.1 Exemplary test scenario using NS-3 . . . . 49

4.2 Queues in NS-3 . . . . 58

5.1 Average Delay vs Traffic Load . . . . 66

5.2 Average Jitter vs Traffic Load . . . . 67

5.3 Packet Loss (%) vs Traffic Load . . . . 68

5.4 Average throughput vs Traffic Load . . . . 69

5.5 Average Delay vs Traffic Load . . . . 71

5.6 Average Jitter vs Traffic Load . . . . 72

5.7 Average throughput v/s Traffic Load . . . . 72

5.8 Packet Loss (%) v/s Traffic Load . . . . 73

5.9 Ethernet bridge delay variation . . . . 75

5.10 Delay system impact (%) v/s Bridge delay variation . . . . 75

5.11 Jitter system impact(%) v/s Bridge delay variation . . . . 76

5.12 5G bridge delay variation . . . . 77

5.13 Average Delay v/s Load for fixed 5G delay . . . . 78

vi

5.14 Packet Loss (%) v/s Load for fixed 5G delay . . . . 78 5.15 Average Delay v/s Load for var 5G delay between 5 and 10ms 79 5.16 Packet Loss (%) v/s Load for var 5G delay between 5 and 10ms 80 5.17 Average Delay v/s Load for var 5G delay between 1 and 10ms 81 5.18 Packet Loss (%) v/s Load for var 5G delay between 1 and 10ms 81 5.19 Delay system impact (%) v/s bridge delay variation . . . . 82 5.20 Jitter system impact(%) v/s bridge delay variation . . . . 82 5.21 Fixed 5G bridge delay v/s variable 5G bridge delay v/s normal

ethernet bridge delay without 5G bridge . . . . 84 5.22 Fixed 5G bridge delay v/s variable 5G bridge delay v/s normal

ethernet bridge delay without 5G bridge . . . . 85

2.1 Definition of Traffic types . . . . 19

3.1 Frame and flow attributes . . . . 30

5.1 Simulation network parameters for NS-3 . . . . 62

5.2 Notations used in Trace File . . . . 63

5.3 Bridge delay variation impact on average delay of TSN bridged network . . . . 76

5.4 Bridge delay variation impact on average jitter of TSN bridged network . . . . 77

5.5 Bridge delay variation impact on average delay of TSN bridged network . . . . 83

5.6 Bridge delay variation impact on average jitter of TSN bridged network . . . . 83

viii

Abstract . . . . iii

Sammanfattning . . . . iv

Acknowledgements . . . . v

List of Figures . . . vii

List of Tables . . . viii

List of Acronyms and Abbreviations . . . xii

1 Introduction 1 1.1 Foundation-Time Sensitive Networking . . . . 1

1.2 Technical Concepts in TSN . . . . 2

1.3 Problem Definition . . . . 4

1.4 Purpose . . . . 5

1.5 Goals for thesis - Research questions . . . . 6

1.6 General Assumptions . . . . 7

1.7 Ethics and sustainability . . . . 7

1.8 Thesis Structure . . . . 8

2 Background 9 2.1 Ethernet and its adaptation to TSN . . . . 9

2.2 Time Sensitive Networking . . . . 10

2.2.1 IEEE 1588-2019, Precision Time Protocol . . . . 12

2.2.2 IEEE 802.1Qbv-2015, Traffic scheduling . . . . 12

2.2.3 Terms used in TSN . . . . 13

2.3 Fully Centralized Model of TSN . . . . 14

2.3.1 Working mechanism . . . . 15

2.4 Transmission gates and queues . . . . 16

2.4.1 IEEE 802.1AS-2020, Timing and Synchronization . . 18

2.5 Traffic Types in Industrial TSN . . . . 18

2.6 5G - Service Based Architecture Model with TSN . . . . 19

2.7 Functionality for 5G Support in TSN . . . . 22

ix

2.8 Related Work . . . . 23

3 Methodology 26 3.1 Research Process . . . . 26

3.2 Architecture model for the scheduler . . . . 28

3.3 Frame and flow model for TSN system . . . . 30

3.4 Calculation of Gate Control List schedules . . . . 31

3.4.1 Gating mechanism . . . . 32

3.4.2 Scheduling transmission calculation . . . . 34

3.4.3 Constraints on the scheduler . . . . 36

3.5 Scheduling Algorithm for CNC . . . . 37

3.5.1 Add event algorithm . . . . 38

3.5.2 TSN scheduling algorithm . . . . 38

3.5.3 Line-wise explanation for TSN Scheduling Algorithm 41 3.5.4 Motivational example . . . . 43

3.5.5 Discussion of the proposed algorithm . . . . 46

4 Experimental Setup 48 4.1 Basic Simulation Design - NS-3 . . . . 48

4.1.1 Topology visualiser for NS-3 . . . . 51

4.1.2 Traffic Generator . . . . 51

4.1.3 Explanation of a NS-3 Network Device . . . . 52

4.2 Queuing disciplines adopted NS-3 basic scheduling . . . . 53

4.2.1 FIFO queue discipline . . . . 54

4.2.2 Running a queue discipline in NS-3 . . . . 55

4.2.3 Enqueuing and Dequeing packets . . . . 56

4.2.4 Multi-queuing and ’tch’ function in NS-3 networks . . 57

4.3 Events and handlers . . . . 58

5 Results 61 5.1 Network Parameters . . . . 62

5.2 Evaluating the Traces . . . . 63

5.3 Randomness in NS-3 simulations . . . . 64

5.4 Results without TSN Scheduler . . . . 65

5.4.1 Average Delay v/s Traffic Load . . . . 66

5.4.2 Average Jitter v/s Traffic Load . . . . 67

5.4.3 Packet Loss(%) v/s Traffic Load . . . . 68

5.4.4 Average Throughput v/s Traffic Load . . . . 69

5.5 Results using proposed TSN Scheduler . . . . 70

5.5.1 Average Delay v/s Traffic Load . . . . 70

5.5.2 Average Jitter v/s Traffic load . . . . 71

5.5.3 Average throughput v/s Traffic Load . . . . 73

5.5.4 Packet Loss (%) v/s Traffic Load . . . . 73

5.6 Bridge delay variations - ethernet and 5G bridge . . . . 74

5.6.1 Ethernet bridge delay variation . . . . 74

5.6.2 5G bridge delay fixed . . . . 77

5.6.3 5G bridge delay variation . . . . 79

5.6.4 Comparisons . . . . 84

6 Conclusions and Future Work 86 6.1 With TSN scheduler v/s without TSN scheduler . . . . 86

6.2 With bridge delay variations v/s without bridge delay variations 87 6.2.1 Ethernet bridge delay variation . . . . 87

6.2.2 5G bridge delay variation . . . . 87

6.3 Rationale behind the research questions . . . . 88

6.3.1 Discussion on results related to 5G bridge integration with TSN . . . . 89

6.4 Limitations and Future goals . . . . 90

A NS-3 Basic Syntax 99

A.1 Discrete Event Simulator-NS3 . . . . 99

List of Acronyms and Abbreviations

AMF Application Management Function AI Artificial Intelligence

API Application Program Interface B.E Best Effort

BMCA Best Master Clock Algorithm bps bits per second

CBR Constant Bit Rate

CSMA Carrier Sense Multiple Access CUC Centralized User Configuration CNC Centralized Network Configuration CoDel Controlled Delay

DS-TT Device Side TSN Translator FIFO First In First Out

GCL Gate Control List gNB g-node B

IoT Internet of Things

IACS Industrial Automation and Control Systems IP Internet Protocol

L2 Layer 2

LAN Local Area Network MAC Media Access Control MQ Multi-Queuing

NE Network Exposure NG Next Generation

NW-TT Network Side TSN Translator OSI Open Systems Interconnect PTP Precision Time Protocol PCP Priority Code Point PCF Policy Control Function PSFP Per Stream Filtering Policy PDU Protocol Data Unit

PDB Packet Delay Budget PCP Priority Code Point

PRNG Pseudo Random Number Generator QoS Quality of Service

qdisc Queue Discipline REV Revision

RAN Radio Access Network

SMF Session Management Function SBA Service Based Architecture TAS Time Aware Shaper

ToS Type of Service

TSN Time Sensitive Networking TSC Time Sensitive Communication TT Time Triggered

TCC Time Co-ordinated Computing Tsi Ingress Time Stamp

TSCAI Time Sensitive Communication Assistance Information UE User Equipment

UPF User Plane Function

UDM Unified Data Management UDP User Datagram Protocol UDR User Defined Routing

URLLC Ultra-Reliable Low Latency Communication ULL Ultra Low Latency

VM Virtual Machine

VLAN Virtual Local Area Network

Introduction

This chapter gives an introduction to Time-Sensitive Networking. Section 1.1 describes the background of the thesis project. Section 1.2 explains the techni- cal concepts required to understand this thesis. Section 1.3 states the problem definition for this thesis and the motivation behind solving that problem. The purpose and goals for the thesis are summarized in Section 1.4 and Section 1.5 respectively. In Section 1.7, we state the ethics and sustainability policy fol- lowed throughout this thesis. Section 1.6 describes the general assumptions taken into consideration for simulating TSN topology on NS-3. Finally, the structure of the thesis report is explained using Section 1.8.

1.1 Foundation-Time Sensitive Networking

Time-Sensitive Networking is a set of Ethernet sub-standards defined in IEEE 802.1 TSN Task Group which aims to provide deterministic communication from point A to point B. The Industrial Internet of Things (IIoT) is driving changes in Industry 4.0, also known as the fourth industrial revolution. These changes present new challenges, that require various manufacturers for eg. au- tomotive industries and robot industries to improve existing real-time compute capabilities and several aspects of time synchronization and timeliness across devices in operation. Current industrial applications require precise control of execution timings and Quality of Service (QoS) capabilities for the control messages that are sent from one device to other.

Historically, real time communication has resorted to many real time pro- tocols in the field of communication technology such as EtherCAT [1] and PROFINET [2] that are used for industrial real time applications. However, they cannot support higher data rates. Above and over real time capabilities,

1

industry 4.0 requires a deterministic approach for message transfer amongst various devices. TSN plays an important role in such situations. This means that every control message has a deadline and is time-bounded. The message delivery from point A to point B must take place within a certain deadline.

Before getting into the technical details of Time Sensitive Networking(TSN), we try to understand the working of TSN with respect to high-level model of the internet and the layered Open Systems Interconnect (OSI) model. As mentioned before, TSN is an IEEE 802.1Q defined standard technology that provides deterministic messaging for standard ethernet. All the sub-standards of IEEE 802.1Q standards work on OSI Layer 2. Time Sensitive Networks are managed centrally by a controller which computes the requirements nec- essary for guarantees of very low latency, jitter and packet loss using time scheduling [3]. Real-time systems require responses with guaranteed time in- tervals. TSN is an ethernet standard where the forwarding decisions made by the TSN bridges use ethernet header contents. It can be used in any industrial environment and can carry the payload of almost any industrial application.

Prior to IEEE 802.1 TSN standards, standard ethernet did not have pure layer 2 deterministic capabilities.

5G wireless networks bring a new era of digital communication with an expansive set of new services and applications. Everyday, we rely on more and more devices with higher compute power and lower latency requirements.

Therefore, existing services must evolve to a cloud native architecture to pro- vide performance and cost-savings that these devices need. Many safety crit- ical applications such as operations of robots in factory or several automa- tion plants, require precision in their operation when it comes to delivering messages in a network. Physical processes in a system of robots are highly time-sensitive. In such cases, one cannot afford to miss the message delivery deadlines that are set by the controller entity. For eg, when we consider a sen- sor in a particular assembly line which is required to stop immediately because of an error/issue on the user side, this control message must simultaneously reach all the robots and in this case the timing guarantee cannot be missed. It can lead to a damage in the system and the system can halt indefinitely until re- pair. In such situations, TSN helps to provide reliability and safety guarantees to the system.

1.2 Technical Concepts in TSN

TSN provides new time based features for standard ethernet. This includes

standard time synchronization and deterministic network functionalities, thus

allowing networks to leverage the advantages of traditional ethernet while meeting the timing and control needs of real-time applications. We define the four key concepts that TSN is based upon as follows:

• Reliability

• Time Synchronization

• Resource Management

• Low Latency

TSN is built upon a multi-hop, multi-switch network based on ethernet. In ethernet networks, messages are transmitted between end systems as frames.

Frames are forwarded on links through switches and on a particular route from the sender to the receiver using MAC addressing. At a particular timing inter- val, a talker can only send the relevant TSN information to one other listener.

Thus, we rely on unicast messaging for individual switches operating per-port.

The TSN traffic queues are held in the individual switches egress port during transmission while waiting for the next slot in the queue to become available.

Each TSN switch has access to multiple queues and there is a time-aware gate associated with each queue. TSN traffic is filtered into a particular queue based on the transmission selection algorithm. Hence, the queuing delay for each frame depends on the priority (in case of priority transmission selection algo- rithm) and on other factors such as; number of other frames already waiting in the queue.

In Industrial Internet of Things (IIoT), all the devices must operate at the same time. Just like all the instruments in a symphony orchestra that need to collaborate with each other, they too need to communicate with each other seamlessly. If each device has its own CPU clock system, it needs to synchro- nize with the clocks of other devices in the network. Eventually, the system reaches an equilibrium where all clocks are synchronized to the Grand Master (GM) clock of the main system. To achieve this, there is a concept developed called ’Time Co-ordinated Computing or TCC’ [4]. With TCC, the messages sent from talker are time stamped and the sending time is marked. Similarly, when the message reaches the end station it is time stamped once again in accordance to the master clock.

For applications such as autonomous driving this matters a lot. Having

time synchronized networking enables the industry to make sure that every

device on the network, regardless the type of connectivity technology, uses a

common set of rules. It is just not enough to say that the network time is being

set correctly. With TCC each device executes it’s operation exactly when it is supposed to. In a smart manufacturing scenario where we have robots speak- ing to other robots the only way to make sure that they effectively communicate with one another is TSN. Thus, TSN helps in unleashing the full potential of Internet of Things devices in real world deployment.

1.3 Problem Definition

The integration of a new generation of technology supported by 5G and Time Sensitive Networking (TSN) is expected to meet the precise timing require- ments of Industry 4.0.[5, 6]. These smart factories require ubiquitous con- nectivity from devices to cloud through fully converged network. In today’s world 5G is tested in factories and the work is carried out at several levels. The integration of a 5G bridge behaving as a logical TSN bridge in Time Sensitive Networking (TSN), is expected to further impact the technology and provide key benefits that are not released in the earlier versions. This integration is specified in Release 16 version of the 3GPP documentation.

The fourth generation of industrial factories aims to make some significant changes in the way the manufacturing process is conducted such as flexibility of products and automation of data exchange. With respect to this, integrating 5G functions addresses two key issues - growing number of devices and ultra- reliable low latency communication. 5G provides wireless connectivity in an industry setting. Wireless connectivity enables new features in factories that cannot be achieved with wired technology such as tracking of moving vehicles or automated guiding of vehicles etc.

For integrating and testing 5G bridge in a TSN system, we create a simula- tion model for the integration and try to measure the impact of the 5G bridge on the overall network. Further, we create a scheduling algorithm for the con- troller of the TSN (CNC) which calculates the transmission schedules for the bridges in the network. The algorithm operates per egress port and outputs the gate control list timing intervals. A gate control list specified in [7] is as- sociated with each port and contains an ordered list of gate operations. Each separate gate operation allows the transmission of traffic within a particular traffic class queue. It is assumed in such scenarios that the implementation of the gating mechanism determines the exact time of next gate-close event from the sequence of gate events [8]

Industrial IoT requires critical control and strict requirements for the traffic

flowing across several devices. This is classified using sub-micro seconds of

latency requirements between each flow. For such a flow, time sensitive net-

working demands a time aware traffic scheduling mechanism. Specifically, the TSN task group provides a Time Aware Shaper (TAS) for fine grained control of the traffic giving desired QoS (Quality of Service) and frame pre-emption mechanisms that are suitable for traffic with deterministic end-to-end ultra low latency (ULL) requirements [9].

Thus, our aim is ’to design a near optimal scheduling algorithm that runs in the controller of a fully centralized model as defined in IEEE 802.1Qcc and to stress test the 5G system with this algorithm. For better results we also consider several other test cases which include variations in the bridge delays of ethernet and 5G bridge respectively’.

1.4 Purpose

TSN has the capability to provide the required deterministic bounded latency without congestion losses for traffic flows that are prioritized on an ethernet based network. One of the most important resource allocation features for preserving latency in TSN is the periodic control of traffic termed as ‘Time Aware Scheduling’. This is standardized in IEEE 802.1Qbv [7] where the transmission queues are time gated in every switch. This requires all the ether- net switches that are involved to always be time synchronized with one another.

Majority of projects in TSN define extensions to IEEE 802.1Q which specifies deterministic communication for bridges and bridged network. These projects address the transmission of very low transmission latency and high availabil- ity. The main motivation behind using TSN in such scenarios is it’s capability to provide deterministic services over standard ethernet. As mentioned before, smart manufacturing requires ubiquitous and seamless connectivity while si- multaneously meeting stringent time requirements [6]. Using TSN we can have a guaranteed transport of packets with low and bounded latency and low packet loss.

The purpose of this thesis is to focus on the scheduling problem of TSN network incorporating a 5G bridge as one of the logical TSN bridge in the fully centralized model of TSN as specified in IEEE 802.1Qcc. For this, we assume the TSN network topology and the Time Triggered (TT) messages are given as an input to the controller (CNC) which generates the gate list timing intervals as output for the individual egress ports. The calculated schedule is distributed to all the bridges as specified in IEEE 802.1Qbv specifications.

Standard switching fabric in TSN contains time-aware queues per egress port

of the switches and implements a standard priority based scheme which is

based on hold and forward mechanism. This mechanism will hold a frame

until the desired transmission port is free for transmission. Hence, a frame might experience queuing delay while waiting for the transmission of higher priority frames and earlier arriving frames with same priority. This leads to network congestion and thus we require an automatic scheduler to decrease the latency in the network and enable strict real-time communication for TSN information [10].

There are many advantages in using 5G with TSN when it comes to con- nected factories and industries. The integration helps in making a fully con- nected industrial communication scenario where the 5G features are used for flexibility and TSN is used for extremely low, bounded latency and near-zero packet loss. However, the main challenge in doing so is to enable the internet- working of TSN and 5G network and for the scheduler to adapt to this integra- tion[6]. In this thesis, we try to solve this particular problem.

1.5 Goals for thesis - Research questions

A 5G deployment with industrial equipment is likely to co-exist in the future and grow as a common support for industrial equipments requiring URLLC communication. Therefore the support of 5G in TSN industrial equipment is specified in Release 16 version of 3GPP documentation [11]. The main features and highlights of TSN include priority queuing along with resource allocation, time synchronization between network nodes and reliability mech- anism for redundant traffic flows. One of the resource allocation features of TSN to achieve bounded latency in case of periodic control of traffic is ’Time Aware Scheduling’ (standardized in IEEE 802.1Qbv). For this the transmis- sion queues are time gated in every switch. These features are required to support time-aware transmission of data across networks. If 5G and TSN co- exist in a factory then it is expected to do wonders for the technology operated by robots in smart factories.

With these needs in consideration, we define the goals of our thesis which make an attempt to address the problems in TSN:

• Goal 1: To simulate a simple ethernet network with TSN node and 5G- System virtual node as specified in the TSN fully centralized architec- ture of IEEE 802.1Qcc.

• Goal 2: To design a per egress port scheduling algorithm running in the

CNC (Centralized Network Controller).

• Goal 3: To evaluate average delay, average jitter, packet loss and through- put of the simulated TSN network, with and without using our proposed TSN scheduler.

• Goal 4: To evaluate the network impact on average delay and packet loss when a 5GS virtual bridge is used as a logical bridge in TSN network by introducing dynamic variations in the bridge delay values.

1.6 General Assumptions

We make some assumptions while executing the initial results. These assump- tions are as follows:

• Three queues of different priorities are implemented. These queues are defined as High, Intermediate and Low (best-effort) priority queues.

• Implementation of CNC only involves the calculation of transmission schedules whereas the path calculation is not executed. Thus, making an assumption that there is only one path available from any talker to any listener and the switches (bridges) are already aware of this path.

• All nodes are assumed to be properly synchronized.

• Scheduler considers only constant bridge delays.

• In the simulator, 5G bridge is assumed to be an simple ethernet bridge with special jitter and special delay properties. No air interface and core network is implemented in the network.

• The flows are static throughout the simulations and last till the end of the simulation time.

1.7 Ethics and sustainability

Our data collection technique strictly follow the rules of ’rights and obliga- tions of students of KTH’, ’code of honor and plagiarism at KTH’ and the

’KTH ethics policies’ [12]. First, a hypothesis of the expected TSN results are

stated. Then an extensive literature survey is carried out using IEEE and 3GPP

documentation of the relevant sections of TSN, precisely industrial TSN. This

is followed by a survey for the selection of an open source tool to be used for

simulations concerning TSN and Layer 2 networks. Then, for the configura- tion of bridges as per IEEE 802.1Qbv protocol, a scheduling algorithm is pro- posed for CNC controller. This algorithm provides the relevant information to the bridges for scheduling the TSN gates per egress port. This improves the efficiency of industrial TSN networks and provides a much faster and a convenient solution for two TSN system interacting with each other.

This thesis is an effort to integrate the features of 5G technology with TSN.

This integration is supported by the Service Based Architecture (SBA) Model of 5GS as shown in Figure 2.5. The SBA models the data and control plane in a 5G bridge. The interaction of (5GS) 5G-System with CNC and TSN end systems is also supported in the same model. Current research studies focus on implementing a cost-efficient and a sustainable development of 5G. The aim here is to provide the users with an enriching experience that comes with speed and data rate associated with 5G, while still reducing environmental footprint.

Thus, integration of 5G with TSN can be a step towards a greener future. In- dustries are improving every day and trying to build 5G with precision which demands the properties provided by TSN [13].

1.8 Thesis Structure

This thesis is structured into six main sections. Chapter 1 gives a foundation and basic information of TSN. It also defines the problem, purpose and the goals for the thesis. Basic information about the fully centralized model in TSN and details about time aware traffic scheduling are explained in Chapter 2. It gives details about the traffic types used in Industrial IoT and 5G func- tionalities in the 5G service based model in compliance to 3GPP documents.

A summary of the research method followed in the thesis is explained using a pyramid figure in Chapter 3. This section also explains the queuing dis- ciplines and traffic differentiation methods along with some important code snippets from NS-3. The most important part of this section is the per egress port priority based scheduling algorithm that we propose for TSN scheduling.

Chapter 4 explains the experimental setup used in formation of basic priority

queues using several concepts such as queuing, events and handlers, and gating

mechanism. Chapter 5 shows the graphs that are plotted for all the interest-

ing scenarios considered for comparison using NS-3 simulator. Conclusions

and Future work are stated in Chapter 6. This chapter also gives answers to

research questions that are stated in chapter 1.

Background

This chapter gives the background information that is required to read and understand the thesis in-depth. Section 2.1 starts with a general introduction to Time Sensitive Networking. The Fully Centralized Model as specified in IEEE 802.1Qcc standards is explained in Section 2.2. Information about time aware traffic scheduling is provided in Section 2.3. Transmission gates with queuing is defined and explained using Section 2.4. Section 2.6 explains the 5G Service Based Architecture Model with TSN and Section 2.7 gives information on functionality for 5G support in TSN. Finally, Section 2.5 gives an introduction to the most frequently used traffic types in industrial TSN applications.

2.1 Ethernet and its adaptation to TSN

DA 6 Bytes

SA 6 Bytes

802.1Q Tag 4 Bytes

Len/Etype 2 Bytes

Data 46 bytes - 1500

bytes

FCS  4 Bytes

DA 6 Bytes

SA

6 Bytes 802.1Q Tag 4 Bytes

802.1Q Tag 4 Bytes

FCS  4 Bytes Data

46 bytes - 1500 bytes Len/Etype

2 Bytes

TPID Priority CFI VLAN ID

Figure 2.1: IEEE 802.1Q Ethernet frame model

9

Time Sensitive Networking or TSN aims to give precision or timeliness properties to standard ethernet network. TSN is a set of standards under de- velopment by IEEE 802.1 working task group. Everyday IP equipment do not have a concept of time and even though the data is delivered from one end to the other end reliably, there are no restrictions or limitations on delay or synchronization precision. Industry 4.0 has a huge demand for strict real time communication. For real-time communication with hard, non-negotiable time boundaries and lower, bounded end-to-end transmission latency, all de- vices/sensors in a particular network must have a common time reference.

A TSN bridge is an ethernet bridge functioning at Layer 2. It identifies it’s packets from various VLAN tags that are assigned to it. IEEE 802.1Q adds a 4-byte VLAN tag between source/destination MAC address and length/type fields of an ethernet frame to identify the VLAN to which the frame belongs.

The bridge then forwards several packets over the public network according to it’s outer VLAN tag that is carried out in the packets, and learns the MAC addresses [14]. As seen in Figure 2.1 the payload size is 1500 bytes and only 4 bytes are used for 802.1Q tagging. The extra 4 bytes are for the QinQ en- capsulation.

QinQ encapsulation is used because the inner and outer VLAN tags can be used to differentiate packets based on users and services, the inner tag rep- resents a user and the outer tag represents a service (for eg. type of service - ToS) [14]. The (TPID) field is the Tag Protocol identifier of 2 bytes which indicates the frame type. In this case, the hex value 0x0800 particularly indi- cates an IEEE 802.1Q frame. A device that is incapable of handling a 802.1Q frame is discarded. The 3 bit (PRI) field is the Priority field which indicates the 802.1p priority of the frame. This value is typically in the range of 0 to 7 where a larger number indicates a higher priority. The VLAN ID on the other hand, indicates the VLAN to which a frame belongs. The VLAN ID is in the range of 0 to 4095. The values 0 and 4095 are reserved for it and therefore the available VLAN IDs are in the range of 1 to 4094. In our case, we consider three priorities denoted by high, intermediate and low (best-effort).

2.2 Time Sensitive Networking

Since Time Sensitive Networking (TSN) functions on lower layers of the OSI

Model as shown in Figure 2.2, the capabilities of TSN are combined at the

data link layer (Layer 2) of the Open System Interconnection (OSI) model

as shown in Figure 2.2. For encapsulating the TSN functionality, data link

layer enhances the mechanism to deal with transmission errors. It also reg-

Payload  Industrial Automation

Protocol A

Industrial Automation Protocol B

Payload 

IP Encapsulated IP Header UDP Header

IEEE 802.1 Standardized TSN Mechanisms

IEEE 802.3 Standardized MACs  

IEEE 802.3 Standardized PHYs  

TSN Elements Layer 5-7

Layer 4

Layer 3

Layer 2

Layer 1

Figure 2.2: OSI Model

ulates the flow of data at that layer and provides a well-defined interface to the network layer. TSN is then used to transport different high level industrial protocols for eg. OPC Unified Architecture (OPC-UA) which is a machine to machine communication protocol for industrial automation developed by the OPC foundation [15]. Further, the Figure 2.2 also shows two industrial au- tomation protocols used in a TSN scenario. Both the protocols A and B work on different architecture but follow the same principles of OSI model[16].

2.2.1 IEEE 1588-2019, Precision Time Protocol

TSN technology is designed to deliver minimum jitter using time schedul- ing for those real-time applications that require determinism and strict tim- ing requirements. TSN technology focuses on time synchronization with co- ordination and quality of service (QoS) requirements [17]. To ensure co- ordination and correlation of data transmitted from one end to the other, a com- mon clock source is required to synchronize each event precisely. IEEE 1588- 2019 Precision Time Protocol (PTP) [18] is specifically designed for time syn- chronization where the end stations exchanging information with one another match their local timing sources with one other. It also supports system-wide synchronization accuracy at sub-microsecond range with minimal network and local clock computing resources. Further, it also defines a best master clock algorithm (BMCA) which takes a decision on which clock is used for syn- chronization by comparison between the individual clocks of each device. The BMCA selects the participants that compete in the system to synchronize their clocks. The highest ranked clock is called ’grand-master’ clock and the oth- ers are called ’slave’ clocks. It also provides a fail-over mechanism where the grand-master clock is switched seamlessly to other clock, during failure, if needed. It enables a higher degree of fault tolerance and adds support for mul- tiple timescales. This ensures a higher degree of efficiency in a system where the talker and the listener follow strict bandwidth and latency requirements.

2.2.2 IEEE 802.1Qbv-2015, Traffic scheduling

The IEEE 802.1Qbv-2015 [19] defines a transmission selection algorithm us-

ing a time aware traffic scheduler. This ensures the QoS requirements in a

stream are efficiently met. Time aware gating mechanism is an important part

of TSN networks where the gate is either opened or closed based on the gate

control list defined as per the end user application requirements. A gate con-

trol list helps in configuring the states of the gates per event. The higher prior-

ity traffic is allowed to be transmitted at known intervals and traffic for lower priority is associated with the rest of the queue time. This ensures that the re- quirements of latency and bandwidth are met. To ensure that the quality of ser- vice(QoS) is maintained, the bridges and the end stations have a well-defined mechanism to distinguish time-aware flows from non-time aware flows. A per- stream filtering and policing rule is enabled by IEEE 802.1Qci [20] standards for this purpose. The Priority Code Point (PCP) field is used to determine the traffic class for a given queue and upto eight traffic classes per port are defined where each class is associated with a dedicated queue.

2.2.3 Terms used in TSN

The basic terms used to understand TSN more accurately is explained below:

[3]

• End Station/Devices: These are the source nodes and destination nodes for TSN flows. The end devices run on an application that require de- terministic communication. For our purposes, we refer to these nodes as talker and listener end stations.

• Centralized User Configuration (CUC): This is an application that com- municates with CNC and end station/devices. The CUC represents the control applications and the end devices in a TSN network. After CNC calculates the scheduling information for the bridges, CUC makes a re- quest to CNC for deterministic communication (TSN flows) with the end stations. Then CUC communicates these requirements back to the end stations for the flow of TSN information from talker and listener.

• Centralized Network Controller (CNC): CNC is the most important unit of the TSN Fully Centralized Model. It is responsible for calculating the scheduling algorithm and distributing the information to the bridges.

The bridges are then configured accordingly as specified in IEEE 802.1Qbv configuration protocol. The CNC communicates with TSN bridges and 5GS virtual bridge using a network management protocol such as NET- CONF or OPENFLOW.

• TSN bridges: Also referred to layer 2 switches, these are special bridges

capable of transmitting ethernet frames of a TSN flow on a per egress

port schedule and receiving ethernet frames of the same TSN flow ac-

cording to the scheduling algorithm calculated by CNC.

• TSN flow: It is used to describe the time critical communication be- tween end devices. Each flow has stringent timing requirements that are followed by network devices and is uniquely identified by a set of network devices. The end stations communicate their requirements to CUC and the CUC communicates it to CNC with a network configura- tion protocol.

2.3 Fully Centralized Model of TSN

Talker End Station Listener End Station

CUC 

Network Configuration

Protocol

CNC TSN Scheduling

algorithm

TSN Bridge TSN/5G

Bridge TSN Bridge

End Station Configuration

Protocol

1

4 3

2

Configuration Protocol

Network Management

Protocol

Figure 2.3: Fully Centralized model of TSN (adapted for thesis) IEEE 802.1Qcc specifies a fully centralized model of TSN allowing the CUC and CNC to know the information required by the end stations and thus allowing the bridges to meet bandwidth, latency and other TSN requirements.

The talker and the listener end station communicate their desired QoS require-

ments with the Centralized User Configuration (CUC) using an end station

configuration Protocol using a stream of data. The stream reservation proto-

col defined in IEEE 802.1Q also defines a mechanism for the talker to convey

their QoS requirements for eg, traffic class, data rate, and the accumulated

worst-case latency via a ’talker advertise’ message. This information is spe-

cific to the application that is running on the network. The function of the

CUC is to convey this group of information that it received from the end sta-

tions to the CNC. With this stream specific information from the end station,

the CUC communicates with the Centralized Network Controller(CNC) using a network configuration interface. The parameters that are transmitted in this case are; frame size of the ethernet frame, maximum latency and maximum jitter that is allowed for the TSN system [17].

The CNC after receiving all this information calculates the required trans- mission schedules and path information required to be distributed to the TSN bridges. The path calculation tells the bridges which device is connected to its egress port. The transmission schedule on the other hand, outputs the timing intervals for gate control list (GCL). These timing intervals define which gate is supposed to open/close at what interval of time of the total simulation time.

This information is also per traffic class.

The bridges are then configured as per the information from the CNC using a network management protocol. For this calculation the bridges too convey their delay (bridge delay) and the propagation delay of the link (latency) back to the CNC. For this information to reach back to the end stations, the CUC overrides the stream information from the previously sent information by the end stations. CUC then requests CNC for QoS requirements to be satisfied for the end station configuration. If all the requirements for QoS are met then the end stations are configured and the transmission of TSN information begins between talker and listener [17].

2.3.1 Working mechanism

Following are the steps executed for functioning of a fully centralized model in TSN.

• Step 1: Device on-boarding: This is where the talkers and the listeners communicate their stream quality of service requirements to Centralized User Configuration (CUC) entity using an end-station specific configu- ration protocol.

• Step 2: The CUC then communicates the stream quality of service (QoS) requirements for eg. Max frame size, max allowed latency, max. al- lowed jitter on behalf of all the talkers and listeners, to the Centralized Network Configuration (CNC) entity using a User/Network Configura- tion Interface protocol.

• Step 3: Bridge on-boarding: The bridges convey their requirements to

CNC for eg. bridge delay which is per port and per traffic class, propa-

gation delay per port, traffic class with their priorities etc.

• Step 4: For the last step, CNC performs necessary calculations to meet the stream quality of service requirements in the bridged network and calculates the transmission schedules. CNC is responsible for calcula- tion of Qbv cycle, start time, the timing intervals for the opening and closing of gates as per traffic classes and the duration for which it stays open/close.

2.4 Transmission gates and queues

Transmission Selection Algorithm

Queue for Traffic class #3 Queue for

Traffic class #2 Queue for

Traffic class #1

Transmission Selection Algorithm

Transmission Selection Algorithm

Transmission Gate

Transmission Gate

Transmission Gate

C C C

Transmission Selection 

Gate Control List

t1 t2 t3 t4 t5 t6 . . . . . t80

C C O C O C O C C C C O C O C O C C

O C C

Figure 2.4: Transmission selection with gates

Figure 2.4 represents transmission selection with gates as specified in IEEE

802.1Qbv time aware scheduling. Each packet received from the input flow

is first inspected to decide which traffic class/queue it belongs to. Then, it

is passed to a Transmission Selection Algorithm (TSA) for eg, strict priority

queuing algorithm or credit based shaper. We use the strict priority queuing al- gorithm. The queuing frame component evaluates the QoS header field of the frame to enqueue it in one of the queues according to the QoS value. Frames are then retrieved by the transmission selection algorithm. The transmission selection component is responsible for selecting the next eligible frame for transmission. TSA utilizes a gate driver mechanism that opens or closes ac- cording to the known port in a bridge. In addition to the normal checks, a verification from the TSA is also involved in checking for a frame in the traffic class queue that is not available for transmission. It checks if the transmission gate is in a closed state or if there is insufficient time available to transmit the frame currently in action before the next gate-close event associated with that queue is started.

The state of the transmission gate determines if the queued frames should be selected for transmission. For a given queue, the transmission gate can be in one of the two states, i.e either open or closed. Open gate means that the queued frames are selected for the transmission following the definition of the selection algorithm that is associated with that queue. Closed gate implies that the frames are not selected for the transmission and the gate remains closed during the process.

At a certain time, packets from only one traffic class are transmitted when the gate corresponding to that traffic class is open and the other gates are closed, to hold the frames of any traffic class other than the ones scheduled to be transmitted. The timing intervals in the gate control event list inform the gates when to open and close so that the frames of only one particular traffic class are dequeued at a particular interval of time. A particular frame gets dequeued as per these timing intervals and the scheduler constantly up- dates the time to schedule other queues from different traffic classes. The gate control list (GCL) represents entries for open and close events for each queue separately.

A queue is defined as a record of all the frames of a given traffic class

that await their transmission on a given port of a bridge [7]. Queuing a frame

awaiting transmission is equivalent to placing the parameters of a certain data

request on an outbound queue. The responsible forwarding process queues

each received frame to every potential transmission ports. The order of the

frames received on the same bridge port is determined by priority of the traffic

class that is assigned to the queue. The forwarding process provides one or

more queues for a given bridge port, each of which corresponds to a distinct

traffic class. Each frame is mapped to a traffic class using the traffic class table

and the priority of frame[8].

2.4.1 IEEE 802.1AS-2020, Timing and Synchronization

The frames at egress port of a given scheduled queue can only be eligible for transmission according to the GCL (Gate Control List) which is synchronized in time through 802.1AS time reference protocol [21]. The frames in TSN model are dequeued periodically depending on the Qbv cycle. The period for each traffic class is different. Each software queues has its own transmission selection algorithm (TSA) eg. Strict Priority queuing or Credit Based Shaper [8]. After TSA, we have the gates associated with per egress port and per traffic class. At a particular time interval one or more gates can be open but for simplicity we consider exclusive gating mechanism where only one gate is open at a time. A TSN switch has a standard ethernet switching feature that is responsible for carrying out the normal function of L2 switch and addition- ally TSN specific features. For TSN, specifically we rely on priority queue implementation and TSN time-aware gates.

2.5 Traffic Types in Industrial TSN

In this section, we define realistic traffic profiles used in industrial IoT scenar- ios. For these traffic profiles, we define the typical period, data size and level of criticality. Every traffic type has a deterministic bound which is related to it’s tolerance to jitter and packet loss in the system. Thus, five important traffic types are as follows [22]:

• Network Control: The network control messages contains control mes- sages for synchronizing time in IEEE 802.1AS - Precision Time Pro- tocol. These messages are low in volume but have critical delivery re- quirements [23].

• Event Control: The event control messages contain control messages for synchronizing events in a simulation or real-time situation. These mes- sages have high criticality and do not have tolerance to jitter in network packets.

• Cyclic Synchronous: Cyclic traffic involves a cyclic or periodic com- munication between devices. Synchronous traffic implies that applica- tions in each device are synchronized to a common time which is strictly monotonic and increasing[24].

• Cyclic Asynchronous: Asynchronous traffic typically has some toler-

ance to loss as compared to synchronous traffic. However the criticality

is high.

• Best-effort: Best-effort suffers from data loss where a high priority traf- fic uses all the allocated bandwidth. Best-effort traffic provides no guar- antee on delivery and bandwidth.

Table 2.1: Definition of Traffic types

Traffic Type Typical Size Typical Period Network Control 50-1500 bytes 5ms-1s

Event Control 100-200 bytes 10-50ms Cyclic Synchronous 50-1000 bytes 500µs-1ms Cyclic Asynchronous 50-1000 bytes 2-20ms best-effort 30-1500 bytes N.A

2.6 5G - Service Based Architecture Model with TSN

In this section, we study some of the core functions used in the service based architecture (SBA) model of 5G [26]:

• Application Function (AF): AF is responsible to receive the bridge in- formation of 5GS Bridge from 5GS, as well as register or update this information to the TSN network.

• Access and Mobility Management Function(AMF): This function is re- sponsible to receive all the information related to a session that is created within the 5G core network and it’s connections. It handles the connec- tions and Mobility Management Functions.

• Session Management Function (SMF): SMF interacts with the decou-

pled data plane and helps in creating, removing and updating the Pro-

tocol Data Unit (PDU). It also manages the session context information

with the User Plane Function (UPF). SMF provides port numbers and

MAC addresses of ethernet ports in DS-TT and NW-TT of the related

PDU session to the TSN AF via PCF.

Ericsson Internal | 2018-02-21

Logical (TSN) Bridge

AMF PCF

Device side of Bridge

(R)AN

UPF N7

N3

N2 N4

N1

TSN AF N5

SMF N11

N9 N8

UDM

N10

UE DS-TT

TSN System

NEF N33

TSN

Syst em

C-Plane

U-plane NW-TT

Bridge capabilities report Exposure

Traffic scheduling Configuration information

CNC

TSN System

TSN traffic class -> QoS flow

TSN-related information exchange

TSN traffic flow

STEP 1 STEP 2 STEP 3

STEP 4

STEP 5

PCF TSN AF as TSN Translator

Device Side TSN Translator

TSN System

NW-

TT U Plane

C Plane

Figure 2.5: System architecture view with 5GS appearing as TSN bridge (Fig- ure 4.4.8.2-1 from [25])

• Policy Control Function (PCF): This function provides policy rules for the control plane functions operating in the service based architecture model of 5G as shown in Figure 2.5. Network Slicing and Mobility Man- agement is included in this function. Support for 5G and QoS policing with the control functions is a part of policy control function.

• Unified Data Management (UDM): This is similar to the (HSS) Home Subscriber Service in 4G. However, it has an added functionality for cloud native functions and design for 5G specifically. The UDM func- tions in collaboration with UDR (Unified Data Repository) stores the customer data and the customer information.

• Network Exposure Function (NEF): NEF provides a way to securely expose 5G services to a third party. Those application functions (AF) who wish to communicate with 5G services are allowed to do so via NEF.

The 5GS - 5G System acts as one of the ethernet bridges in the TSN Fully

centralized model. The 5G System integrates with TSN system as a TSN

bridge which supports 5G capabilities. It interacts with the outside world us-

ing a TSN translator implemented on both the sides of the user plane virtual

tunnel formed between the user equipment (UE) and the user plane function (UPF). The device side (DS) of the bridge where a TSN System is located consists of Device Side Translator [DS-TT] whereas the network end consists of the Network Side Translator [NW-TT]. The 5G System is transparent to the other wireless links and the Radio Access Network System or (RAN) system.

This transparency is achieved by ingress and egress ports of DS-TT and NW- TT which optionally support per-stream filtering policy (PSFP) as defined in IEEE 802.1Q. The 5GS bridge is composed of ports on a single UPF side, the user plane tunnel between UE and UPF, and the ports on the DS-TT side. A protocol data unit (PDU) session is established with these ports which pro- vides connectivity for a TSN bridge. The basic structure of the system in TSN supporting 5G is shown in Figure 2.5.

The systems explained in Figure 2.5 and Figure 2.3 are considered as ’time aware systems’. The TSN translators at the edges support 5G operations such as UE, gNodeB (gNB), UPF, NW-TT and DS-TT which are synchronized with the 5G internal clock. This is based on two different synchronizations, the fifth generation system (5GS) synchronization and the TSN domain synchroniza- tion. The 5GS synchronization is used for next generation (NG)-RAN and the latter is used for the TSN synchronization. In this system, the message from a TSN bridge is first received on the ingress port of the NW-TT. This message is marked with an ingress port Timestamp (TSi). The link delay for the re- ceived message expressed in the 5GS time is calculated by the cumulative rate ratio (CRR) which is the ratio of frequency of the 5GS clock to the frequency of a TSN Clock. The payload of the message is then modified according to the specifications and the ingress time stamp is added to the suffix field of the payload.

The next step is to forward the message to the UE via user plane. This is

done using user plane function (UPF). The 5G internal clock is made available

to all the user plane nodes in a 5G System. The job of the message received

by UE is to forward it to DS-TT which then creates an egress time stamp (Tse)

as per the generalized precision time protocol (gPTP). The time spent within

the 5G system is the difference between the timestamp of ingress and egress

ports. This time is converted to the TSN time for other TSN systems to com-

municate with 5G system. This is done by a rate Ratio contained inside the

gPTP message payload.

2.7 Functionality for 5G Support in TSN

This section explains the interaction of a Centralized Network Controller (CNC) with 5GS virtual bridge. 5G Time sensitive communication is a service that supports wireless deterministic communication with high reliability and avail- ability [25]. The entire 5G-system is considered as an IEEE 802.1AS time aware system. In the Figure 2.5 CNC is connected to TSN Application Func- tion (AF) on the control plane side and the user plane function (UPF) on the user plan side.

In step 1, the ethernet bridges communicate their QoS requirements and the bridge specific information such as bridge ID (MAC addresses of bridges that identifies them), name of the bridge, number of ports, bridge delay (per port and per traffic class) and propagation delay per port to the CNC. Next, in step 2 the CNC responds to this information from the bridges with the calculation of traffic scheduling configuration/transmission information as per the TSN algorithm running in CNC and provides it to the respective TSN bridges. The bridges are configured accordingly where the configuration information per stream and configuration information of scheduled traffic is provided by the ports on the device and network side, DS-TT and NW-TT.

In step 3, the Policy Control Function (PCF) and the Application Function (AF) determine the traffic class for the traffic and segregates the traffic based on the priority. They decide the traffic class to which a frame belongs and help in fulfilling the QoS requirements per flow. Then in step 4, the UPF and UE start communicating the TSN information with each other. This information contains the QoS requirements of the end stations communicated to the AF which acts a TSN translator. Note that UE, AF and UPF have a TSN translator connected to them as shown with the acronym ’TT’ which enables them to exchange TSN information with one another. TSN Translators (TTs) at the edges of the 5G system need to support IEEE 802.1 AS operations. UE, gNB, UPF, NW-TT and DS-TTs are synchronized with the 5G-GM (5G grand master clock) which keeps these network elements synchronized [27] with each other.

Finally, the TSN traffic starts flowing from the rest of the TSN enabled

system to CNC and vice versa as shown in step 5. During this time, the TSN

ports are bound to the Protocol Data Unit (PDU) session which is connected

between UE and UPF. All PDU Sessions which connect to the same TSN net-

work via specific UPF are grouped into a single virtual bridge. To ensure that

the communication in a 5G system is always periodic and deterministic, the

nodes support ’hold and forwarding’ buffering mechanisms for DS-TT and

NW-TT respectively. Hold and forwarding mechanism is sometimes referred

to as store and forward mechanism where the information desired to be send to next station is held for a fixed time and then transmitted to the required destination when required [25].

For a 5G-system, the frames with lower priority cannot be transmitted be- fore all higher priority frames finish transmission to avoid packet delay and packet jitter. The hold and forward mechanism avoids jitters that have tra- versed in the 5G-system during communication and fulfills the QoS require- ments of the system. In this way, the 5GS bridge can participate transparently in the TSN network as if it is a part of it. Hold and Forward mechanism allows the packet delay budget (PDB) based 5GS QoS to be used for Time Sensitive Communication (TSC) traffic packets since they only arrive at the NW-TT or the DS-TT egress before their scheduled transmission. When the 5GS system participates in the TSN bridge system, the 5GS system is supposed to provide the 5GS bridge delays for each port pair of the traffic class [6],[5].

The TSN Application Function (AF) obtains the (Per Stream Policy Fil- tering) PSFP parameters and uses them to calculate traffic patterns that are responsible for forwarding the traffic as per QoS requirements. The TSN Ap- plication Function (AF) is also capable of merging multiple TSN streams of same periodicity and same class. There can be one or more 5GS TSN bridges in a TSN fully centralized system. The port information for DS-TT Side ports and NW-TT side ports are conveyed to the TSN AF and correspondingly to the CNC to register the bridge. Since there are several ethernet devices that are connected to DS-TT and NW-TT, they support a link layer discovery pro- tocol for configuration of the bridge [27], [11]. The 5G virtual bridge maps the information from TSN network to a 5G network for QoS scheduling and efficient time aware scheduling.

2.8 Related Work

In this section we explain some of the previous work that has been done on scheduling of Time Sensitive Networking and integration of 5G with TSN.

The industrial paper from Bosch from Ginthor et al. [28] is the most rele-

vant to this work because it attempts to integrate the features of 5G and carries

out an analysis of multi-user scheduling in industrial applications. The authors

explain the specifications of Release 16 version of 3GPP standards required

for 5G support in TSN and the support for ethernet protocol data unit (PDU)

sessions that are established between the UE and a core network. Instead of

using IP address for its operation it uses MAC source and MAC destination

addresses. For differentiating traffic classes in TSN and 5G they use internal

framework based on QoS flows.

The system level model used in this paper is based on OMNEST simulator with an INET framework which supports VLAN tags and traffic shapers like strict priority TAS mentioned in IEEE 802.1Qbv. For the 5G part, the authors have created their own framework based on a previous 4G-LTE system level simulator. This framework uses a strict prioritization policy that assigns re- sources to each TSN packet in a first come, first served manner. As per 3Gpp, it uses time sensitive communication assistance information (TSCAI) to pro- vide details about cycle time and scheduling offset. Remaining resources such as best-effort traffic are scheduled using proportionally fair algorithm (P.F).

A CQI index is used to measure the signal strength based on SINR per UE and gives an analysis of worst-case latencies under different channel condi- tions and the total cell throughput. The system is tested using mix traffic and utilises full buffer capacity for the TSN system.

The technical report by Michael Lander Raagaard and Paul Pop as given in [29] aims to solve combinatorial optimization problems related to communi- cation networks for distributed real-time safety-critical systems. It also takes into consideration the missed deadlines in the context of robots that could lead to damaged equipment or even endanger the lives of workers near the produc- tion line. Thus, the scheduling algorithm calculates the GCLs for all egress ports make up a deterministic schedule for when to send TT frames on links.

The optimality of this schedule is calculated w.r.t no.of queues and end to end latency in the network.

Santos et al. in their paper [30] have discussed the importance of solving the problem of multiple schedules for a Time Sensitive Network based sce- nario. TSNSched reduces the TSN Scheduling problem to an SMT Solver which is solved using a Z3 SMT solver. This delivers a quick solution and provides multiple scheduling solutions for the stated problem. A tool named

’TSNSched’ is created based on JAVA APIs to evaluate real size and realistic networks based on TSN. This tool enables scheduling per switch and param- eters such as switch queue time slots, cycle times and priority queues associ- ated with each flow. The paper runs several experiments with a maximum of 10 TSN switches and 50 physical devices. TSNSched encodes both the Tx of individual packets and the time window in which the packet is placed.

Craciunas at al. in their paper [31] address the computation of fully de- terministic schedules for 802.1Qbv-compliant multi-hop switched networks.

The factors responsible for affecting real time communication such as a bot-

tleneck and the constraints in real-time networks are tested in this paper. The

real time communication with respect to IEEE 802.1Qbv scheduling is also