Orders From the Cloud : Business Integration as a Service

(1)

ORDERS FROM THE CLOUD

BUSINESS INTEGRATION AS A SERVICE

Karl Hammar

MASTER THESIS 2009

INFORMATICS

(2)

Detta examensarbete är utfört vid Tekniska Högskolan i Jönköping inom ämnesområdet informatik. Arbetet är ett led i teknologie magisterutbildningen med inriktning informationsteknik och management. Författarna svarar själva för framförda åsikter, slutsatser och resultat. Handledare: Ragnar Nohre Examinator: Vladimir Tarasov Omfattning: 30 högskolepoäng (D‐nivå) Datum: 2009‐08‐21 Arkiveringsnummer:

(3)

Abstract

This thesis describes the development of a SOA‐based architecture for integrating large EDI‐using manufacturing companies purchasing processes with smaller non‐EDI capable manufacturing companies, using online services. The underlying need for this project lies with the fact that these small manufacturing companies risk to miss out on business, due to their inability to communicate via the industry standard EDI format. At the same time, getting EDI‐ capable involves significant investments in software licenses, connectivity services and consulting or training that these small companies may not be ready to make. The mentor company of this thesis project, System Andersson, produces resource planning software for this type of companies and would like to be able to provide them with an easy to use way of “jacking in” EDI support into their businesses, without having to make such significant investments. Ideally this feature would be developed as a standalone subscription based service that could be provided to users in such a way that their existing System Andersson software could connect to it and no further hard‐ or software would be needed on site. This EDI‐enabling should be as easy as to be entirely transparent for the end‐user companies. The task handed to the author was thus to develop an architecture for how such a subscription based service could be developed. Furthermore, in order to promote re‐use and simplify development, the architecture was to be based on SOA concepts. As a result of the project, such an architecture has been developed. The architecture details two services for translating and storing for later delivery a number of EDI message types of the EDIFACT variety. The architecture also specifies communications protocols (SOAP over HTTPS and AS2 over HTTPS) and APIs (web services) for how to communicate with these services. These specifications can be used to implement a system that performs the necessary integration, so that the smaller companies may indeed communicate via EDI. The fitness of the developed architecture has been tested by implementing a prototype version of such a system based on it. It has also been validated by way of comparing to how well it adheres to SOA design principles. All in all, this design appears to be quite sound and presents a working solution to the studied problem.

(4)

Sammanfattning

Den här uppsatsen beskriver utvecklingen av en SOA‐baserad arkitektur för att bygga integration av mindre och större bolags sälj‐ respektive inköpsprocesser. Den underliggande problemställningen är att de mindre bolagen riskerar att missa potentiella affärsmöjligheter på grund av att de inte kan ta emot ordrar och andra affärsdokument i EDI‐format, vilket de större bolagen ofta kräver. Samtidigt så kan det vara mycket kostsamt att implementera EDI‐stöd i en organisation, eftersom det ofta krävs såväl mjukvarulicenser som anslutningar till nättjänster och konsulthjälp, varför de här små bolagen kan dra sig för att investera i EDI‐kompatibilitet. Det här examensarbetets mentorföretag, System Andersson, utvecklar materialplaneringssystem för företag i segmentet mindre producerande bolag. Man har önskat se en lösning på det här problemet varvid det skulle bli enkelt för System Anderssons kunder att ”jacka in” EDI‐stöd utan att behöva ha så stora kostnader. Helst skall denna lösning fungera som en fristående online‐baserad prenumerationstjänst, till vilken System Anderssons produkter skall kunna ansluta utan att någon vidare hård‐ eller mjukvara behövs ute hos kunden. Tjänsten bör fungera så enkelt att den är helt transparent för användarna. Författarens uppgift blev alltså att utveckla en arkitektur för hur en sådan tjänst skulle kunna byggas. För att säkerställa framtida flexibilitet och premiera återanvändning av kod så skulle denna lösning bygga på SOA‐koncept. Under projektets gång så har en sådan arkitektur utvecklats. Lösningen inbegriper två tjänster som används för att översätta respektive mellanlagra inkommande EDI‐meddelanden som följer EDIFACT‐standarden. Vidare innehåller lösningen specificationer för kommunikationsprotokoll (SOAP över HTTPS samt AS2 över HTTPS) och programmeringsgränssnitt (web services). Denna lösning kan användas för att implementera ett system som används för att integrera små och större företag med hjälp av EDI såsom ursprungligen efterfrågat. Lösningen har testats genom att en prototyp har utvecklats utifrån den. Det har också studerats hur väl lösningen håller sig till SOA‐koncepten. Lösningen verkar på ett tillfredsställande sätt behandla det beskrivna problemet.

(5)

Acknowledgements

The author would like to thank System Andersson for offering to mentor this thesis project. Particular thanks are owed to their head of development Andreas Käll and managing director Thomas Candemar, who have provided time for many interesting discussions as well as advice and guidance. The staff of JTH has also been very helpful throughout the project. Ragnar Nohre, the project’s mentor at JTH, and Vladimir Tarasov, examiner and main teacher of the thesis course, have both provided valuable input and discussions. Lastly, many thanks are owed to Franz Ferdinand, Atomic Swing, Jenny, and Gordon the Dachshund.

(6)

Keywords

Integration, EDI, SOA, Web Services, MPS, Architecture, SOAP, AS2, BizTalk, ERP.

(7)

List of Figures

Figure 1 ‐ Components of an X12 Transaction...21 Figure 2 ‐ Example X12 850 Purchase Order...22 Figure 3 ‐ UN/EDIFACT PAORES Message...23 Figure 4 – SOAP‐formatted RPC method request. ...30 Figure 5 – SOAP‐formatted RPC method response...31 Figure 6 ‐ System Map Version 1 ...43 Figure 7 ‐ System Map Version 2 ...44 Figure 8 ‐ System Map Version 3 ...45 Figure 9 ‐ Mapping of ORDERS Message to SOAP Message...58 Figure 10 ‐ BizTalk Orchestration of Incoming ORDERS Message...59 Figure 11 – BizTalk Orchestration of Outgoing INVOIC Message...60

(10)

List of Tables

Table 1 ‐ Size Distribution of Swedish Companies 1968‐1993...16 Table 2 ‐ Size Distribution of Swedish Manifacturing Companies 1968‐1993 ...17 Table 3 ‐ Size Distribution of European Companies 1988‐1991...18 Table 4 ‐ Comparison of Common EDI Messages...24 Table 5 ‐ Tight Versus Loose Coupling (Banke Krafzig, Slama, p. 47) ...28 Table 6 ‐ Description of OrderMessage class...50 Table 7 ‐ Description of Invoice Message API ...51

(11)

List of Abbreviations

SOA – Service Oriented Architecture ERP – Enterprise Resource Planning MPS – Materialplaneringssystem (Materials Planning Systems, Swedish term) AS2 – Applicability Statement 2 EDI – Electronic Data Interchange EDIFACT – Electronic Data Interchange For Administration, Commerce and Transport HTTP / HTTPS – Hypertext Transfer Protocol / Hypertext Transfer Protocol Secure API – Application Programming Interface XML – Extensible Markup Language

(12)

1 Introduction

The following chapter explains why this project is important for the associated company and their customers, what the project entails, and what limitations have been placed upon the project tasks. It aims to give the reader a cursory overview of the problem at hand, to provide context for the rest of the report. There is also a short outline of how the rest of the report is arranged.

1.1 Background

System Andersson AB is a software corporation based in Jönköping, Sweden that produces an Enterprise Resource Planning (ERP) system called System Andersson QwickMPS, as well as various software plugin modules for this system. QwickMPS has been developed to be appropriate for use in small businesses in the manufacturing industry, and has a focus on features that are appropriate in this sector. These features, such as time and cost estimates, planning of machine usage, supplies and warehouse management, etc., all help companies increase efficiency of operations and lower costs. Apart from the system itself, System Andersson also develops and produces the terminal computers used to access the system on the industry floor. Historically, the type of companies that use a system from System Andersson have been too small to have to concern themselves with large advanced IT infrastructures and projects. QwickMPS and the products in the QwickMPS family have therefore been designed to be very easy to use for employees without extensive IT experience. The user interfaces have been designed to be easy to access and use and in the newer versions of the terminal computers all input is performed via touch screen technology, meaning that even demands on typing proficiency have been removed. Recently however, the demands for ever greater efficiency in the manufacturing industry have led System Andersson to realize an opportunity to add new integration features to the QwickMPS system. The idea is to add the capacity for QwickMPS to recieve industry standard Electronic Data Interchange (EDI) documents. Such a feature could benefit small manufacturers who wish to be able to accept digital orders and other business documents from their medium size customer companies (who are often in turn subcontractors to large industry companies). Currently, business communication between these types of parties is performed via fax or telephone calls. Accepting EDI documents and having them automatically imported into the small manufacturer’s QwickMPS system could save a lot of money in administration costs for both the manufacturer and the medium size customer companies. Such a feature might also eventually open up possibilities for the small manufacturer to approach and do business with the large end‐user industry companies directly, thereby bypassing the middleman subcontractors. Such

(13)

direct business contact is today very unusual and difficult, due to the high degree of automation of order handling and such that the larger companies require in their business processes, which the small companies cannot handle. Simply building and bundling an own EDI integration feature with each QwickMPS install would however probably not be a good solution. It would add a large degree of complexity to each customer install, with many new potential error sources. For instance, when the EDI standards are updated, all clients who wish to receive documents in the latest format would require software updates. It would also require that the senders and receivers of EDI documents agree upon network configuration details that might need to be changed over time, requiring occasional reconfiguration of the system. The receiver of the EDI documents would also have to let their QwickMPS server expose at least some ports or interfaces to the sender via the Internet, a configuration that poses serious security risks. Finally, the cost of developing such a solution would in the integrated feature case have to be largely covered by the initial sale of the feature. This would likely put the price of the product at a point where it might be deemed cost‐prohibitive. It is therefore the opinion of System Andersson that it would probably be a better idea to develop such an integration feature as an online subscription service. The task assigned to the author of this thesis work is to construct an architecture for such a system, as well as a prototype system proving that the developed architecture specifications are sound. In order to simplify development and to promote re‐use of system components and a future‐proof solution, an architecture based on SOA concepts is suggested.

1.2 Purpose / Objectives

The main task of the project is to develop an architecture for an online integration system connecting partners who wish to communicate via EDI documents, as described in chapter 1.1. The architecture should be based upon SOA (Service Oriented Architecture) concepts. That is, it should consist of interoperable services that are loosely coupled and that interact using open protocols and standards (for more information about SOA, the reader is advised to read chapter 2.3). In addition, as described above, there is a demand that a prototype system be constructed according to the concepts developed in the specification. Such a prototype should be able to handle certain EDI messages and deliver them to a QwickMPS client system.

1.3 Limitations

There are a few limitations on the task that have been set in discussions with System Andersson, in order to make the project achievable within the allotted timeframe.

(14)

Primarily, the specification should be workable enough to implement the aforementioned prototype based upon them. Thus, the main components of the system should be laid out and their intercommunication studied and discussed in depth. However, the specification does not need to encompass details on implementation level such as database schemas or the values of every field transmitted in an EDI message. The task is to develop a workable base upon which the service can be built, not to build an exact blueprint for a system. Furthermore, the specification should not include how the QwickMPS system handles the delivered data. The QwickMPS developers will provide a wish list for methods they would like to see implemented, but studying how they interpret and load the data provided via these methods into the application is outside the scope of the task. The reason for this limitation is twofold – firstly it limits the required work for completing the task, and secondly, System Andersson are currently in the early stages of developing a new version of QwickMPS, meaning that the code handling this data import will be very much in flux during the length of the project, changing based on system design decisions which are outside the scope of this project. The prototype should work in a lab environment using certain predefined EDI message types/schemas appropriate to a given test customer. The prototype does not have to be elegant or complete in any way, but it does have to deliver the messages under optimum conditions. Error handling and resending of messages that fail to be delivered are not required.

1.4 Thesis outline

The thesis is divided into four main content chapters. Firstly there is a chapter on the theory relevant to the task at hand. In this chapter the technological concepts present in the problem context are described thoroughly, and some of the discussions from the background discussion above are revisited and expanded upon. The chapter was written in bursts at various times during the length of the project, when the concepts in question were especially present in the mind of the author. This chapter references primarily printed text on the various subjects, and to a certain degree web based content. Following this is a chapter on the methods used during the project work. This chapter deals with how the task was achieved, different choices and designs considered during the process, problems faced and how they were overcome. The chapter has been constructed incrementally during the length of the project, detailing the latest issues and questions studied. Most sources here are from technical texts and manuals, which are often published as web pages. Thirdly, there is a chapter detailing the results of the project. Here the final specifications and the answers to the questions posed in chapter 1.2 are presented, and the code of the prototype solution is analyzed briefly.

(15)

Finally there is a chapter concluding the report, discussing how well the project has gone and what conclusions can be drawn from the presented results, as well as what future work research may complement this project. Apart from these chapters there are also the mandatory support sections, appendices, etc. It might interest the reader to know that all web sources references in this work are attached as appendixes to the report in digital form. It may however be the case that they are not present in the printed version, depending on space constraints.

(16)

2 Theoretical Background

This chapter covers some of the historical background and current trends that necessitate the work presented in the thesis.

2.1 Structure of the Swedish Manufacturing Industry

2.1.1 Company Size Distribution

In most of the industrialized world, there has since the 1970s and onward been a trend away from large companies and large units of production in favour of smaller firms and organisations. In Sweden however, this trend is not seen – as Table 1 shows, the per capita number of companies with more than 200 employees has in fact increased in the period 1970‐1990. The same is true for companies with up to 49 employees. However, in the same time period there has been a decrease in the per capita number of companies employing 50‐200 people. Table 1 Size Distribution of Swedish Companies 196819931 If one looks at the manufacturing industry specifically a similar pattern can be seen. While the number of companies per capita has in fact decreased in this sector for almost all company sizes, relative to each other, the changes in numbers partially mirror the situation described above – the numbers of larger and smaller companies have decreased less than the number of intermediate size corporations, as displayed in Table 2. The main difference that we can see 1_{Henrekson, M. & D. Johansson, ‘Institutional Effects on the Evolution of the Size} Distribution of Firms’, Small Business Economics, Vol 12, No. 1 (February 1999), p. 13

(17)

compared with the situation in the previous table is that the dividing line between relative success and relative failure in the manufacturing industry runs at 10 employees rather than 50, thus indicating that the number of companies within the brackets 10‐19 and 20‐49 employees are decreasing more than average as well. Table 2 Size Distribution of Swedish Manifacturing Companies 196819932 For companies employing 10‐99 people, these changes over time are very much consistent with a point‐in‐time view of the size distribution of companies in some comparable European countries as displayed in Table 3. Sweden quite simply has few small industrial companies employing 10‐99 people. Henrekson and Johansson show that the reason for this is that these small companies have a hard time growing in Sweden, due to unfavourable entrepreneurial and business conditions for companies of this size.3 2_{Henrekson, M. & D. Johansson, p. 14} 3_{Ibid., p. 18‐21}

(18)

Table 3 Size Distribution of European Companies 198819914 Why are all of these numbers and employee number brackets important or relevant in the context of this project? Well, since System Andersson have defined their customers as being Swedish companies in the manufacturing industry with 10‐50 employees, we can tell that even before the current economic downturn, these customer companies were facing unfavourable conditions and were having a tough time, especially when compared to larger companies. It is therefore logical to assume that they should show an interest in obtaining systems and technologies that would help them to offset this imbalance. The service which this project aims to develop could be such a technology, in that it would allow for small companies to be more efficient and competitive by (a) decreasing the administrative costs associated with sales and purchasing, and (b) allowing them to bid for contracts that they might previously have been unable to aim for, due to not having EDI integration capacity. 2.1.2 Supply Chains In the Swedish manufacturing industries it is common for firms to have different sourcing and supply chain strategies depending on how critical a certain component or product is for their production, or how many suppliers there are on the market in question. It is generally the case that these companies try to achieve partnership type relationships with suppliers who provide critical or rare components, while they treat other suppliers with a bit more scepticism and keep them at arm’s length. This latter group tends to receive shorter terms on their contracts and are selected largely on price. The term used for this type of 4_{Henrekson, M. & D. Johansson, p. 17}

Sweden on average has the greatest number of large industrial firms per capita among the coun-tries compared. Here Sweden is followed closely by Germany, but then there is a leap to Finland. In terms of medium-sized industrial firms per capita Sweden is about average, while it ranks below average in the small size class. The impres-sion that Sweden has many large but compara-tively few small firms compared to other European countries is strengthened when we analyze the size distribution of firms. Sweden now ranks as number 1 in the large size class, and the distance to the 2nd ranking country is greater. Sweden also has a large share of medium-sized firms according to this definition. The share of small-sized indus-trial firms, however, is reported to be the second smallest among the countries included in the com-parison.

The low number per capita and the small pro-portion of small sized Swedish firms are particu-larly interesting considering the analysis carried out in the previous section. The small size class in the international comparison includes three of the four size classes that we named intermediate-sized firms viz., the 10–19, 20–49 and 50–99 classes. In the previous section it was shown that the number of intermediate-sized firms had declined

substantially in manufacturing. Thus, the interna-tional comparison at a single point in time is con-sistent with the above finding of a poor development for intermediate-sized firms.

Taken together these empirical findings raise the question whether Swedish institutions have been an obstacle to growth for intermediate-sized firms (and favoured large-sized firms)? We will investigate this question in the next two sections.

4. Some evidence on the willingness to grow

2. among Swedish firms

A number of studies have documented a weak inclination to grow among small firms in Sweden. For instance, Lundström et al. (1993) find that merely 10–15 percent of the small firms expand employment. But do small and newly started busi-nesses want to grow? In order to shed light on this question a number of studies are useful to look at. Aronsson (1991) examined 535 firms estab-lished in 1988. Three years later turnover exceeded SEK 1 million in no more than one fourth of the firms, and one third of them had no full-time employee, the owner included. From Aronsson’s study one can infer a number of reasons why the proclivity to expand was so low

Institutional Effects on the Evolution of the Size Distribution of Firms 17

TABLE V

The average number and the size distribution of firms per million inhabitants in the industrial sector, 1988–91 (rank in parentheses)

Firms per million inhabitants Size distribution Sum Small Medium Large Small Medium Large

Germany 1187 0(3) 144 0(2) 33 0(2) 87.0 0(7) 10.6 0(6) 2.4 0(6) 100 France 0669 (10) 084 0(9) 19 (10) 86.70 (8) 10.9 0(5) 2.40 (7) 100 U.K. 0650 (12) 075 (11) 22 0(5) 87.1 0(6) 10.0 0(7) 2.90 (4) 100 Sweden 0774 0(8) 112 0(5) 34 0(1) 84.1 (11) 12.1 0(2) 3.7 0(1) 100 Italy 1365 0(2) 077 (10) 11 (11) 93.9 0(1) 05.3 (12) 0.8 (12) 100 Spain 1055 0(5) 074 (12) 10 (12) 92.6 0(2) 06.5 (11) 0.9 (11) 100 Belgium 0678 0(9) 091 0(7) 21 0(8) 85.8 (10) 11.6 0(3) 2.6 0(5) 100 Portugal 1504 0(1) 161 0(1) 21 0(7) 89.2 0(4) 09.5 0(9) 1.3 (10) 100 Denmark 1131 0(4) 124 0(4) 21 0(9) 88.7 0(5) 09.70 (8) 1.6 0(9) 100 Luxembourg 0668 (11) 141 0(3) 26 0(4) 80.1 (12) 16.9 0(1) 3.1 0(2) 100 Finland 0787 0(7) 101 0(6) 27 0(3) 86.0 0(9) 11.1 0(4) 3.0 0(3) 100 Norway 0895 0(6) 086 0(8) 21 0(6) 89.3 0(3) 08.6 (10) 2.1 0(8) 100 Average 0947 106 22 87.5 10.2 2.2 100

Note: Small, medium, and large firms are defined as firms with 10–99, 100–499 and 500+ employees, respectively. The

geographical coverage for Germany is the former Federal Republic of Germany.

(19)

business relationship is “adversarial arm’s length with buyer dominance”, indicating quite clearly that the buyer is the one calling the shots. The seller has rather little influence over the process.5 Additionally, the promise of lower labour costs have led a lot of larger companies to source components from other countries. While this purchasing behaviour is not yet as common in Sweden as it is in the rest of Europe (a fact partially explained by the strong labour unions who oppose it here), it is projected to increase faster in the years to come. In the manufacturing industry this scenario is particularly common.6 This is the reality that the small companies previously mentioned operate in. They are under constant threat to loose business to other cheaper competitors, at home or abroad. They are dependant upon their customers, but have little possibility of influencing their customers purchasing patterns, other than by lowering prices and trying to stay ahead of the competition. This again underscores the need these companies likely have for any measure that would make them more efficient or competitive in the market.

2.2 Electronic Data Interchange

Electronic Data Interchange, or EDI for short, is a generic term for the transmission of electronic documents between business partners, as well as the standards and technologies used to implement such transmissions. Typical uses of EDI include the ordering and invoice processes in large enterprises, as well as logistics management (where despatch notes are sent electronically). While information exchanges using technology such as web services might semantically qualify as being Electronic Data Interchanges, it is generally considered that when talking about EDI one is referring to a text message formatted according to one of the four established EDI standards: UN/EDIFACT, ANSI X12, TRADACOMS, OR ODETTE. These four standards were developed at around the same time for similar purposes though in different regions or industries. The UN/EDIFACT standard is the most prevalent today, and is used around the world, supporting a large variety of business domains and documents. It’s name gives away it’s origins – the standard was developed by the United Nations. The X12 standard was developed by the American National Standards Institute (ANSI) and is mostly in use in the USA and Canada. Like the UN/EDIFACT standard, ANSI X12 is used in many different industries. The ODETTE standard is used in the European 5_{Håkansson, F., T. Johansson & H. M. Missola, Sourcing Strategies and Supplier} Relationships, Växjö University School of Management and Economics, 2009 6_{Alajääskö, P., ’International Sourcing in Europe’, Statistics in focus, Nr 4/2009, Eurostat}

(20)

automotive industry and is developed by the Odette Organisation, and industry group representing companies on that market. TRADACOMS, finally, is used on the UK retail market.7 Each of these standards is very large and encompasses a great many message specifications used in many different business scenarios. Each message specification defines what data fields should be used in the message, those fields’ sizes, whether they are mandatory (must be present) or optional, and in what order they appear in the message. In addition to the standardised messages, it is possible for individual corporations to narrow the specifications of the messages they use further, by applying profiles of the messages. A profile is essentially a specification of a number of optional fields in the original message that are mandatory when the profile is applied. By defining a set of profiles, a company can thus define a subset of the chosen EDI standard that it wishes to use in communications, as well as specify what fields from the standard that it requires be present in each message.8 2.2.1 Message semantics When discussing message syntax and semantics, the two most interesting EDI standards are the largest, UN/EDIFACT and ANSI X12. TRADACOMS is used solely within the UK and is considered likely to be phased out in favour of the EDIFACT standard in the future, and the ODETTE standard actually uses the EDIFACT message syntax for its messages.9 7_{Swatman, P., ’Electronic Commerce: Origins and Future Directions’, Proceedings of the} 1st Australian DAMA (Data Administration Management Association) Conference, Melbourne, 1996 8_{Intervju med Scania InfoMate} 9_{SAA Consultants, The EDI Zone: EDI Guide – EDI Standards, viewed on 2009‐06‐10,} <http://www.theedizone.com/edi_resources/edi_standards.html>

(21)

Figure 1 Components of an X12 Transaction10 In X12 terminology a single EDI interchange (that is, communication session or file) transmits one or more functional groups. A functional group is a grouping containing one or several related business documents (called transaction sets). If one were to compare these concepts to physical documents, one could say that a transaction set is a business document such as a purchase order or an invoice, whereas a functional group is an envelope that holds several purchase orders that are being sent to the same supplier. The transaction sets, our business documents, in turn consist of a number of tagged segments holding the actual data. A segment can hold a single data element (such as a phone number) or multiple elements (such as a list of line items). Segments are delimited by a control character (normally a linebreak), and multiple elements within one segment are delimited by another control character (normally an asterisk). Special segments define the beginning and end of transaction sets and functional groups, such as the SE segment which indicates the end of a given transaction set or GS which indicates the beginning of a functional group. For an illustration of 10_{Tallim, P. & J Zeeman, Electronic Data Interchange: An Overview of EDI Standards for} Libraries, International Federation of Library Associations and Institutions, updated 1995‐04‐27, viewed 2009‐06‐10, <http://archive.ifla.org/VI/5/reports/rep4/index.htm>

(22)

how these concepts relate to each other, see Figure 1. For an example X12 document, see Figure 2. 111213 Figure 2 Example X12 850 Purchase Order14 The X12 standard has specifications for a large number of transaction sets, which are referred to by their three‐digit code number (such as purchase order which goes by the number 850, as displayed in the image). For each of these transaction sets, the required segments that make up the document in question are specified. 11_{Tallim, P. & J Zeeman} 12_{Rawlins EC Consulting, X12 Technical Tutorial Syntax and Control, updated 2007,} viewed 2009‐06‐10, <http://www.rawlinsecconsulting.com/x12tutorial/x12syn.html> 13_{GXS, Inc., ANSI ASC X12 Standards Overview Tutorial, viewed 2009‐06‐10,} <http://www.gxs.com/pdfs/Tutorial/Tutor_ANSI.SO_GXS.pdf> 14_{Amosoft, EDI 850 X12 Example, viewed 2009‐06‐10,} <http://www.amosoft.com/edi_docs/EDI_850_X12_Example.html>

(23)

Figure 3 UN/EDIFACT PAORES Message The EDIFACT standard is similar to X12 in many ways, to the extent that if it weren’t for the difference in tags used, one might confuse an X12 document for an EDIFACT document and vice versa. Functional groups are present in both, the transaction sets from X12 are simply renamed to messages in EDIFACT, and each such message is composed of segments, just like X12. However, there are some significant differences in the details between the two standards. To begin with, EDIFACT allows for composite data elements. Whereas, the smallest possible unit of information in X12 is the data element, an element in EDIFACT can be composed of several values separated by a component data element separator. This means that EDIFACT needs one more control character, to use for the purpose of separating such components. A second new control character has also been added, a release character used to escape occurrences of control characters within the data. Be prefixing such characters by the release character, they are ignored by document parsers. Thirdly, EDIFACT can use two different syntax levels, depending on the environment in which the EDIFACT documents are transmitted and what characters are available for use. Syntax level A disallows the use of certain characters (such as lower case letters and some special symbols) that are not possible to transmit to a telex machine. Syntax level B permits most of these characters, and for this reason cannot be used with legacy telecoms equipment such as telex. Figure 3 gives an example of an EDIFACT PAORES (Product Availability Request) message utilizing both composite data elements (separated by the colon character) and the release character (in this case a question mark).

2.2.2 Common message types

As mentioned previously, the EDI standards X12 and EDIFACT are very large and detailed. In fact, EDIFACT covers over 200 standard messages and X12 is said to

(24)

be at least as complex.15_{Not all of these messages are necessarily used by all} companies. In fact, it is common for businesses to mandate use of only a few messages that correspond to their business needs. As discussed previously it is also common for companies to specify the messages they use even further, through the use of profiles. Another usage pattern is when multiple companies or an industry group decides upon a subset of messages and/or message profiles of one overarching standard to use for communications. One example of this is how the Odette group in 1990 decided that the ODETTE standard should migrate from being its own independent standard, to becoming a subset of UN/EDIFACT. 16 The most common messages used in a number of manufacturing and commerce companies studied by Fällén and Holmgren are DELINS, DELFOR, SYNCRO, FORDIS, AVIEXP, INVOIC, KANBAN and ORDERS. 17_{Table 4, based on Fälldén &} Holmgren as well as Progress Software Corporation and ERPGenie.COM details how these messages correspond to each other and to other X12, EDIFACT and ODETTE EDIFACT subset messages. 18 Table 4 Comparison of Common EDI Messages

Description EDIFACT X12 ODETTE subset of EDIFACT

Planning

Schedule/Material Release

DELFOR 830 DELINS

Shipping Schedule DELJIT 862 SYNCRO

Shipping Schedule DELJIT 862 KANBAN

Ship Notice /

manifest (ASN) DESADV 856 AVIEXP

Ship Notice /

manifest (ASN) DESADV 856 FORDIS

Invoice INVOIC 810 INVOIC

Purchase Order ORDERS 850 ORDERR

15_{Progress Software Corporation, EDIFACT – UN/EDIFACT and UN/CEFACT Message and} Segment Directory, viewed 2009‐06‐15, < http://www.stylusstudio.com/edifact/> 16_{Odette International Ltd., Milestones, viewed 2009‐06‐16,} <http://www.odette.org/html/about_milestones.htm> 17_{Fällén, S. & A Holmgren, EDI mot leverantörer till Sandvik Coromant, Luleå Tekniska} Universitet, Luleå, 1997, pp. 22‐23 & 69‐107 18_{ERPGenie.COM, ANSI X12 Transaction Sets, viewed 2009‐06‐17, <} http://www.erpgenie.com/sapedi/ansix12transactionsets.htm>

(25)

2.2.3 Transmission methods Since EDI files are stored as normal ASCII files just like any other in a computer system, transmitting them between business partners can be done in just as many ways as transmitting any other type of file. One can use FTP, SFTP, WebDAV, email (SMTP), or any other transmission method that allows the transfer of computer files. Since EDI files often hold information of significant importance however, transfer methods that provide delivery notification to the sending party are generally preferred. Using such transfer methods the sender can be sure that the file has not been corrupted or lost along the way, but that it has been delivered just as intended to the recipient. There are a few special protocols for delivering EDI files that have been developed with this need for delivery notification in mind. The most common ones are the OFTP (Odette FTP) and AS2 (Applicability Statement 2) protocols. The OFTP protocol was developed by the Odette group, an industry group for members of the automotive industry in Europe19_{. While the first version of the} protocol initially supported only direct transmissions between communicating partners via point‐to‐point links such as X.25 or ISDN, it has since been extended to include support for operating on top of the Internet protocol stack20_. The second version of the protocol supported the TCP/IP protocols natively21_. The AS2 protocol has been developed by the IETF, following the normal Internet protocol standardization process. AS2 operates on top of HTTP, using HTTP POST operations to transfer data files packaged using MIME and (optionally) encrypted using S/MIME, implemented using standard asymmetric cryptography ciphers. AS2 is specified in RFC 413022_.

2.3 Service Oriented Architecture

This goal of this project is to develop an architecture based on SOA, but what does that really mean? SOA (short for Service Oriented Architecture) is a method of constructing software and information systems that aims to produce flexible, reliable and future‐proof systems. This section discusses the driving forces 19_{Odette International Ltd., Milestones} 20_{Nash, D., ODETTE File Transfer Protocol, September 1997, viewed 2009‐07‐28,} <http://www.ietf.org/rfc/rfc2204.txt> 21_{Friend, I. ODETTE File Transfer Protocol 2, November 2007, viewed 2009‐07‐28,} <http://www.ietf.org/rfc/rfc5024.txt> 22_{Moberg, D. & R. Drummond, MIMEBased Secure PeertoPeer Business Data} Interchange Using HTTP, Applicability Statement 2 (AS2), July 2005, viewed 2009‐07‐28, <http://www.ietf.org/rfc/rfc4130.txt>

(26)

behind SOA adoption in the industry, as well as what SOA actually means in practice.

2.3.1 SOA Drivers – Business Agility Requirements

In today’s rapid moving global marketplace the ability to adapt one’s business to meet changing demands is key. One must anticipate and preempt the competition’s moves, as well as stay on top of consumer trends and technological discoveries and inventions. The ability to handle such a changeable environment is dubbed agility, and it is, according to several sources, one of the most important traits in a successful business. The opposite of agility, inflexibility, is then obviously a bad trait for a corporation to display. It is so bad, in fact, that one source claims that it is the “mother of all business problems”, which encompasses all other problems that an enterprise may experience23_{. The argumentation goes} a little bit like this: an agile company can change its behavior to meet any challenge it may come up against. It can change product offerings to match changing customer demand or new technologies. It can change strategy or enter or leave markets to maintain long‐term viability. It can even respond to political events or legislative challenges effectively by being prepared for change. Whether this claim of inflexibility being the single largest problem for companies is true or not, it can be said with a high degree of certainty that agility or lack thereof is at the very least one of several very important factors for business success. Companies that are unable to change with the times and adapt to their customers’ needs go bankrupt or get bought out by more successful companies. So, how does one in practice achieve agility in business? What does the word actually imply when you get down to it? Depending on specialization, a broad range of approaches are available, from removing leader figures and decentralizing business24_{to performing business process reengineering to} outsourcing non critical business functions or performing various other types of management changes. The IT department’s contribution to business agility then becomes to provide IT systems that can handle rapid changes in company strategy and policies25_{. It is important when discussing agility in IT to distinguish} organizational business agility as discussed here from other uses of the word agility common in the IT industry, such as agile software development methods. 23_{Bloomberg, J. & R. Schmelzer, Service Orient or Be Doomed!, John Wiley & Sons, Inc.,} Hoboken, 2006, p. 3 24_{Brafman, O. & R. A. Beckstrom, The Starfish and the Spider : the unstoppable power of} leaderless organizations, Portfolio, United States of America, 2006 25_{Krafzig, D., K. Banke & D. Slama, Enterprise SOA, Pearson Education, Inc., Upper Saddle} River, 2005, p. 2

(27)

These methods (such as SCRUM and Extreme Programming) represent tactics for producing and delivering individual projects. Business agility on the other hand is a more strategic concept and refers to the flexibility of the whole of the organization, rather than individual development teams26_. SOA is by many sources considered to be an architecture and an approach that is well suited for developing and maintaining agile IT systems, that is, systems that flexible enough to accommodate frequent changes in business environments and policies. For this reason, the increasing need for business agility has become the main motivation driving SOA adoption.

2.3.2 What does SOA mean?

The basic idea of SOA is to try and break down large and complex systems into smaller simpler services, that one can then plug together in order to achieve the business functionality that is required. These services should be small and simple enough to be easy to develop, debug and deploy as a unit, yet advanced enough to represent a business concept or activity that makes sense. A good metric for measuring the size or complexity of a single service is to see if non‐technical businesspeople understand what the service does. If the businesspeople do not understand the business need for a particular service, it is possible that the scope of the service is in fact too narrow27_{. For instance, a good example of a service} might be a service that handles outgoing shipments. Such a service maps well to some of the requirements of a business domain, without being large enough to be a complete general logistics system. Businesspeople could certainly understand the need for a shipping service. However, if one were to break such a service down further into even smaller services, each one handling technical details such as transmitting dispatch notices or coordinating GPS trackers, those services would probably be too fine‐grained and narrowly scoped. That is not to say that the designed services should not be able to handle such more fine‐grained tasks. (Erl, p. 69) discusses services as collections of capabilities, grouped together by functional context. Our shipping service would very likely have capabilities for adding and removing shipments, reporting on shipments, handling dispatch notes, tracking packages by GPS (if there is a business need for this) and so forth, and depending on the needs of the organization, these capabilities might be published and accessible to other services and employees within the organization. However, these capabilities are part of the service and have no life independently of it. The code that invokes all 26_{Erl, T., SOA: Principles of Service Design, Pearson Education, Inc., Boston, 2008, p. 63} 27_{Josuttis, N. M., SOA in Practice, O’Reilly Media Inc., Sebastopol, 2007, pp. 26‐27}

(28)

of these related features is developed, debugged and deployed as a unit within the company infrastructure. The point of splitting systems up into services like this is to make it easier to reorganize the systems to meet changing needs, and to lessen the impact of failure within any one of the services. Another goal of using a Service Oriented Architecture is to make services reusable within multiple systems. For instance, a customer registry service could be used both within sales systems and customer support systems. All of these requirements mean that one should try to lessen interdependence between services, so that required changes or any instability in one service has as small an impact as possible on other services or the system as a whole. In order to minimize such dependencies, SOA development emphasizes the use of loose coupling between services. Service interfaces should be separate from service implementations, meaning that it should be possible to swap implementations easily without disrupting operation of the system as a whole, provided the new service implementations provide the same interfaces as the ones they replace. This is more difficult than it sounds in a distributed system where services might need to share data or implement distributed workflows. Using loose coupling implies adhering to at least some (if not all) of the principles for connecting systems listed in the Loose Coupling column of Table 5 (Krafzig/Banke/Slama p 47). Table 5 Tight Versus Loose Coupling (Banke Krafzig, Slama, p. 47)

Level Tight Coupling Loose Coupling

Pysical coupling Direct physical link

required Physical intermediary

Communication style Synchronous Asynchronous Type system Strong type system (e.g.

interface semantics) Weak type system (e.g. payload semantics) Interaction pattern OO‐style navigation of

complex object trees Data‐centric, self‐contained messages Control of process logic Central control of process

logic Distributed logic components Service discovery and

binding Statically bound services Dynamically bound services Platform dependencies Strong OS and programming language dependencies OS‐ and programming language independent All of the rows in the table are important when developing SOA type solutions. However, in the interest of brevity only a few rows are discussed here. The

(29)

interested reader is strongly encouraged to study Banke, Krafzig and Slama further. The first row deals with physical coupling of services (unfortunately this is a bit of a misnomer – we are not discussing Ethernet level wire protocols here, but rather how services communicate on a rather abstract level). The row points to the fact that communication in loosely coupled systems most often is performed using a message oriented middleware (MOM) as an intermediary communications bus, rather than using direct connections between services. This makes it easier to swap out individual services without rewriting too much code, since the services do not directly rely on one another. If one were to swap out this MOM one would of course need to modify a lot of code, but hopefully this is a less common occurrence. Another important line in the table above is the one that deals with interaction patterns. In a tightly coupled system it is not unusual to have a common object model for all involved services or components that make up the system, which means that complex deeply nested objects can be passed around the system easily. This obviously introduces many interdependencies, as all services rely on identical data representations, and if one’s needs for one of the services changes, the entire object model used by all services will have to be updated. In a loosely coupled system this is frowned upon, and simpler data‐centric messages are used as far as possible. Finally, a SOA system should be as independent as possible of any particular operating system or programming language. Since interfaces and implementations of services should be separated and implementations be interchangeable, the stability and usability of the interfaces becomes very important. It is therefore against such interfaces that one should develop, rather than against any particular implementation in any particular programming language or environment.

2.4 Web Services

While Service Oriented Architectures can be developed using a number of technologies and interconnect methods, arguably the most common of these is family of technologies known collectively as Web Services. This chapter introduces web service technologies and some related concepts. 2.4.1 Overview Web services are software components that are designed to be accessed over a network using standardised open protocols operating on top of the HTTP protocol traditionally used for transmitting web pages. These services have interfaces defined and published in the strictly defined WSDL format, which allows other software components (“service consumers”) to interpret such an interface and use the methods provided by the corresponding service. Interaction

(30)

with the service is normally performed by way of SOAP messages that hold requests to or responses from the service, in typical RPC (Remote Procedure Call) fashion. There are a number of other technologies that provide remote invocation of code in similar manners, for instance CORBA, Microsoft’s DCOM, and Java‐RMI. However, none of these technologies have seen as widespread adoption as web services, which are supported by more or less every major programming language on the market, either natively or through the use of extension libraries. 2.4.2 SOAP The SOAP protocol is a specification for exchanging structured data between systems and services. SOAP uses XML as its message format, meaning that SOAP messages are essentially XML messages following a certain SOAP‐specific schema. Apart from the message format, the SOAP protocol also specifies various other details necessary for establishing communications, such as bindings to underlying transport protocols, message exchange patterns, and how extensions to the protocol should be handled. While HTTP is the most commonly used transport protocol for SOAP messages, SOAP also supports binding to SMTP or the more uncommon AMQP or JMS protocols. In all of these cases, SOAP operates one level above the application level protocol in the network stack, which means that the protocol draws more overhead than had it been implemented as its own application protocol. 28 The choice of using XML as its message format means that SOAP is fairly verbose, as illustrated by Figure 4 (in this figure, the namespace declarations have been omitted for brevity, so that in real life, this message would be even larger). While this makes the protocol easy to read and interpret for humans, it also means that 28_{Snell, J., D. Tidwell & P. Kulchenko, Programming Web services with SOAP, O’Reilly,} 2002, p. 25 <s:Envelope xmlns:s="..."> <s:Body> <checkStatus xmlns="..." s:encodingStyle="http://www.w3.org/2001/06/soap-encoding"> <orderCode xsi:type="string">abc123</orderCode> <customerID xsi:type="string"> Bob's Store </customerID> </checkStatus> </s:Body> </s:Envelope> _{Figure 4 – SOAPformatted RPC method request.}

(31)

automatic parsing and interpretation of SOAP messages by software can be relatively computationally expensive. A SOAP message consists of an envelope containing a header section and a body section. The header holds metadata fields and protocol data, whereas the body contains the message that is being transmitted. In a web service scenario, typically a SOAP message would be sent into the service with a body holding service parameters, and the service would respond with a SOAP message with a body holding the response. Figure 4 and Figure 5 illustrate this scenario for a call to a method with the signature String checkStatus(String orderCode, String customerID); that has been published as a web service. The call in question is invoked as: result = checkStatus("abc123", "Bob's Store"), and the actual messages that are sent down the wire are the ones displayed in the figures.29 30 2.4.3 WSDL Web Service interfaces are published in the WSDL format, short for Web Service Description Language. By parsing such a published WSDL file a service consumer can learn how to call the service from which the WSDL file was generated. WSDL files are commonly published to the web in the same directory as the service implementation, but can be delivered to the service consumer just like any other file via email, FTP, or any other transfer mechanism. The WSDL format is rather complex and “heavyweight”, for which reason the whole of the format is not described here. The essentials primitives described in a WSDL file are, however: 29_Ibid. 30_Ibid. <s:Envelope xmlns:s="..."> <s:Body> <checkStatusResponse s:encodingStyle="http://www.w3.org/2001/06/soap-encoding"> <return xsi:type="xsd:string">new</return> </checkStatusResponse> </SOAP:Body> </SOAP:Envelope> Figure 5 – SOAPformatted RPC method response

(32)

• Service – An envelope that represents the service as a whole. Can contain multiple operations. • Port – Defines a connection point for interacting with the service (basically, an URI on which the service listens) • Binding – Ties a number of operations to a given service, as well as a transport protocol. • Operation – In programming parlance, a method. Tied to input and output messages. • Message – Defines the required schema for any request or response messages used in the service, i.e. what fields must be present in any SOAP messages sent to/from the service.

2.5 Existing Solutions in the Field of Study

2.5.1 VAN providers The EDI formats were developed before the widespread business adoption of the Internet or other types of always‐on network connections. The prevailing method of data transfer at this time was via dial‐up modem connections, which were quite costly to operate. In this environment, the most common way of sending and receiving EDI documents were through Value Added Networks, or VANs. The VANs provided a number of services to facilitate EDI communications between businesses. Among the services provided were (and still are, to the extent VANs are used): • Mailboxes – Provides storage of messages that cannot be delivered immediately. • Protocol conversion – Provides translation and mapping between different EDI dialects, so that companies utilizing different EDI standards can still communicate. Can also sometimes be used to translate to entirely different formats, such as XML, E‐mail, fax, etc. • Security – Provides auditing and backup of transferred messages, as well as usage reports and other such features. These VANs were accessed via modem connections or other forms of Wide Area Networks (WANs), often owned and operated by the VAN operators. The VAN operators also often provided the mapping software that translated the output from their clients’ systems into EDI format. In this way, the VAN operator provided everything that their clients needed to use EDI for business‐to‐business communications.31 31_{Barnes, S., Ecommerce and vbusiness, Elsevier, 2007, p. 296}

(33)

However, these VAN services were often rather expensive, both in terms of initial setup, monthly cost, and per‐transaction fees32_{. With the breakthrough of the} Internet and the increased availability of cheap network connections, it appears that usage of VAN services has declined, in favour of direct connections between parties, either over the internet or via services such as X.25 or ISDN. The most well known Swedish VAN operator still in business is Strålfors, a subsidiary of Posten AB, formerly the Swedish Post Office. 2.5.2 Web-EDI Despite not needing to use a VAN any longer, EDI is still quite expensive for small companies to implement (which is the main motivation behind this thesis project). Some larger companies (for instance Husqvarna and Scania) wishing to make it easier for their small suppliers to use EDI have therefore developed web‐ based EDI solutions. These Web‐EDI solutions commonly consist of a web page through which the supplier logs in with a username and password, and from which they can retrieve EDI messages such as orders from the company hosting the Web‐EDI solution. Often, these web systems are advanced enough to display the EDI document in a human understandable format, so that the supplier in question can then enter for instance order details into their own systems simply by copying and pasting. The downside to using Web‐EDI systems is that they still require a lot of manual work, that a fully automated EDI solution does not. The supplier has to sit down at his terminal and use his web browser to go to a certain page, enter authorization credentials, and then copy every order (or other relevant document) into his own system. This work is then repeated for every single customer that the supplier has Web‐EDI access to. This also leaves a lot of room for error, since it is a manual process. For instance, the wrong data could be copied into the wrong field. Finally, while some of these Web‐EDI solutions support the sending of EDI documents (such as invoices or despatch notices), many do not, meaning that a lot of time‐consuming communication still has to take place out‐of‐band.

2.5.3 Watson Research Center Method

Another take on the Web‐EDI method was proposed by a group of researchers at the Watson Research Center in a 1999 paper. The idea was to use the then popular technology of embedded Java applets to build a Web‐EDI type solution. A 32_{Fu, S., J. Chung, W. Dietrich, V. Gottemukkala, M. Cohen, & S. Chen, ‘A Practical} Approach to Web‐based Internet EDI’, Distributed Computing Systems, International Conference on, 1999 ICDCS Workshop on Electronic Commerce and Web‐Based Applications, 1999, pp. 53.

(34)

supplier receiving EDI orders from a customer would then log into the customer’s server with a web browser and retrieve both the EDI order document and a Java applet. The Java applet would be used to display the order in a human readable format (along with the possibility of replying to it with an invoice), as well as for forwarding the EDI file to the suppliers back end systems.33 With the development of modern web application technology (AJAX, DHTML, web services, etc), most of the motivations for using a specific language technology such as Java applets for this purpose is gone. Any sufficiently advanced web application toolkit and programming language could be used to build such a solution for reading and responding to EDI documents, and the export of EDI documents into backend systems could most likely be performed using web service technology. However, the Watson Research Center method is interesting nonetheless, as an early example of the viability of web applications in this field. 33_Ibid.

(35)

3 Methods

This project is largely exploratory in nature. Not only is a solution to the problem of business integration for the small enterprise required, but the problem itself needs to be explored and studied, in order to learn what the requirements of the involved parties are. Thus, in order to provide input for the system requirements, a number of interviews are to be conducted with relevant stakeholders. These stakeholders will primarily be large and small businesses using the system, but also co‐workers at System Andersson who are interested in developing it. The architecture will be developed in accordance with these system requirements. Apart from these requirements that stem from interviews with stakeholders, a study of existing solutions within the relevant fields (SOA and Business Integration) is necessary. While it is possible that there is little material dealing extensively with both of these topics at once, there must certainly be a breadth of material available on each one separately. The studies of these topics are performed as part of the general background theory studies, and are detailed in chapter 2. Based upon both the system requirements and the study of existing alternatives, a system architecture is then to be developed. The development process and architectural choices made should be described, discussed and motivated. The resultant architecture should specify the services required within the system and how these services interact, on a level that is specific enough that an implementation of the architecture can be performed by a third party. The architecture should be validated by way of studying how well it compares with the developed requirements and how well it adheres to SOA design principles. Based upon this architecture, a prototype is also to be developed. The prototype should be used to test whether some integration tasks can be performed as intended, such as for instance, whether a limited number of message types can be passed back and forth through the system. This provides an additional real world test of the architecture. The results of the project consist of both the architecture and the prototype, and the success or failure of both parts of the project are to be discussed and measured in the results section of the project report.

Orders From the Cloud : Business Integration as a Service