Enterprise Search Management Maturity: - A model for the assessment of an organization’s maturity level within enterprise search management

(1)

Enterprise Search Management Maturity

-

A model for the assessment of an organization’s maturity level within enterprise search management

FELICIA DENBU WILHELMSSON MALIN ERIKSSON

Master of Science Thesis

(2)

Att mogna inom styrning av enterprise search

- En modell för att utvärdera en organisations mognad inom styrning av enterprise search

FELICIA DENBU WILHELMSSON MALIN ERIKSSON

Examensarbete

(3)

Att mogna inom styrning av enterprise search

- En modell för att utvärdera en organisations mognad inom styrning av enterprise search

av

Felicia Denbu Wilhelmsson Malin Eriksson

Examensarbete INDEK 2013:59 KTH Industriell teknik och management

Industriell ekonomi och organisation

(4)

Enterprise Search Management Maturity

- A model for the assessment of an organization’s maturity level within enterprise search management

Felicia Denbu Wilhelmsson Malin Eriksson

Master of Science Thesis INDEK 2013:59 KTH Industrial Engineering and Management

Industrial Management

(5)

Examensarbete INDEK 2013:59

Att mogna inom styrning av enterprise search - En modell för att utvärdera en organisations mognad

inom styrning av enterprise search

Felicia Denbu Wilhelmsson Malin Eriksson

Godkänt

2013-juni-11

Examinator

Matti Kaulio

Handledare

Pernilla Ulfvengren

Uppdragsgivare

Findwise AB

Kontaktperson

Maria Björk

Sammanfattning

När organisationer växer ökar även deras interna informationsmängd. I takt med detta, ökar även behovet av att finna relevant information inom organisationen. Tyvärr är detta något som ofta försummas av dagens organisationer. Enterprise search är en sökapplikation som möjliggör sökning i flera olika informationskällor från endast en applikation. Detta innebär att istället för att söka igenom olika informationskällor via separata sökfunktioner, behöver användare av en enterprise search-lösning endast göra en enda sökning. Findwise AB är ett svenskt IT-konsultbolag som är specialiserade på att utveckla sådana enterprise search-lösningar. Ett stort problem är att organisationer ofta saknar ett konkret arbetssätt och en strategi för enterprise search. Detta leder till att det blir mycket svårt att dels realisera värdet av söklösningen, men även till svårigheter med att definiera hur det interna arbetet kring söklösningen bör se ut. På grund av detta vill Findwise kunna erbjuda sina kunder enterprise search- strategier. En sådan strategi skulle underlätta kunders interna utveckling av styrprocesser för enterprise search. Detta kräver dock förmågan att definiera en organisations nuvarande och framtida tillstånd med avseende på styrningen av enterprise search, något som fram tills nu ej har varit möjligt. Således var syftet med detta arbete att ta fram en mognadsmodell för utvärdering av styrprocesser för enterprise search. Denna mognadsmodell ska bistå Findwise-konsulter med ett verktyg för att definiera vilken mognadsnivå en organisation befinner sig på i dagsläget, men även se var man vill befinna sig i framtiden. Från denna utvärdering kan sedan de aktiviteter identifieras, som behöver utföras för att uppnå ett framtida mål och skapa en strategi för enterprise search. Denna rapport beskriver utvecklandet av en sådan mognadsmodell för styrning av enterprise search. Modellen utvecklades genom en kvalitativ fallstudie på Findwise AB, där resultat från en enkät och semistrukturerade intervjuer utgör grunden för studien. Fakta och diverse koncept har även grundats i teori. Resultatet blev en mognadsmodell bestående av fem olika mognadsnivåer: kunskap, hantering, prioritering, standardisering och hållbarhet. Dessa används för att utvärdera mognaden inom tre huvudområden av enterprise search: information, teknik och användare. Inom varje område är tre olika faktorer identifierade, som bidrar med riktlinjer för vad man bör titta på vid en utvärdering av enterprise search. Modellen möjliggör för Findwise att kunna definiera en organisations nuvarande- och framtida tillstånd med avseende på styrning av enterpris search.

Dessutom skulle en sådan modell möjliggöra för organisationer att få maximalt utbyte av sin enterprise search-lösning.

Nyckelord: enterprise search, mognad inom styrning av enterprise search, mognadsmodell, Findability.

(6)

Master of Science Thesis INDEK 2013:59

Enterprise Search Management Maturity - A model for the assessment of an organization’s

maturity level within enterprise search

Felicia Denbu Wilhelmsson Malin Eriksson

Approved

2013-June-11

Examiner

Matti Kaulio

Supervisor

Pernilla Ulfvengren

Commissioner

Findwise AB

Contact person

Maria Björk

Abstract

As organizations grow larger, so does the information base of the organization, and the need to find important information increases along with it. However, many organizations are neglecting the importance of information. Enterprise search lets users search for information in multiple information repositories from one single application, without having to know where the information resides. The focal firm of this research, Findwise AB, is a Swedish IT consultancy firm specialized in delivering search solutions to its customers. It has been shown that not many organizations have a strategy for their internal work with enterprise search. Therefore, Findwise wants to be able to deliver enterprise search strategies to its customers. Such a strategy would facilitate the customers’ development of their internal enterprise search management. However, this requires the ability to define an organization’s current and future state with regard to enterprise search management, something that, up until now has not been possible.

Therefore, the purpose of this study was to develop an enterprise search management maturity (ESMM) model, which Findwise’s consultants can use for assessing an organization’s current state, as well as possible future state within enterprise search management. Through an extensive literature study, in combination with an empirical study consisting of a questionnaire and semi-structured interviews, the ESMM model was developed. The themes of the identified maturity levels of enterprise search management are; knowledge, management, priority, standardization, and sustainability. It is concluded that the parameters to evaluate on each of these maturity levels are the three parameters information, technology, and user. Each of these parameters consists of three specific factors to evaluate, providing guidelines for what to look at in an assessment. This will enable Findwise to define an organization’s current and future state within enterprise search management, as well as enable organizations to gain the most benefit out of their enterprise search solution.

Key-words: enterprise search, enterprise search management maturity, maturity model, Findability.

(7)

Acknowledgements

This master thesis is the final milestone in the researchers’ academic career for a master’s degree in Computer Science and Industrial Engineering and Manage- ment at the Royal Institute of Technology (KTH) in Stockholm.

The work with this thesis has been extensive, but also very interesting and rewarding. We would therefore like to express our gratitude, first of all to the employees of Findwise AB, who let us conduct this research in collaboration with them. Not only did they supply us with an environment in which we could do our research, but they did also let us utilize all the available resources of the company. In particular, we would like to express our greatest gratitude towards Maria Bj¨ork, our supervisor at Findwise, who took us under her wings, and supported us through the entire process of this work.

We would also like to send a big thank you to Pernilla Ulfvengren, from the department of Industrial Economics and Management at the Royal Institute of Technology, who has shown us great support and guidance with our work.

Lastly, we would like to thank our families and colleagues, for their great pa- tience and support.

(8)

List of Figures

1 An illustration of a regular search solution, where the user has to search in all information repositories separately. . . . 6 2 An illustration of an enterprise search solution, where the user can

search through all information repositories from a single search application. . . . 6 3 A holistic view of Findwise’s way of defining Findability (Find-

wise, 2013a). . . . . 8 4 An illustration of the five maturity levels of the information tech-

nology maturity model developed by Leem et al. (2008). . . . 12 5 An illustration of the five maturity levels concerning capability

of the information management maturity model developed by Ladley (2010). . . . . 13 6 An illustration of the five maturity levels concerning capability

of the information management maturity model developed by Ladley (2010). . . . . 13 7 An illustration of the six maturity levels of the usability maturity

model developed by Earthy (1998). . . . 14 8 A summary of the maturity levels of the information technol-

ogy maturity model, the two information management maturity models, and the usability maturity model. . . . 15 9 An illustration of the methods used during this research, in the

chronological order in which they were used. . . . 16 10 An illustration of the difficulties in identifying a current and fu-

ture state within enterprise search. . . . 25 11 An illustration of how much easier it would be to define an or-

ganization’s current and future state (i.e. maturity level) within enterprise search management, if a maturity model existed to use in the evaluation. . . . 26 12 The model created from the results from the questionnaire, show-

ing the different themes that the answers were categorized according to. . . . . 29 13 The identified maturity levels of enterprise search management. . 34 14 The enterprise search management maturity (ESMM) model. Hor-

izontally are the five maturity levels of enterprise search management, and vertically are the parameters to evaluate . . . . 35 15 The identified working process for the ESMM model. . . . 49 16 An example of what the current state of an organization might

look like. Green boxes indicate that these areas have fulfilled the requirements, and red boxes indicate that these areas need to improve. . . . 50 17 An example of what the future state of an organization might

look like. All boxes are green, indicating that the organization has reached its goal. . . . 51

(11)

List of Tables

1 An illustration of which of the research methods of this research that answer to which research questions. . . . 16 2 A description of the interviewees’ roles at Findwise, how long

experience they have with their work, and which Findability dimension that they were interviewed for. . . . 22

(12)

Glossary

Crawler - A program that goes through information repositories, searching for new information objects to sort.

Findability - A state in which all information is findable (Findwise, 2013a) ILM - Information lifecycle management, refers to activities included in the information lifecycle, such as the creation, use, and disposal of information.

Interaction design - The practice of designing interactive digital products, environments, systems, and services.

IT - Information Technology

KPI - Key Performance Indicator. A performance measurement that organizations can use to evaluate the of a particular activity.

Metadata - Data about data, describing different information objects to a search engine.

Persona - An imaginary character representing a typical user of a system, used in interaction design.

Repositories - A place where information objects are stored.

Scenario - A role play describing a potential way of using the system, typically used in interaction design.

User Groups - Refers to different clusters of users.

(13)

1 Introduction

In today’s organizations, there are a large amount of decisions made each and every second of the day, decisions which often need to be made quickly and be based on a proper set of information. One of the major reasons for wrongly made decisions, is the absence of the right piece of information (White, 2012).

It has been shown that when looking at how critical it is for a company’s business to find the right information, the need increases with the size of the company.

However, it has also been shown that the larger the organization, the harder it is to find the right information. It seems as if the importance of finding information is severely neglected by the management of many organizations.

Some of the larger obstacles seem to be that a lot of companies have; multiple repositories, poor search functionality in the search solutions, inconsistency in tagging of the content, and the fact that the employees have low awareness of where to look for the content they are searching for. (Findwise, 2012)

The ‘Enterprise Search and Findability Survey 2012’ conducted by Findwise (2012), showed that 60% of the respondents found it very or moderately hard to find relevant information within their organization. Enterprise search plays a large role in this context, as it provides a tool for searching through all of an organization’s repositories from one single application. Such a search solution removes obstacles for the end-users, and enables quick information retrieval from all of the internal information repositories. The ‘Enterprise Search and Findability Survey 2012’ also showed that only 14.2% of the 170 respondents answered that their organization had a search strategy in place.

The above shows the importance of finding information, but also indicates that concepts facilitating the search process are not being widely used. Organiza- tions tend to not manage their internal information base sufficiently, nor do they have any strategy or management in place for internal search functionality.

This research examines the area of enterprise search management, with the aim to provide a model for evaluating enterprise search management maturity. Us- ing such a model would facilitate the internal development of enterprise search management, as it creates a clear goal for the organization to aim for. This goal could thereafter be used to build a strategy and roadmap upon. The research of this thesis is based on a case study at Findwise AB, a Swedish IT consultancy firm, specialized in search driven solutions.

1.1 Background

The ‘Enterprise Search and Findability Survey 2012’ (Findwise, 2012) showed that most organizations do not have a search strategy in place. Findwise has a plan to introduce enterprise search strategies as a part of the services that they offer to their customers.

(14)

The offering of such a strategy to customers should include a vision, a strategy to follow, and a strategic roadmap that defines the steps necessary to reach the vision. To create such a roadmap, it is first of all necessary to know what the desired future state looks like, but also what the current state of the organization looks like. Defining the current and future state of an organization should visualize the difference between the two states, indicating what factors to build the strategic roadmap on.

1.2 Identified Gap in Current Literature

In current literature, no method for evaluating enterprise search management maturity was to be found. There are some models related to this subject, which will be described in this section. However, these models are not suitable to use when aiming to describe a current and future state within enterprise search on a strategic level. This is due to the fact that these models are merely assessing the operational and functional levels of enterprise search.

White (2010) developed ‘Search Check’, a framework which aims to check current performance level of enterprise search. It is specifically aligned to enterprise search, and considers some management practices, as well as the more functional aspects of a search solution. The model provides an assessment of the current enterprise search performance of an organization. However, it does not provide any clear picture of what a possible future state could look like.

‘Findability Audit’ by Rymert and Magnusson (2011) is a master thesis previously outlined at Findwise. The thesis helps to assess an organization’s enterprise search solution, and the framework includes measures of specific functions included in an enterprise search solution. However, the outlined framework is too technically detailed to be useful on a strategic level.

1.3 Problem Formulation

In order to create a strategic roadmap for an organization, the current and future state of the organization must be defined. However, in the current situation, no model for conducting such evaluations of an organization exists.

1.4 Objective

The objective of this research is to develop a model to use for assessing an organization’s enterprise search management maturity level.

(15)

1.5 Research Questions

To ensure that the most appropriate model is created, the research questions to be answered throughout the work of this thesis are:

• RQ1 Why is there a need for an enterprise search maturity model?

• RQ2 What should the graphical representation of an enterprise search management maturity model look like?

• RQ3 What are the characteristics of the maturity levels of enterprise search management?

• RQ4 What parameters should be evaluated when identifying an organization’s maturity level with regards to enterprise search management?

1.6 Delimitations

This report is based on a case study and was limited to investigate how enterprise search management maturity should be assessed, and develop a maturity model for such an assessment. The work has been focused on the development of the maturity model, hence, no work has been conducted to apply and test the developed model on an organization in real-life settings.

The developed maturity model is thought to be used as a part in strategy work with organizations aiming to implement an enterprise search strategy. Such strategy work will focus on organizations aiming to include at least three different repositories in their enterprise search solution. This results in the model being applicable first and foremost to larger organizations who are implementing extensive enterprise search solutions.

As there are not many players on the market when it comes to enterprise search, all interviews that were conducted throughout this research were held with employees at the focal company of the case study.

1.7 Report Disposition

The remainder of this paper starts with section 2: Theoretical Framework where the theoretical concepts that lie as a basis for this research are described.

The aim of the theoretical framework is to give the reader the knowledge needed to understand the developed maturity model.

In section 3: Methodology, the methods used throughout this research are described in detail. Pros and cons of all methods are discussed. In addition, the reader is given a matrix which shows what research questions that each method aims to answer.

(16)

The next section, section 4: Results describes all the results from this research. The aim is to give the reader an understanding of the empirical findings of this research that eventually led to the developed maturity model, which is presented in the end of this section.

Section 5: Conclusions summarizes the findings of this research, by answering the research questions.

Lastly, section 6: Discussion contains discussions about the research, where the authors of this thesis discuss; the validity, reliability and generalizability of the results, limitations of the research and source criticism, and the application and applicability of the developed maturity model. In the end of this section, the area of sustainability is discussed, followed by recommendations of future research.

(17)

2 Theoretical Framework

The theoretical framework section aims to provide the reader with an understanding of the literature and theoretical concepts that lie as a foundation for this research.

2.1 Enterprise Search Technology

This section aims to describe the area of enterprise search technology, what it is, and how it is used. Within the area of enterprise search technology, there is not a lot of literature. Therefore, all the literature about enterprise search that is used in this research is written by Martin White (2012), an enterprise search expert.

The definition of the term ‘search’ is quite broad. In a dictionary, search is defined as an action for trying to find something that is missing. By only having this definition in mind, search could be anything from a person searching for a book in a library, to a well-implemented method for research. In the sense of this thesis, search is defined as a search solution working on a computer.

Two of the most fundamental components of any search solution are an index and a search query. The index is a list containing data, which indicates the identity and location of a file or a document. The search query is constructed by the person searching for information and typed into a textbox placed on a user interface (e.g. intranet). When a user types a search query into the textbox, a search engine starts to search through the index for the right piece of information. This process is facilitated by the structure and hierarchy in which the information objects are sorted. In addition, the process is facilitated by the fact that information objects are tagged with metadata, which is data about data, describing each unique object. The search engine matches the query against the metadata and then presents the result for the user. (White, 2012)

For this to function well, the index must be properly sorted and well updated.

There are two ways of updating the index. The first way to do it is by using a crawler which crawls through all of the information repositories and updates the index. The other way of updating the index is to let the information repositories push information objects onto the index when they detect them. (White, 2012) The architecture on which all of these different modules are built upon plays an important role for the enterprise search solution. It affects both the performance level of the search solution, and how well the search solution satisfy user demands. (White, 2012)

Enterprise search lets the users search for information in multiple repositories (e.g. e-mail, intranet, the Internet, databases, and file systems), without having to know where the information resides.

(18)

This is illustrated in Figure 1 and Figure 2, where Figure 1 illustrates a regular search solution, and Figure 2 illustrates an enterprise search solution.

Figure 1: An illustration of a regular search solution, where the user has to search in all information repositories separately.

Figure 2: An illustration of an enterprise search solution, where the user can search through all information repositories from a single search application.

(19)

An efficient and well implemented enterprise search solution decreases the amount of time that employees spend on searching for documents they need, as well as decreases redundancy in the information base. Designing an enterprise search solution is not easy, as the end users tend to have very high anticipations on the result. Employees of an organization often know exactly what information object they are searching for. For example, if a person at work searches for a specific presentation, which he or she knows was constructed a year ago, this person will be highly disappointed if the presentation is nowhere to be found.

People searching for information on the web on the other hand, are in many cases only hoping to find something of relevance, as they cannot guarantee that the information that they are searching for has even been created. In addition, that person would probably be pleased with whatever search result that comes up and is linked to the topic that was searched for. (White, 2012)

White (2012) underpins the importance of having a search solution that does not demand users to understand the technology behind it. The enterprise search solution provided to customers needs to support functionality for measurements and updates, since the owners of the solution should be able to self-administer it. The user should be able to select, test, and manage the whole search solution in order to continuously improve it. Quoting White (2012, p.55) “It is very much a question of the whole only being as strong as it weakest link in the chain. If there are some limitations in the way that content is indexed then it does not matter how elegant the user interface looks information will be not able to be found that could be critical to the operations of the business”. An organization must be able to evaluate and test the search solution to be able to identify current weaknesses of it. It is an issue of grasping the whole picture, as the requirements on the search solution depends on user-, organizational-, and information-specific needs, which are developed and changed over time. (White, 2012)

2.2 Findability by Findwise

This section aims to describe Findability, a concept developed by Findwise, which highlights all the areas that should be considered when developing an enterprise search solution.

When working with customers, Findwise soon realized that people do not want to search for information, they want to find it. The search solutions that they build for their customers are about enabling the customers to find the information that they need, not about actually making the information findable.

Therefore, Findwise developed the concept that they refer to as Findability -

“a state in which all information is findable” (Findwise, 2013a). From this, Findwise developed ‘Findability by Findwise’. ‘Findability by Findwise’ is a methodology that Findwise uses to deliver state of the art Findability solutions to their customers, with the aim of helping the customers go from searching to finding.

(20)

The core of the ‘Findability by Findwise’ concept is that when building search solutions, there is more to it than to simply consider the search technology.

Therefore, the Findability concept consists of five dimensions that all need to be considered to reach true Findability. These so-called ‘Findability dimensions’

are; business, users, technology, information, and organization, as can be seen in Figure 3. (Findwise, 2013a)

Figure 3: A holistic view of Findwise’s way of defining Findability (Findwise, 2013a).

2.2.1 Business

The business dimension puts a lot of focus on the fact that the search technology is applied in a way that it supports the business, meets the information needs, and reduces the barriers that impedes the business from reaching its goals and function at its best. The business dimension is about mapping out the different application areas for search driven solutions, and the synergies between these.

In addition, it is about defining goals and measuring points (e.g. KPI:s), and make sure to follow up on the effects of the search investment. (Findwise, 2013b) The search solution has to support the values and goals of the business, as well as also support security and jurisdiction. The system should make use of the full potential of the search technology, but it is also of high importance that the platform in use is well aligned with the overall strategy of the company.

For this to come true, it is important to be well aware of what the objective of the search solution is, and what benefits the search solution brings. (Findwise, 2013b)

2.2.2 Users

The focus in the user dimension is to make sure that the search solution is designed and customized to meet the needs and conditions of the users. To do so, it is vital to know the users, who they are and what their needs are.

(21)

This process can be facilitated by dividing the users into different target groups, and determining which target groups that are prioritized, meaning which users’

needs that should be met first. Knowledge about how the users act in different situations, what they are looking for, what they need to find, and what makes them satisfied is also needed. To establish this knowledge correctly, information about the difference between different target groups is needed. (Findwise, 2013b)

The search experience should be customized based on what information the user needs to find, and personas as well as scenarios are good tools for that. To make sure that the customer company not only implements the search solution, but also keeps up with testing etc., they need to use a peer group of users as well as ‘search ambassadors’. The task of these people is mainly to make sure that the search solution is up-to-date and that search statistics is being used.

In addition, they should continuously evaluate the usability of current search solutions. (Findwise, 2013b)

2.2.3 Information

The focus within the information dimension is to ensure that new information, as well as already existing information, is being published and structured in a way that makes the information optimally findable. It is necessary to know how much information that is to be handled in the search solution (i.e. number of documents and size of data), and what kind of information it is (i.e. web page, document, picture, etc.). It is also important to know the priority of the information, if any information is more vital to find than some other information. For example, it could be the case that new information is more vital to find than old information. For information to be findable, it needs to be well structured. Sometimes templates are used to illustrate how documents should be structured, to ensure a unified structure for the organization. (Findwise, 2013b) Another important factor to consider regarding information is the quality of it, since the quality of the search results is highly dependent of the quality of the information. The underlying factor here is that if the information is not properly tagged and structured, the search engine will not be able to find it.

Metadata tagging plays a major part in the information quality, where information is tagged with appropriate metadata to make it easier to find. This is something that companies can teach their employees, how they should name files, and add metadata to the documents they create. To facilitate this process, companies can incorporate standards for metadata tagging, but also create a controlled vocabulary where synonyms, abbreviations and translations are stored. However, information quality is not only determined by metadata. The information lifecycle needs to be considered, meaning that it is important to keep track of old and outdated information which ought to be removed from the information set. There is also the possibility of having multiple versions or duplicates of documents, and these need to be managed.

(22)

In addition, jurisdiction needs to be considered, to make sure that users are provided access only to the documents and information they should have access to. When looking at the situation not only short-term, but also long-term, it is important to know how the information will change and what the information will look like in the future. For example, perhaps the amount of information will grow over time and this needs to be taken into consideration when designing a search solution. Therefore, analyses need to be performed regarding existing information and routines for information management. (Findwise, 2013b) 2.2.4 Organization

The focus in the organization dimension is mainly to ensure that the recipient of the search solution (i.e. the customer) has an organization and a process in place to manage and further develop the search solution in line with the needs of the business. Someone needs to be in charge of administration and analysis of the search solution, since it is of high importance that the search solution changes along with the organization. The competence and tools that are necessary to analyze and take actions on search analytics need to be developed. (Findwise, 2013b)

2.2.5 Search Technology

The Findability solution should be based on the search technology platform that is most appropriate for the specific customer, and this platform should be utilized to its full potential. It needs to be established which the most appropriate platform is. If the customer is currently using the most appropriate platform, then the solution can be further developed from its current state. The choice of platform is determined by wanted functionality, source system, availability of components, price, technological capacities of the company, flexibility, security, and management of the recipient organization. If the customer company has any licenses, information is needed regarding when these expire, and decisions must be made whether or not to keep them. The company might have an IT- strategy regarding decisions like; existing or new solutions, buying or in-house development, and open source or commercial. If such an IT-strategy exists, this needs to be taken into consideration when planning the development of the Findability solution. Total cost of ownership also needs to be taken into account, as well as the fact that the technological architecture can grow over time. (Findwise, 2013b)

A major part in the search technology dimension is the features of the search engine, like the fact that the search engine has to deliver answers fast enough.

However, the search engine can also be built to assist the user searching for content. This can be done for example by providing spelling suggestions, auto- completing the search query, or giving the user the possibility to filter the search results based on his or her preferences. (Findwise, 2013b)

(23)

Every search made by a user can be stored in a log file and used to perform search analytics. This is a very useful tool, since it demonstrates which content that users can find, but also the content that they do not find. This information can be used to analyze the reasons to why users do not find the desired content, whether it is because the users use a different vocabulary and hence the ‘wrong’

search queries, or simply because the content does not exist. (Findwise, 2013b)

2.3 Maturity Models

This section describes different maturity models within the areas of information technology, information management, and usability. In the end of the section, a summary of similarities of the maturity models is presented.

One of the initial questions for this paper was how to assess an organization’s maturity level with regards to enterprise search. A maturity model facilitates for managers, by giving them a step-by-step framework to use when evaluating the current state and capability of an organization, within a specific field of management. Such a maturity model explains different stages in a logical order, steps that need to be undertaken in order for the maturity to grow. In this sense, ‘mature’ would define the last step of the hierarchy, which is where an organization can gain the greatest value. On the contrary, the first step ought to visualize the most immature level, on which little or no value is experienced from the particular area of interest. (Leem et al., 2008) A framework like this should not evaluate an organization in terms of good and bad, but instead try to evaluate the current ability to perform (Ladley, 2010). A maturity model is a very useful tool to use in consultancy environments, as it provides technical specialists with a reasonable picture of the customers’ initial knowledge on the subject at hand. It defines a scale on which the customer could be graded.

In addition, it is a very helpful tool in the continuing work, as the technicians will know how to communicate their knowledge to that particular customer.

Shortly, it defines what methods and tools the customer will be able to accept and handle. (Earthy, 1998)

Important areas to consider when trying to identify an organization’s current maturity level are; culture, business environment, current state of IT capability and infrastructure, usefulness of existing content, effectiveness of leadership in providing business vision, the potential for communication and collaboration within the organization. It is important to grasp an overall understanding of the whole organization, and a maturity level framework should provide the tools for the evaluation, but also facilitate in picturing a suitable goal to achieve in the future. (Ladley, 2010)

(24)

2.3.1 Information Technology Maturity Model

Leem et al. (2008) have explored the topic of IT maturity models and present a validated and statistically tested IT maturity model consisting of the levels;

initiation, recognition, diffusion, control, and integration. The maturity levels of this model can be seen in Figure 4.

Figure 4: An illustration of the five maturity levels of the information technology maturity model developed by Leem et al. (2008).

The first maturity level is the initiation level. At this level, small IT investments are initiated, aimed at developing systems in isolated environments.

The next step to undertake is recognition. At this level, the importance and competitive advantage of well implemented IT systems are recognized. Differ- ent IT activities such as planning, acquisition, development, and maintenance are identified and become a part of the business processes. Thereafter follows the third level, diffusion, in which IT becomes a part of all value chains of an enterprise that is being managed by an independent IT department. At this level, IT does also acquire support from business parts of the organization. The following level is called control. At this level, the utilization and satisfaction of IT investments are increased by better control of IT processes, with activities such as periodical planning and rules of documentation. At the last level, integration, business processes are integrated by the use of IT, and there is a mutual agreement of the performance level to achieve in the enterprise. (Leem et al., 2008)

2.3.2 Information Management Maturity Model

According to Ladley (2010), two important areas to evaluate when it comes to information management maturity are capability and organization.

The capability perspective deals with the capability of the organization. Within this perspective there are five different levels to mature in; initial, repeatable, defined, managed, and optimized, as can be seen in Figure 5.

(25)

Figure 5: An illustration of the five maturity levels concerning capability of the information management maturity model developed by Ladley (2010).

At the first level, initial, the culture is chaotic and reactive. There are no business rules or common ways of handling information, and information objects are in most cases owned by individuals in the organization. At the next level, repeatable, information is handled and owned by departments of the organization, and there are some common rules for how to handle information.

The third level is referred to as the defined level. At this level, the organization has a more holistic view of information handling. Processes and structures are identified, and the organization takes a more strategic approach by developing abilities such as the ability to identify possible cost-reduction activities. The fourth level is referred to as the managed level, where analysis is carried out and internal assets are tracked and evaluated with the aim of securing value acquisition. The last level is referred to as the optimized level, where information handling becomes an integrated part of the fabric of the organization, with efficient measures in place to continuously evaluate the performance. The organization becomes more proactive and increases the ability to cope with change.

(Ladley, 2010)

The organization is evaluated on the basis of internal culture together with the ability to perform. The maturity levels of the organization area can be seen in Figure 6.

Figure 6: An illustration of the five maturity levels concerning capability of the information management maturity model developed by Ladley (2010).

The first level is referred to as the operate level, in which information processes and tools are individual and formal. The culture is identified as ‘skills combined with charisma wins’. The next level is referred to as the consolidate level, in which processes are streamlined and measured, and there exist team- work within functional units. The third level is referred to as the integrate level. At this level, workers are thinking enterprise wide and there exists collaboration between peer groups. The following level is referred to as the optimize level. At this level, the organization is capturing tacit knowledge, and performs constant market alignments as well as incremental improvements.

(26)

The last level is the innovate level, which is identified by an organization which has an innovative mentality and expects change. (Ladley, 2010)

2.3.3 Usability Maturity Model

Earthy (1998) has developed a maturity model measuring how human-centered an organization is. A human-centered organization develops interactive IT systems based on the knowledge and skills of the employees, with the aim of making all systems highly usable. The model provides a reference system to use as a basis for evaluating an organization’s understanding of user-focused design, and the importance of such a design. As can be seen in Figure 7, there are six different maturity levels, namely; unrecognized, recognized, considered, implemented, integrated, and institutionalized.

Figure 7: An illustration of the six maturity levels of the usability maturity model developed by Earthy (1998).

The lowest and most immature level illustrates an organization which has not recognized any need for user-focused processes or systems. On the contrary, the most mature level pictures an organization with prominent focus on the internal user. At this level, the internal users have been recognized and the organization is continuously working on improving systems with regards to the users. (Earthy, 1998) The model illustrates how an organization moves from being unaware to becoming more aware of the users, a change enabled by an internal culture shift. Such a culture shift in an organization causes the issue to be recognized and prioritized, highlighting the importance of efficient imple- mentations and improvements of functions. The outcome of performing such a culture shift, is an organization where human-centeredness is strongly embedded in the organization.

2.3.4 Summary of Characteristics

Some common themes can be identified among the previously described maturity models. The first levels are consistently all about initiation and recognition.

These levels describe the learning- and implementing procedure of an organization. The following levels identify characteristics such as diffusion, integration and definition. Thereafter follows a phase of identifying control, management and optimization. On the last maturity level, the organization is characterized by an innovative culture, optimized processes, and fully integrated systems. In general, most of the maturity models have a scale ranging from recognition and initiation, through repeatable, managed, and/or integrated levels, aiming towards optimization and innovation.

(27)

A collocation of the different levels can be seen in Figure 8.

Figure 8: A summary of the maturity levels of the information technology maturity model, the two information management maturity models, and the usability maturity model.

2.4 Strategy

This section aims to describe the area of strategy. In particular, this section discusses the relation between strategy and information technology systems, and how a strategic roadmap supports this relation.

IT-systems play a large role in providing today’s businesses with a competitive advantage, but the provided advantage is in many cases severely under- utilized. Reasons for this are; senior management’s ignorance of information technology and its potential uses, poor communication between the information systems group and the rest of the business, resistance to change, lack of focus for competitive advantage, and lack of instruments to measure the benefits gained. For organizations to overcome this, and start to better utilize their internal IT-systems, they must strengthen the link between corporate strategy and the development of their internal IT-systems. (Bakos and Treacy, 1986) A strategic framework enables organizations to understand and align current businesses towards long-term goals, instead of focusing on short-term financial objectives (Kaplan and Norton, 2006).

In terms of technology, road mapping has become a widely used management technique for supporting innovation and strategy work. A roadmap is a structured framework that enables communication, and it should be used to answer the questions: ‘Where do we want to go? ’, ‘Where are we now? ’, and ‘How can we get there? ’. It provides a visual view of the system in question, through the use of some descriptive key perspectives. In addition, a roadmap supports the strategic dialogue between organizational units. (Phaal and Muller, 2009)

(28)

3 Methodology

This section will describe the methods used during this research, in order to prove their scientific value and the validity of the research design.

Figure 9 illustrates all the methods used during this research in a chronological order.

Figure 9: An illustration of the methods used during this research, in the chronological order in which they were used.

Table 1 illustrates which of the methods used during this research that aim to answer which research question.

RQ1 RQ2 RQ3 RQ4

Introductory Meetings x x

Literature Study x x x

Model Prototyping x

Questionnaire x x

Interviews x x

Feedback Sessions x x x x

Table 1: An illustration of which of the research methods of this research that answer to which research questions.

(29)

3.1 Introductory Meetings

During the first two weeks of this research, introductory meetings with experienced employees at the focal company were outlined. The purpose of these meetings was to understand the general concepts of enterprise search, but also to understand how the focal company is working with these on a more strategic level. The reason for this was to understand the scope of the research, as well as the background to the problem at hand. Doing this should ensure that the problem could be rightfully identified. Having a well identified problem is important, as it will increase the chances of fulfilling the purpose of this research.

In total, four meetings were outlined, and each of the meetings lasted for about one hour. The first meeting was with the researchers’ supervisor at the focal company. This meeting aimed at giving a more general introduction to the problem, and provide possible directions for further research. After this, two sessions concerning the more strategic aspects of enterprise search management were outlined. These sessions provided information about how the focal company works with enterprise search on a strategic level. These introductions were followed by an additional session with the supervisor at the focal company, where some fundamental aspects of enterprise search technology were described.

3.2 Literature Study

With the information gained from the introductory meetings, the work pro- ceeded with a literature study. Collis and Hussey (2009) argue that a literature study should provide some specific areas of interest to concentrate on during the following work. With this in mind, the objective of the literature study was to investigate the current body of knowledge about the topic at hand. The objective with the literature study was also to deepen the researchers’ knowledge about topics related to enterprise search. With the aim to conduct semi-structured interviews, there was a need for the researchers to possess a quite broad knowledge base. The interviews were designed to be open-ended, which puts a higher amount of pressure on the researchers to create probing questions. Therefore, this quite extensive literature study was outlined.

Literature and information regarding the technical parts such as enterprise search, search technology, and Findability, were first and foremost found on the focal company’s website and intranet. In particular, a lot of effort was put on understanding the different aspects of information management, search technology, and usability, as these were the main areas of the interviews. In addition, the researchers read the book ‘Enterprise Search’ by Martin White.

White (2012) is a well-known enterprise search professional, who in this book tries to explain enterprise search to managers who are or should be working with the subject in some way.

(30)

As one of the main issues of this research was to identify different maturity levels, a multitude of various maturity model literature was read. In particular, literature related to information management maturity, information technology maturity, knowledge management maturity, and user experience maturity was read.

The scientific articles regarding maturity models, as well as literature about knowledge management and sustainability, were searched for using Primo, a search service on the website of the KTH library (i.e. KTHB). Primo searches through a variety of databases, such as Scopus, Nationalencyklopedin, Science Direct, and Emerald. As a complement, Google Scholar was used, as it tended to give better search results when the search queries were broad and more general.

When the literature had been reviewed, there were still some issues concerning what kind of model to use for visualizing enterprise search management maturity. In order to determine which model that was the most suitable, the researchers moved on with the following stage of model prototyping.

3.3 Model Prototyping

The model development process started by evaluating a set of different management- and strategy models. The purpose of these prototyping sessions was to find out what graphical representation of an enterprise search management maturity model that would be most suitable. During two workshops with experienced consultants at the focal company, an initially broad number of models were reduced down to one model, which was considered as the most suitable. The workshops both lasted for about an hour.

The process of finding a model to use for the illustration of the enterprise search management maturity model, started with a broad literature screening phase.

During this phase, a total of eight different models were collected. These models were presented in the first prototype workshop, which consisted of one consultant from the focal company and the two researchers. In this workshop, the eight models were discussed and critically evaluated. The results were analyzed and the models that were considered to be too complicated and abstract were dismissed. This resulted in four remaining models, which were all found to be pedagogical.

The remaining four models were further developed and aligned towards the context in which the final model should function. Thereafter, these four models were discussed in detail in a second workshop with the two researchers, the consultant from the first workshop, and an additional senior consultant. The purpose of this workshop was to get the opinion from a senior consultant with long experience in enterprise search management.

(31)

The second workshop resulted in one final model, which was decided to be the most suitable for this specific context. This decision was based on the fact that the final model was considered to be easy to use to develop a roadmap. In addition, the final model was easy to understand and communicate.

The model prototyping was performed in an early phase of the research process, where the researchers’ possessed knowledge about the problem at hand was still limited. Because of this, it was somewhat difficult to understand what kind of model to search for. However, discussing different models with more experienced consultants did provide much deeper background knowledge about the problem and how it could be solved. Therefore, having these discussions ensured that the developed model was based on the most suitable graphical representation. In addition, the prototyping sessions provided the researchers with a more developed view of what to investigate as well.

3.4 Questionnaire

When the model prototyping was finished, the main question to answer was the fourth research question of this research, namely ‘What parameters should be evaluated when identifying an organization’s maturity level with regards to enterprise search? ’. The previously outlined problem definition and model prototyping session had revealed a need for some kind of categorization into different enterprise search management maturity levels. As the researchers did not have any experience about what characteristics this should be based on, it was decided to send out a questionnaire to answer this. A questionnaire is a method for collecting primary data from a chosen set of respondents. From answering a carefully chosen list of questions, the objective is to produce reliable results.

(Collis and Hussey, 2009)

The questionnaire was distributed by e-mail and sent to all employees currently working as project managers at the focal company, which at the time were 30.

The goal was to get a better view of how these project managers would grade and describe the current state and desired future state of the customers in their current projects. The project managers were asked to grade the maturity of the customer on a scale from one to five, and describe on what basis they did this grading. In total, the project managers were asked four questions, which can be found in Appendix A. The intention was to find out if there were any common denominators among the answers, which could be a good basis for the development of the model. By choosing to send this questionnaire to project managers only, the chances of getting more comprehensible answers increased. For example, if the questionnaire was sent to the programmers only, chances were that the answers would be more focused on the technical aspects and details of the search solution. The same yields for other specialists such as information- or user-experienced personnel. The questions in the questionnaire were written in English, and the participants were asked to give their answers in either English or Swedish.

Enterprise Search Management Maturity: - A model for the assessment of an organization’s maturity level within enterprise search management

Enterprise Search Management Maturity

A model for the assessment of an organization’s maturity level within enterprise search management

FELICIA DENBU WILHELMSSON MALIN ERIKSSON

Att mogna inom styrning av enterprise search

- En modell för att utvärdera en organisations mognad inom styrning av enterprise search

FELICIA DENBU WILHELMSSON MALIN ERIKSSON

Att mogna inom styrning av enterprise search

- En modell för att utvärdera en organisations mognad inom styrning av enterprise search

av

Felicia Denbu Wilhelmsson Malin Eriksson

Enterprise Search Management Maturity

- A model for the assessment of an organization’s maturity level within enterprise search management

Felicia Denbu Wilhelmsson Malin Eriksson

Acknowledgements

Contents

List of Figures

List of Tables

Glossary

1 Introduction

2 Theoretical Framework

3 Methodology