Efficient user interaction for clinical diagnosis using digital volume images

Full text

(1)LiU-ITN-TEK-A--08/088--SE. Efficient user interaction for clinical diagnosis using digital volume images Andreas Fältskog 2008-06-17. Department of Science and Technology Linköping University SE-601 74 Norrköping, Sweden. Institutionen för teknik och naturvetenskap Linköpings Universitet 601 74 Norrköping.

(2) LiU-ITN-TEK-A--08/088--SE. Efficient user interaction for clinical diagnosis using digital volume images Examensarbete utfört i vetenskaplig visualisering vid Tekniska Högskolan vid Linköpings universitet. Andreas Fältskog Handledare Magnus Björklund Examinator Örjan Smedby Norrköping 2008-06-17.

(3) Upphovsrätt Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under en längre tid från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/ Copyright The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: http://www.ep.liu.se/. © Andreas Fältskog.

(4) Abstract Medical imaging is going through a continuous development leading to more available information for the reviewing doctors. The information is a powerful tool in the strive to make reliable diagnosis but it can also be a challenge for the doctor to make use of all the information. This thesis investigates if the computer-based review workplace can be made more efficient using other input devices than the traditional mouse and keyboard. To acquire knowledge about the existing user interactions six interviews have been conducted at two Swedish hospitals. In the thesis a 3D mouse has been integrated into Sectra’s review workstation to show what value a complementary device can bring. The interviews show that the doctors are quite satisfied with the existing workplace but there are areas of improvement. This together with the integration of the 3D mouse gives a hint about the complexity of finding a device that adds enough value to take place on the reviewing doctor’s desk.. v.

(5)

(6) Acknowledgments Since this thesis conclude my studies of Applied Physics and Electrical Engineering at Linköping University I would like to thank my wonderful wife Camilla for her support and patience. I could not have made it myself, although our kids are wonderful, they are not so supporting and patient when you want to study on your own.. vii.

(7)

(8) Abbreviations 2D Two dimensional 3D Three dimensional CR Conventional Radiography CT Computer Tomography IDS Image Display System, IDS5 and IDS7 are registered trademarks of Sectra Imtec AB MPR Multiplanar Reconstruction MRI Magnetic Resonance Imaging PACS Picture Archive and Communications System. ix.

(9)

(10) Contents 1 Introduction 1.1 Problem description 1.2 Purpose . . . . . . . 1.3 Scope . . . . . . . . 1.4 Method . . . . . . . 1.5 Structure . . . . . .. . . . . .. . . . . .. . . . . .. 2 Background 2.1 Review of medical images 2.1.1 Image sets . . . . . 2.1.2 2D Image Window 2.1.3 3D module . . . . 2.1.4 MPR module . . . 2.2 Interaction devices . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . . . . . . . .. . . . . .. 1 1 1 2 2 2. . . . . . .. 3 3 4 5 6 6 7. 3 Interview 3.1 Interview setup . . . . . . . . . . . . . . 3.2 The answers . . . . . . . . . . . . . . . . 3.2.1 Hours at the PACS Workstation 3.2.2 Types of examination . . . . . . 3.2.3 The use of MPR/3D . . . . . . . 3.2.4 Size of stacks . . . . . . . . . . . 3.2.5 Bottlenecks . . . . . . . . . . . . 3.2.6 Existing interactions . . . . . . . 3.2.7 Health problems . . . . . . . . . 3.2.8 The desk space issue . . . . . . . 3.2.9 3D vs. 2D . . . . . . . . . . . . . 3.2.10 Visions . . . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. . . . . . . . . . . . .. 11 11 11 11 11 12 12 12 12 12 13 13 13. 4 Implementation 4.1 Implementation basics . . 4.1.1 Hardware . . . . . 4.1.2 Coordinate system 4.1.3 Software . . . . . . 4.2 Interaction rules . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. 15 15 15 15 15 16. . . . . .. . . . . .. . . . . .. . . . . .. xi. . . . . .. . . . . .. . . . . .. . . . . ..

(11) xii. Contents 4.2.1 4.2.2 4.2.3. 2D Image Window . . . . . . . . . . . . . . . . . . . . . . . 3D module . . . . . . . . . . . . . . . . . . . . . . . . . . . MPR module . . . . . . . . . . . . . . . . . . . . . . . . . .. 16 16 17. 5 Result 5.1 Interaction in the Image Window . . . . . . . . . . . . . . . . . . . 5.2 Interactions in the MPR module . . . . . . . . . . . . . . . . . . . 5.3 Interactions in the 3D module . . . . . . . . . . . . . . . . . . . . .. 19 19 20 20. 6 Discussion, conclusion and future work 6.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 21 21 23 23. Bibliography. 25. A Interview questionnaire. 27. B Interview answers B.1 Radiologist 1, Hospital B.2 Radiologist 2, Hospital B.3 Radiologist 3, Hospital B.4 Radiologist 4, Hospital B.5 Radiologist 5, Hospital B.6 Radiologist 6, Hospital. A A B B A A. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. 29 29 30 31 31 32 33.

(12) Chapter 1. Introduction This chapter will give an introduction to the problem, purpose and structure of the thesis.. 1.1. Problem description. Looking back 10 years from now, the work at radiology departments typically involved film based examinations reviewed on light boxes. Since then, many radiology departments have moved from the analogue film technique to digital computer-based diagnosis. This digital system is often referred to as a PACS, Picture Archiving and Communication System. This digital transition has enabled a far more efficient work flow in the radiology department. Furthermore, the digital work place enables the ongoing increase in data volume that comes with more advanced medical imaging devices such as Computer Tomography (CT) or Magnetic Resonance Imaging (MRI) scanners. The increased data volume available for the radiologist is of course important in the strive to make reliable diagnosis but it can also be a challenge for the radiologist to make benefit from the mass of information. This thesis will focus on the interaction techniques in the PACS and the question is: Is the traditional technique, based on mouse and keyboard, still adequate or is there a need to add a more advanced input device to cope with the increasing amount of data? As an example, today the mouse wheel is used to scroll in image stacks. If the image stacks contains thousands of images this can be quite a strenuous work and cause problems with muscles or wrists.. 1.2. Purpose. The purpose of this thesis is to investigate how to make the clinical review process more effective and efficient by using an alternative input device, both from a diagnosis perspective and from an ergonomic perspective. 1.

(13) 2. Introduction. Other work has been made showing that scrolling in large data volumes can be made more efficient by using other input devices than the mouse and the keyboard [4]. This thesis will go one step further and not only focus on scrolling through large image stacks but as well investigate if the review process can benefit from putting other interactions such as panning, zooming, gray scale settings etc into an alternative or complementary input device.. 1.3. Scope. The thesis does not intend to be an evaluation of input devices. The intention is to make a study with input from radiologists on how the clinical review process can be made more efficient using an alternative or complementary input device. The technical solution will be implemented on Sectra’s latest application platform and the IDS7 workstation. The user interviews will be based on the old platform workstation IDS5 as this is the one available for most end users for the time being. The user interactions in IDS5 is very much the same as in IDS7 so this should not affect the result. The implementation will focus on enhancements of the interactions in the MPR module in the IDS7 workstation.. 1.4. Method. The thesis consists of two parts, one study part and one implementation part. The intention of the study is to collect input from radiologists regarding the existing interaction techniques, and what the review process can gain from a complementary input device. To collect input, interviews with a number of radiologists from two different Swedish hospitals was performed. Most likely the results of the interviews would have been more reliable if more than two hospitals were visited but it would have taken too much time. The implementation part integrates one complementary input device and the result from the study decides the interaction rules.. 1.5. Structure. The report starts by providing background theory of the problem, describing the clinical review process and the interaction techniques in Sectra’s PACS workstation. This will be followed by a chapter containing detailed outcome of the interviews. The implementation part is described in the upcoming chapter followed by a chapter summarizing the results of the interviews and the implementation. The last chapter contains discussion, conclusion and future work..

(14) Chapter 2. Background This chapter will give basic theory that is needed to understand the problem presented in the thesis.. 2.1. Review of medical images. Imagine that you suffer from a bad ache in your back. You pay a visit to your local health care center to consult a doctor. To be able to make a statement about your condition the doctor writes a referral letter to the radiology department at the main hospital. You get an appointment the following week and a nurse takes care of you and performs a CT scan of your back. After a quality check, the images from the CT scan is sent to the PACS and you are sent home.. Figure 2.1. A radiologist reviewing images at a PACS workstation [1]. After some preparations, the images are reviewed by a doctor specialized in xray images or radiology, also known as a radiologist. Figure 2.1 depicts a radiologist working at a PACS workstation. After a thorough review of your images and perhaps some consultation with his peers the radiologist writes an answer to the 3.

(15) 4. Background. referring doctor at your local health care center. One week later you receive a call from your local doctor who tells you about the result from the CT scan and proposes a suitable treatment. What did the radiologist do during the review of your images to be able to make a statement about your condition? Sectra PACS Workstation IDS7 offers mainly two ways of reviewing medical images: a two-dimensional image window and a three-dimensional volume rendering module. These two will be covered in the following sections as well as a module called Multiplanar Reconstruction, MPR, which is used to reformat a stack of images. Before that a short explanation is needed of how images from e.g. a CT scan is organized.. 2.1.1. Image sets. The resulting set of images from a radiology examination varies. As an example , a CR (Conventional Radiography) examination projects the tissue passed by the x-rays onto one single image while the CT scan used in the scenario above produces thin slices of the body part scanned and generates hundreds or thousands of images. The examination can also have a resolution in time. One area of the body is studied at different points in time to examine e.g. blood flow in a vessel. Figure 2.2 shows a CT scan equipment just about to examine a patient. The images are greyscale images where the greyscale level depends on the radiological density of the tissues in the body. Back to the CT scan in the example. The original slices from the scan are lined up perpendicular to the axis that goes from the patients head to his feet, the body part is said to be sectioned in the axial plane. In many cases it is interesting to review the images in other planes as well, the coronal plane (from the nose to the back of the head) or the sagittal plane (from side to side). Images in these additional planes are normally generated in the CT scan equipment and sent to the PACS. There is also a need to use an arbitrary plane such as a plane perpendicular to a blood vessel or other body structures of interest. Images in such a plane can be generated in the CT scan equipment but also in Sectra’s already mentioned MPR module. It is important to mention that even if the original image set contains thousands of images, the set that ends up in PACS for reviewing is often downsampled. One reason for doing the downsampling is to enhance the image quality. The original images with a slice separation of often less than one millimeter are quite noisy. The downsampling algorithm acts a lowpass filter and reduces the noise, the resulting slice separation is typically five millimeter. Another aspect of the downsampling is that the radiologist rarely need the fine resolution in their review work due to the structure of the tissue in the human body. Using higher resolution than necessary would only delay the work..

(16) 2.1 Review of medical images. 5. Figure 2.2. A CT scan equipment in action [5]. 2.1.2. 2D Image Window. The two-dimensional review is the conventional and most common method in use. The tools provided in the PACS workstation are only a help for the radiologist in his review, they will not make any diagnosis for him. The radiologist looks for abnormalities by studying a single image or by scrolling through a stack of images. In Sectra’s PACS Workstation this interface is called the Image Window, figure 2.3 shows a screen shot from the Image Window. The most important tools for interacting with the Image Window are: Panning Moving the image in the screen plane to put an interesting region in the center of the screen. Zooming Zoom the image to see more details in a region of interest. There are two different kind of zoom, one continuous zoom and one with discrete steps. The continuous zoom is used by first pressing the control key on the keyboard and then moving the mouse forwards or backwards. Scrolling Scroll to another image in the image stack. There are several ways of scrolling in the Image Window. To scroll one image at a time one can use Page-Up and Page-Down on the keyboard or the mouse wheel. By first pressing left and right mouse button one can scroll continuously by moving the mouse back or forth. From the application menus it is also possible to move to the first or last image in the stack. Adjustment of Window/Level The Window/Level settings are used to select which greyscale levels to show. For example: If the radiologist is interested in bone structures he can select a Window/Level setting that emphasize the greyscale levels belonging to bone tissue..

(17) 6. Background. Figure 2.3. Screen shot from IDS7 Image Window. Make annotations Annotations are markings in the image such as a text or a measurement that helps the radiologist to mark abnormalities found. To facilitate these interactions the mouse and the keyboard is used. The radiologist can also configure his own shortcuts in the PACS workstation for quick function access.. 2.1.3. 3D module. In this module the images from the examination are used to render a three dimensional image volume that can be used for reviewing or visualization of the examination data. See figure 2.4 for a screen shot from IDS7. In the 3D module the interactions from the Image Window are complemented with for example: Rotate Freely rotates the 3D volume by pressing the left mouse button and then move the mouse. This interaction is the default interaction of the 3D module. Clip Plane A tool to cut out a subvolume of the 3D volume by applying a Clip Plane. One can also set a thickness on the clip plane and thereby keep the information that is between two parallel clip planes.. 2.1.4. MPR module. The MPR module in IDS7 provides means for reviewing images in an arbitrary sectioning plane. The MPR view is split into four tiles of which three are reference tiles, see figure 2.5. The reference tiles are the smaller ones at the left part of the screen. The reference tiles are used to adjust the sectioning plane and the resulting image is viewed in the fourth larger tile..

(18) 2.2 Interaction devices. 7. Figure 2.4. Screen shot from the 3D module in IDS7. The MPR module has roughly the same interactions as the Image Window but has additional interactions for adjusting the sectioning plane. The module also offers a feature called Slabbing, which is the same as the downsampling introduced in section 2.1.1. Panning The panning in the MPR view is panning in the fourth tile, the tile showing the resulting image stack. Zooming. Zooming is also performed in the fourth tile.. Sectioning-plane rotation The rotation is done using the sectioning lines in either of the three reference tiles. The lines can be translated or rotated using the mouse. Slab mode and slab thickness Slab mode is the algorithm used when the image stack is downsampled. There are three different slab modes: Maximum Intensity Projection (MIP), Minimum Intensity Projection (MinIP) and Average Intensity Projection [3]. Slab thickness is the wanted image separation in the calculated image stack. Slab mode and thickness is set using the toolbox on the right side of the screen or from the application menus.. 2.2. Interaction devices. The characteristics of an input device is often described by the degrees of freedom, DOF, how many axis it can operate on. An input device with smaller number of.

(19) 8. Background. Figure 2.5. Screen shot from the MPR module in IDS7. DOFs, for example a mouse with 2 DOFs, can emulate a device with more DOFs by the use of buttons or modifier keys [2]. Unfortunately the approach of emulating more DOFs easily gets too complicated for the user who has to remember which combination to use in different situations. In this thesis a 6 DOF input device from 3DConnexion called SpaceExplorer, see figure 2.6, has been used for the implementation part. The choice of device for the implementation was more of a coincidence and not preceded by any special research, the goal was simply to have a device with more DOFs than the mouse.. Figure 2.6. SpaceExplorer from 3DConnexion. In the thesis the complementary input device often is referred to as a joystick. The reason for that is the nomenclature used in DirectInput which is a part of Microsoft’s DirectX framework and that has been used to integrate the input device into IDS7. A device in DirectInput is either a mouse, a keyboard or a joystick. Joystick includes 3D mouse, driving console and other devices with more.

(20) 2.2 Interaction devices. 9. degrees of freedom than the mouse and they all use the same data structure to report input from axes and buttons..

(21)

(22) Chapter 3. Interview This chapter will cover the results of the interviews conducted in the thesis.. 3.1. Interview setup. The aim of the interviews was to find out how the radiologists experience working with existing PACS workstation (IDS5) and what they think about their work in the future. The interviews were short qualitative interviews based on a questionnaire, see appendix A. The interviewees were six radiologists from two different hospitals in Sweden with different level of seniority and experience. Detailed answers from the interviews are collected in appendix B.. 3.2. The answers. The result of the interviews is presented per question.. 3.2.1. Hours at the PACS Workstation. The first question asked was the amount of time spent at the PACS workstation per day. This is important to understand the need of an effective and efficient workstation. The time spent reviewing images among the interviewees varies from approximately 6 to 8 hours per day. This is a fair amount of time and it definitely justifies the search for the most effective and efficient workplace.. 3.2.2. Types of examination. The next question was what types of examination the interviewee works most with. The purpose was to find out if the radiologist works with stacks of images or with single images. 11.

(23) 12. Interview. The answer is that all interviewees more or less work with a mix of MRI, CT, CR and other types. The conclusion is that they all work with stacks of images in their review.. 3.2.3. The use of MPR/3D. The question was if the interviewee uses MPR or 3D in their daily work. This question revealed that the use of MPR and 3D varies a lot. Some prefer to do the MPR or 3D on the imaging device due to the fact that they find it better than the modules in IDS5. The performance is often an important issue, they find the IDS5 modules too slow. The demands on the functionality differs however and one radiologist is fully satisfied with MPR and 3D in IDS5. The general conclusion is that most of the interviewees are not satisfied with the functionality for MPR and 3D in IDS5 and they do not use these modules on a regular basis.. 3.2.4. Size of stacks. How large is the stacks that the radiologists work with? The size of one image stack varies roughly between 20 and 500 images and in most cases the review involves comparing a new examination with older examinations and the sum of all stacks easily ends up at several thousands of images.. 3.2.5. Bottlenecks. Do you encounter any bottlenecks in your work? The general opinion is that the amount of data is a bottleneck. Both when it comes to the time spent on reviewing them but also the time it takes for IDS5 to load the data into the application. One interviewee compares the digital workplace with the previously used light box and points out that the physical area available for review was larger on the light box. The performance of the MPR module in IDS5 is also experienced as a bottleneck.. 3.2.6. Existing interactions. How do you rate the existing interactions? All interviewees are generally satisfied with the existing interactions in IDS5. The ability to program individual shortcuts is mentioned as a very valuable feature. The mouse wheel scrolling in large image stacks is however mentioned as a problem.. 3.2.7. Health problems. Have you had any problems with muscles or wrists? One of the interviewees has experienced “cold fingers” due to strenuous mouse scrolling. The others has had no health problems related to the review work..

(24) 3.2 The answers. 13. However, most of them point out that a full working day at a computer workstation can be strenuous although they have not had any problems so far.. 3.2.8. The desk space issue. Is it possible to add another device to your workplace? This can be a problem as the workplace already has a keyboard, a mouse and a dictaphone, a device used by the radiologist to record his diagnosis by speech. The general opinion is that it is possible to add another device as long as it adds enough value to the workplace.. 3.2.9. 3D vs. 2D. What do you think of 3D diagnosis vs. 2D diagnosis? Most of the interviewees consider the 3D diagnosis as a complement to 2D diagnosis and thinks that this will be the case in the near future as well. 3D is in this case used mainly to visualize already found abnormalities. One interviewee reports however that 3D is already in frequent use for diagnosis of some types of examinations, such as bones and blood vessels. The same interviewee mentions that he uses 3D on the imaging device, producing a film sequence that is sent to PACS. He prefers to do this in the PACS workstation but he founds the performance inadequate in IDS5.. 3.2.10. Visions. How is the review work performed in ten years from now? The interviewees can not foresee any dramatic changes to their work. The tools and imaging devices in use today will continue to be refined. More visions: • Decrease in CT exams due to regulatory issues regarding radiation dose. • Increase of MRI exams (if the MRI equipment gets fast enough). • More functional studies, looking at short sequences showing organs in action. • Tools for computer aided diagnosis • More distributed work, Teleradiology world wide.

(25)

(26) Chapter 4. Implementation This chapter describes how the joystick device is integrated into the PACS workstation.. 4.1 4.1.1. Implementation basics Hardware. The input device that has been used is a 3D mouse from 3DConnexion called SpaceExplorer. The SpaceExplorer has six degrees of freedom: translation along x, y and z and rotation about x, y and z. It also has fifteen buttons that are programmable for the user. The device is self centering, i.e. when the knob is released it always returns to the same initial position and the sensor data is zero on all axes. The more you translate or rotate the knob, the greater the sensor output.. 4.1.2. Coordinate system. The coordinate system used in the implementation is copied from the SpaceExplorer device. If the device is laying on the desk the “desk plane” is spanned by the x-axis (from left to right) and the y-axis (forward). The z-axis raises from the desk.. 4.1.3. Software. The implementation is made in Sectra’s latest PACS Workstation IDS7. The development platform for IDS7 is built on Microsoft’s programming language C# and uses components from the .NET framework. The software implementation is divided into two classes, the JoystickManager and the JoystickInteraction. The JoystickManager handles the communication between the joystick device driver and Sectra’s software. Microsoft’s DirectX framework and its interface 15.

(27) 16. Implementation. DirectInput is used for the integration. This enables the use of other devices than the SpaceExplorer with minor modifications to the code. The JoystickInteraction is where the interaction rules are implemented, input device data is mapped onto interactions in IDS7. The following section covers the interactions rules implemented in the different review modes described in section 2.1. 4.2. Interaction rules. 4.2.1. 2D Image Window. In the Image Window the following mappings are implemented: Panning Panning is mapped to translation along x-, and y-axis on the joystick device. There are two possible solutions of this mapping: either you create a feeling of moving the image or you create the feeling of moving the camera. The first option was choosed but this should preferably be configurable in the user profile of IDS7. Zooming Zooming is mapped to translation along the z-axis. In this interaction the user gets a feeling of pulling the image towards himself by pulling the joystick knob upwards and pushing the image away by pushing the knob downwards. This should also be configurable in the user profile of IDS7. Scrolling Scrolling between images in an image stack is mapped onto rotation about the z-axis at the same time as the alternate key is pressed. The more you rotate the knob, the faster the scrolling gets. Rotation about the z-axis is only used for this interaction and one can ask why it has to be used in combination with the alternate key. The reason is that with the SpaceExplorer it is difficult to translate or rotate the x- or y-axis without also causing a small rotation about the z-axis. Adjustment of Window/Level This interaction is mapped to translation along x- and y-axis at the same time as pressing the shift key on the keyboard.. 4.2.2. 3D module. Panning and Zooming is mapped in the same way as for the Image Window. Rotation Rotation around all three axes is mapped onto rotation of the 3D volume. The 3D object is rotated around a fixed point inside the volume..

(28) 4.2 Interaction rules. 17. Clip plane translation and rotation This interaction rule was implemented to demonstrate a possible extension to the MPR module. In MPR, a threedimensional interaction is performed when changing the sectioning plane of an image volume. If a three-dimensional input device is to be used, the output display should preferably visualize the interaction in 3D. The concept of a 3D plane that is rotated and translated with the image volume fixed, as done with the Clip Plane, can perhaps be used in MPR to achieve this visualization, as a complement to the existing two-dimensional output. The Clip Plane translation is the same as panning and zooming and the rotation is the same as rotation of the 3D volume. Clip Plane thickness As mentioned in section 2.1.3 the Clip Plane can be given a thickness and the resulting volume is the information between two parallel Clip Planes. Pressing button 1 or 2 together with the shift key will respectively, decrease and increase the thickness by 0.5 mm. Predefined projections The SpaceExplorer device has four buttons for predefined views, i.e. pressing them will set the view of the 3D volume to one of the following: left, right, top, bottom.. 4.2.3. MPR module. Panning The same mapping as for the Image Window. Panning in the MPR view is panning in the fourth tile, the tile showing the resulting image stack. Zooming Zooming is also in the fourth tile and the mapping is the same as for Image Window. Sectioning plane rotation Rotation about x-, y- and z-axes are mapped to rotation of the sectioning plane. Slab mode and slab thickness Pressing button 1 or 2 on the SpaceExplorer will toggle respectively, forwards and backwards between the different slab modes introduced in section 2.1.4. Pressing button 1 or 2 together with the shift key will respectively, decrease and increase the slab thickness by 0.5 mm..

(29)

(30) Chapter 5. Result This chapter will cover the results of the interviews and the implementation.. 5.1. Interaction in the Image Window. The question in this thesis was whether the traditional input devices still was valid for use. When it comes to the two dimensional image review the interviews shows that all interviewees are quite satisfied with the existing interaction techniques with one exception: the stack scrolling. The total number of images reviewed in an examination can be several hundreds or even thousands according to the interviewees. It takes a long time to review the images and sometimes the PACS workstation is slow when it comes to loading the images in the stack. Besides that, the radiologist often uses the mouse wheel to scroll between images in the stack and one interviewee reports having problem with “cold fingers” due to this scrolling. The implementation for the 3D mouse done for the Image Window includes panning, zooming and scrolling. Adjusting the window/level settings is also included. The crucial stack scrolling was difficult to implement due to the construction of the 3D mouse. With one knob controlling all six axes it is difficult to affect a single axis without also affecting some of the other axes. In the Image Window one does not want to accidently scroll while panning or zooming. The solution in this implementation is to combine rotation around the z-axis with pressing the alternate key. This is not a preferred solution as the use of accelerator keys was what we wanted to avoid by using a more advanced input device. Panning and zooming is performed without the use of accelerator keys but adjusting the window/level settings is performed by translation along x- and yaxis together with pressing the shift key. The conclusion is that the 3D mouse is not suited for the Image Window, it would probably be better to use e.g. a JogShuttle, an input device that has shown good performance for this kind of application [4]. 19.

(31) 20. 5.2. Result. Interactions in the MPR module. In the thesis the main focus was on the MPR module. The interviews unfortunately shows that the MPR module in IDS5 was rarely used by the interviewees. There are several reasons for that. One is the fact that the performance of the MPR module in IDS5 is inadequate according to the interviewees. For example the time it takes to start the module is too long. Another reason is that the original image stack is never sent to PACS, only the reformatted one. Reformatting the already down-sampled data does not always give the required quality. Instead of using MPR in IDS5 the interviewees often use MPR on the imaging device, where the raw data is kept for a short period. After reformatting the raw data a new image stack is sent to the PACS for review. Due to the rare usage, no valuable information about the existing interactions in MPR could be collected. For that reason the implementation in MPR is based mainly on the author’s own ideas. The interaction of most interest in MPR is the one that adjusts the sectioning plane. This interaction takes place in three dimensions which opens up for a 3D input device. The problem is that the output is only two dimensional. This fact makes it difficult to achieve an intuitive interaction using a 3D mouse, it is just confusing and hard to learn. A solution to this could be to add a guide showing the body part and the sectioning plane in three dimensions. The 3D mouse does not suit MPR either. Unless the interactions in MPR is enhanced, the situation for the MPR module will be the same as for the Image Window: the driver for a complementary input device is the stack scrolling.. 5.3. Interactions in the 3D module. The 3D module was also rarely used by the interviewees. The reason for this is probably that they do not work so much with the kind of exams that suits well for 3D. The performance of the 3D module in IDS5 is also mentioned as a reason for not using it. The implementation in the 3D module was, however, the most successful, a 3D mouse is of course best suited for 3D interactions. The 3D mouse provides an intuitive interaction with a feeling of holding and being able to freely rotate the body part. If the radiologist works a lot with 3D review, there is a chance that the 3D mouse can take a place on his desk. The interviewees were allowed to test the SpaceExplorer, unfortunately not together with the implementation in IDS7, but in a 3D sketch program to get an idea about how it works..

(32) Chapter 6. Discussion, conclusion and future work 6.1. Discussion. The interviews show that the review process can be made more efficient using an alternative or complementary input device. However, the interviewees are quite satisfied with the existing techniques, at least in the two-dimensional review, and a new device has to add great value to take place on the radiologists desk. Additionally, the implementation shows that it will be difficult to find an input device that suits all purposes in the PACS, i.e. interactions in 2D, MPR and 3D. Sherbondy et al [4] concluded a statistical evaluation of several alternative input devices. Radiologists were told to look for artificial targets in large CT stacks and the time required to find the objects was recorded. The focus was on two-dimensional review and four different devices were included in the study. The conclusion of the study was simply that other devices may improve the efficiency when reviewing large CT stacks. The study shows that it is difficult to find a suitable device even if one is focusing solely on two-dimensional review. That is because the device needs to provide both speed and accuracy. Speed for locating the targets and accuracy for examining the targets. The aim of the interviews in this thesis was to gain knowledge about the existing interaction techniques and to understand the problem in a better way. Unfortunately the interviewees could only provide feedback for the interactions in the two-dimensional Image Window, simply because they did not use the other modules on a regular basis. A question one has to ask is if this is the case on all hospitals. The answer is probably no. The innovation level differs between different radiologists and different hospitals. There is a difference between a radiology department focusing on medical research and a down town department performing standard examinations at a high rate and getting paid per examination. Discussions with developers at Sectra also reveals that there are several customers that regularly review exami21.

(33) 22. Discussion, conclusion and future work. nations in 3D, using Sectra’s workstation or the imaging device workstation. The interview results would thus have been more reliable if it had included more radiologists and the selection of hospitals had been based on other parameters than the distance from Linköping. Such a study would though have been difficult to fit into the time frame and budget of this master thesis. I also think it would have been more interesting to make the implementation, or at least parts of it, first and then let the interviewees test the implementation in IDS7. This would probably have moved the focus from the performance issues in IDS5 to the interactions. It would also have given the radiologists a hint about what can be achieved with new types of input devices..

(34) 6.2 Conclusion. 6.2. 23. Conclusion. The question in this thesis was whether there is a need to add a more advanced input device than the keyboard and the mouse to the PACS workstation. The reason for asking was the rising amount of information generated by the medical imaging devices and the radiologists challenge of making benefit from the information. The interviews shows that there are problems that could be solved using another input device, such as the stack scrolling. On the other hand, the interviewees are satisfied with the majority of the existing interactions. This together with the fact that the radiologists desk is already occupied by keyboard, mouse and often a voice recorder, makes it tough for a new device to take place. Another problem is that there are different needs for the different modules in the PACS workstation. In the two dimensional Image Window for example, it is important to solve the issue of scrolling in large image stacks. This problem is not present in the 3D module where it is more important to have a device that can support the 3D interactions. The 3D mouse from 3DConnexion used in the implementation is only valuable for use in the 3D module and not suited for solving the stack scrolling problem. There are however numerous types of input devices that has to be investigated and perhaps there are devices out there that can be used both for 2D and 3D. On the other hand if the radiologist does not review examinations in 3D, the workstation can be complemented with a 2D suited device and vice versa. The answer to the thesis question is that the medical review work can gain from a complementary input device but the device has to add great value to take place on an already rather efficient workplace.. 6.3. Future work. The following issues has been identified as improvements and future work: • Evaluation of input devices. There are numerous kinds of input devices on the market that has to be evaluated. For example the already mentioned JogShuttle. • A configuration interface has to be added to the implementation. The use of different input devices requires e.g. the ability to scale the input on the different axes and also map axes on the device to interactions i IDS7. It would be a good idea to enable a per user configuration. • Handle multiple input devices. Due to the fact that it is hard to find a device that can serve both 2D and 3D it would be a good idea to support multiple devices. • 3D guide for the MPR module. A 3D input device can be used in MPR if the results was displayed in 3D. A 3D guide showing the sectioning plane in the image volume could provide an intuitive 3D interaction in MPR..

(35)

(36) Bibliography [1] Sectra Imtec AB. Product brouchure for Sectra PACS Clinical Edition. http://www.sectra.se/medical. 2008-03-15. [2] DA Bowman, E Kruĳff, JJ LaViola Jr, and I Poupyrev. An Introduction to 3-D User Interface Design. Presence: Teleoperators and Virtual Environments, 2001. [3] IDS5 User’s Guide, December 2006. Version 11.1. [4] AJ Sherbondy, D Holmlund, GD Rubin, PK Schraedley, T Winograd, and S Napel. Alternative Input Devices for Efficient Navigation of Large CT Angiography Data Sets. Radiology, 234(2):391–398, 2005. [5] www.radiologyinfo.org. The radiology information resource for patients. http://www.radiologyinfo.org/photocat/popup/philip25.jpg. 2008-03-15.. 25.

(37)

(38) Appendix A. Interview questionnaire This is the questionnaire used at the interviews.. Current analysis How much time do you spend reviewing medical images per day?. What type of examinations do you work most with?. Do you use MPR or 3D? If, in what way?. How large image series do you work with? Is there a problem of scrolling in large series?. 27.

(39) 28. Interview questionnaire Do you encounter any bottlenecks in your review work?. How do you rate the existing navigation interactions for panning, zooming, scrolling etc. Ergonomics Have you had any problems with muscles, wrists or else, as a result of your work at the PACS workstation?. Is it possible to add another input device at your workplace? Perhaps one needs to have one hand free?. Visions What do you think of 3Ddiagnosis vs. traditional 2Ddiagnosis?. How is the review work performed in ten years from now?.

(40) Appendix B. Interview answers B.1. Radiologist 1, Hospital A. Time spent per day Works approximately 6 hours/day with image review at PACS-workstation Type of examinations. Works with CT and MR examinations.. Do you use MPR/3D? Uses MPR to reformat stacks. Uses 3D occationally to visualize an injure to for example the orthopedist. Size of series? Works with series containing from 35 up to 300 images. Experience problems with scrolling in large stacks, strenuous if performed during long periods. Bottlenecks • The large amount of data -> Too many images to review • The time to load the data is sometimes slow Existing interactions Does not experience any problem with the existing interaction besides scrolling in stacks. Problems with muscles and wrists? No actual health problems yet but finds the work sometimes strenous at long working days. Add another device? Acceptable with additional device as long as it adds enough value? He has already a mouse and a dictaphone, can be a problem! 29.

(41) 30. Interview answers. 3D vs. 2D • Reviewing slices will continue to be the dominating method for diagnosis • 3D will be used mainly for visualizing already found injuries Ten years from now Believes that there will be more tools for computer aided diagnosis in the future.. B.2. Radiologist 2, Hospital A. Time spent per day Works at PACS-workstation approximately 8 hours/day Type of examinations extension. Mainly CT examinations but also "slätröntgen" at some. Do you use MPR/3D? Uses MPR on the modality-workstation, not so much in Sectra-PACS. 3D is also rarely used in Sectra-Workstation, used occationally for bones and vessels. Size of series? Typical size of stacks in PACS: up to 200 images, 5mm thick slices, originally 1500 images and 1mm thick slices. Bottlenecks Large number of images in examination that is also to be compared to old examinations -> Too many images to review! Existing interactions Experience no problem with existing interaction, works fine with mouse, keyboard. Problems with muscles and wrists? No problems with muscles or other health issues as a result of working at the computer. Add another device? Could be a good idea, for example a jogshuttle or likewise. Today is often both hand occupied by mouse and dictaphone but that could be changed. Could be positive to alter between different devices to avoid static work! 3D vs. 2D 2D will continue to be the method of choice, 3D for some examination types like bones and vessels. Ten years from now Cant foresee any dramatic changes to the way the work with the review..

(42) B.3 Radiologist 3, Hospital B. B.3. 31. Radiologist 3, Hospital B. Time spent per day Works at PACS-workstation 80% of his time Type of examinations. All types of examinations except ultrasound. Do you use MPR/3D? Does not use MPR or 3D in PACS, reconstructions is made by the nurses on the modality workstations. One problem with MPR is that they don’t have access to the full examination in PACS, only reduced 5mm slices which leads to poor quality on the reconstructed images! Size of series? Typical size of stacks: 100-150 images/stack 3-4 stacks/examination. 5mm thick slices. Bottlenecks • Sometimes the loading of images is too slow • MPR is too slow to use Existing interactions mouse and keyboard.. No problems with existing interaction, works fine with. Problems with muscles and wrists? No health problems as result of the computer-work! Add another device?. Sees no problem in adding another device.. 3D vs. 2D • 3D suitable for certain bodyparts such as bone, vessels • 2D and axial slices will continue to be the dominating method for diagnosis. Ten years from now • Foresees a decrease in CT-exams due to regulatory issues regarding radiation dose. • Instead MR will increase (if the MR-equipment gets fast enough) • Information from different techniques merged into on exam (as in PET/CT) • More functional studies, looking at short sequences showing organs in action. B.4. Radiologist 4, Hospital B. Time spent per day Works approximately 95% of his time at PACS-workstation.

(43) 32. Interview answers. Type of examinations sound). Mostly bones and wrists , MR, CT, CR no UL (Ultra-. Do you use MPR/3D? Does not use MPR or 3D i PACS, works a lot with 3D on GE (General Electrics) modality workstation Size of series?. Typical size of stacks: 250-500 images. Bottlenecks Couldn’t come up with any PACS-related bottlenecks. One of those happy customers! Existing interactions No problems with existing interaction techniques, works great thanks to the possibility to program your own keyboard shortcuts. Problems with muscles and wrists? No problems with muscles or wrists! Add another device?. Sees no problem in adding another input-device.. 3D vs. 2D 3D suitable for certain exams, for example bone injuries. Already used frequently today. 3D sequences produced on modality workstation and sent to PACS as an image sequence. Would in some cases (demonstration) be better if this could be made in PACS. Ten years from now • No dramatic changes in the next 10 year period • Computer aided diagnostics • Continue to refine existing imaging techniques. B.5. Radiologist 5, Hospital A. Time spent per day Works 6-8 hours per day at PACS workstation Type of examinations. Types of examination CR, MR, CT. Do you use MPR/3D? Uses MPR/3D in Sectra Workstation and is satisfied with it! He would though like the ability to put colour on MR-exams for visability. Size of series? Typical size of stacks: 20-1000 images and 2-15 stacks/review (old and new examinations). Bottlenecks The physical area available for hanging images are smaller than it was on the light-cabinet..

(44) B.6 Radiologist 6, Hospital A. 33. Existing interactions Satisfied with the existing navigation, specially with the ability to configure own shortcuts. Problems with muscles and wrists? No problems with muscles or wrists Add another device? adds enough value! 3D vs. 2D. No problem to add another input device as long as it. 3D mainly for visualization 2D for diagnosis!. Ten years from now Cant foresee any dramatic changes in the area. Continue to refine existing techniques. More distributed work, Teleradiology world-wide. B.6. Radiologist 6, Hospital A. Time spent per day Works 60-70% of his time at PACS workstation Type of examinations. Work mostly with CT and MR occationally with CR. Do you use MPR/3D? Uses both MPR and 3D in Sectra PACS but he is not satisfied with it. To slow and lacks funtionality. Size of series? Typical size of stacks: up to 300 images and then 4 or 5 old exams of the same size to compare with. 0.625mmm slices is reduced to 5mm slices before sent to PACS in most cases. Bottlenecks • Scrolling in stacks, imageloading takes too long time • MPR too slow Existing interactions. Satisfied with interaction in 2D but not in 3D. Problems with muscles and wrists? Has had problems with “cold fingers” due to strenous mouse scrolling Add another device? Could be a problem to add another device as he already has his hands occupied by mouse and speech mike! 3D vs. 2D. 3D only as a complement to 2D diagnosis.. Ten years from now No dramatic changes in the work. The imageing devices keep getting more advanced and techniques like Dual Energy CT and MIP will get more attention..

(45)

No results found