05. Digital Image Processing

PAPER PRESENTATION ON DIGITAL IMAGE PROCESSING (An automated radiographic absorptiometry system for quantitative rheumatoid arthritis assessment)

PRESENTED BY AMEER BASHA 04391A0561 (3/4CSE) [email protected]

SANJEEVIAH 05395A0508(3/4CSE) [email protected]

VIGNAN’S ENGINEERING COLLEGE VADLAMUDI

ABSTRACT Medical imaging is a field which researches and develops tools and technology to acquire, manipulate and archive digital images which are used by the dimensional function , f, that takes an input two spatial coordinates x and y and returns a value f(x, y). The value f(x, y) is a gray level of the image at that point. The gray level is also called the intensity. Digital images are a discretized partition of the spatial images into small cells which are referred to as pixels – picture elements. Digital image processing is a field for processing digital images using a digital computer. Processing of digital images include operations involving digital images such as acquisition, storage, retrieval, translation, compression, etc. Quantifying disease progression of patients with early stage Rheumatoid Arthritis (RA) presents special challenges. Establishing a robust and reliable method that combines the ACR criteria with bone and soft-tissue measurement techniques, would make possible the diagnosis of early RA and/or the monitoring of the progress of the disease. In this paper and automated, reliable and robust system that combines the ACR criteria with radiographic absorptiometry based bone and soft tissue density measurement techniques is presented. The system is comprised of an image digitization component and automated image analysis component. Radiographs of the hands and the calibration wedges are acquired and digitized following a standardized procedure. The image analysis system segments the relevant joints into soft-tissue and bone regions and computes density values of each of these regions relative to the density of the reference wedges. Each of the joints is also scored by trained radiologists using the well established ACR criteria. The results of this work indicate that use of standardized imaging procedures and robust image analysis techniques can significantly improve the reliability of quantitative measurements for rheumatoid arthritis assessment. Furthermore, the methodology has the potential to be clinically used in assessing disease condition of early stage RA subjects. Keywords: Automated hand image analysis, Hand image segmentation, Radiographic absorptiometry, Rheumatoid arthritis

contents •

Introduction 1. Hand bone density 2. Hand radiographic absorptiometry

•

IMAGE ACQUISITION

• Enhancement Image and Restoration •

IMAGE ANALYSIS 1. Control point placement 2. Bone ridge delineation 3. Wedge profile computation

•

EXPERIMENTAL RESULTS

•

MULTIPURPOSE PACKAGE FOR THE DOCTORS (Our Contribution in Digital Image Processing) 1. Goal of project 2. Introduction to important terms 3. Project definition 4. Code for enhancing the image size

•

DISCUSSION AND CONCLUSION

•

REFERENCES

1. INTRODUCTION Conventional examination of the hand radiographs is well established as a diagnostic as well as an outcome measuree in Rheumatoid Arthritis (RA). It is readily available and has been correlated with measures of disease activity and function. Xray changes are, however, historical rather than predictive, and there is significant observer variation in quantifying erosive changes. The earliest radiographic changes seen in the hand are soft-tissue swelling symmetrically around the joints involved, juxta-articular osteoporosis and erosion of the ‘bare’ areas of bone (i.e. areas lacking articular cartilage).These changes help to confirm the presence of an inflammatory process. The presence of early soft-tissue swelling is easily recognized on plain radiographs but not readily quantified. Although the presence of early osteoporosis is recognized in the affected hand, a mild osteoporosis may be extremely subtle to the eyes. The recognition of the changes in soft-tissue and bone density is subjective and is known to vary from assessor to assessor. Therefore, attention has been focused on the more objective erosion and joint narrowing assessment. Use of magnetic resonance (MR) technique has been shown to sensitively detect early local edema and inflammation prior to a positive finding on plain film radiographs. However, MR is an expensive examination and may not be used as a routine technique. Presently, radiographs of the hands and wrists are employed to assess disease progression. The parameters used to determine progression are the changes in erosions and joint-space narrowing observed on the radiographs. There are some problems with both of these parameters. First, both erosion and joint narrowing are not the earliest changes in RA and further they may be substantially irreversible. Second, these two changes may occur independently of each other. Third, there is a tremendous variability in erosive disease: some patients never develop erosions; some go into spontaneous remission of their erosive disease; and for some, the progression is relentless. Fourth, joint-space or cartilage loss may be caused by either the disease itself or by mechanical stress. Present scoring methods require that any degree of joint-space loss be recorded as a progressive change due to RA. Quantitative techniques currently available may provide a new approach in monitoring disease progression in patients with RA. Adoption of these techniques may have implications for the management of patients with RA and for possible detection of the disease at an early stage. 1.1. Hand bone densitometry Considerable advances have been made over the past two decades in developing radiological techniques for assessing bone density. However; all of these techniques have been utilized on aging-related osteoporosis, a pathological change involving general bone mineral reduction. Owing to the wide availability of DXA, recently published research describes the use of Bone Mineral Density (BMD) measurements in the hands of patients with chronic RA. Most published observations on RA have examined BMD changes, focusing on only the general bone loss around the joints. Quantification of the difference of bone loss between the juxta-articular bone and the shaft of tubular bones in the hands could be a sensitive index for quantitative analysis of RA patients. Hand BMD measurements offer an observer independent and reproducible means of quantifying the cumulative effects of local and systemic inflammation. The technique could be of use in the assessment of patients with early RA, in whom conventional measures of disease are not helpful until disease is (irreversibly) more advanced. 1.2. Hand radiographic absorptiometry

In conventional Radiographic Absorptiometry, radiographs of the hand are acquired with reference wedges placed on the films. The films are and subsequently analyzed using an optical densitometer. The resulting density values computed by the densitometer are calibrated relative to that of the reference wedge and are expressed in arbitrary units. Recent improvements in hardware and software available for digital image processing have led to the quantitative assessment of radiological abnormalities in diagnostic radiology. Such improvements have also enabled introduction of several radiographic absorptiometry techniques. One such technique uses centralized analysis of hand radiographs and averages the BMD of the second to fourth middle phalanges. Another technique developed in Japan uses the diaphysis of the second metacarpal to determine BMD. A third technique developed in Europe measures the diaphysis and proximal metaphysics of the second middle phalanx. Based on published short-term precision errors, computer-assisted Radiographic Absorptiometry appears to be suitable for the measurements of the BMD of phalanges and metacarpals, and is used in several hundred centers worldwide. In this work we present preliminary results of an ongoing research work aimed at developing an automated radiographic absorptiometry system for the assessment and monitoring of both BMD and soft tissue swelling in early stage RA. This paper focuses on the reproducibility and accuracy of the methodology being developed. The paper is organized as follows: the next section provides an overview of the image acquisition procedure. In section 3 the image analysis algorithms used in this work are presented. In section 4 we present results obtained by analyzing the data collected in a small reproducibility study involving 10 normal subjects. 2. IMAGE ACQUISITION One key factor influencing the outcome of any radiographic absorptiometry technique is the standardization of the image acquisition technique. Variability in acquisition parameters can significantly affect the measured values. In order to carry out this work, a standard image acquisition protocol was defined. This protocol has been successfully used in earlier large scale multinational phase 3 clinical trials for Rheuatoid Arthrtis related drugs. Radiographs of the left and right hands are taken one at a time. Templates were developed to guide the positioning of the hand with respect to the center of the x-ray beam. The X-ray beam was centered between the 2nd and 3rd metacarpo-phalangeal joints and angled at 90° to the film surface. This results in a tangential image of the joints. Improper beam centering generally results in overlapping joint margins. The X-ray exposure parameters were maintained constant for all subjects. All normal subjects were imaged at the same clinic at UCSF. In addition to providing a template for hand positioning, two sets of calibration wedges were also provided to the clinic. Each set of wedges consisted of one Acrylic wedge, for soft tissue and one Aluminum wedge for bone tissue. These wedges were custom designed for the purposes of this research work.

Figure1.Template used to position the left hand according to the standardized protocol.

3.Enhancement Image and Restoration The image at the left of Figure 1 has been corrupted by noise during the digitization process. The 'clean' image at the right of Figure 1 was obtained by applying a median filter to the image.

Figure 1. Application of the median filter An image with poor contrast, such as the one at the left of Figure 2, can be improved by adjusting the image histogram to produce the image shown at the right of Figure 2.

Figure 2. Adjusting the image histogram to improve image contrast

3.IMAGEANALYSIS One of the major difficulties in analyzing hand radiograph images is the high level of noise present in the images. Additionally the trabecular texture of the hand in the vicinity of the joints increases the noise in edge maps of this regions. Use of nonstandard acquisition protocol can add additional challenges at it can result in further degradation of image quality. This last challenge is minimized in this work, as a standard image acquisition protocol is followed. Given a particular application varying degrees of accuracy in anatomy segmentation can be considered acceptable. For instance in detecting joint-space narrowing there is a need for accurate and reliable determination of the joint-space of any finger and the bone edges in this region. However, accurate delineation of the bone edges in the vicinity of the joint is not as relevant. Depending upon the application there can be additional constraints on performance issues as well. In an application for which off-line processing of the data is acceptable, more sophisticated algorithms can be employed. This particular application requires that the overall process be fast, accurate and reproducible enough for on-line processing. Accurate estimation of the bone edges in the middle shaft and in the joint vicinity of high relevance in this work. This is primarily because the disease progression follows different patterns in the joint area as compared to the middle phalange area. Also, the manifestations of the disease symptoms in its early stage have different effects on soft-tissue and bone as well, which require reliable segmentation of these two types of tissue at different time points. The algorithm for hand segmentation can be outlined into the following main stages: • Hand outline delineation • Joint identification • Bone outline delineation • Segmentation of soft tissue and bone The first stage of the algorithm has been well studied in the literature and is not described here. The second stage can be more challenging, especially when dealing with hands of patients in advanced stage of disease progression. As this system will be applied to a patient population that is in their early disease stage it is expected that the joints will be well defined. The system provided to the radiologists allows them to adjust the location of the automatically identified joints. Results

presented in this work were obtained by having the radiologists place control points to identify the joints, rather than having them automatically computed by the system. 3.1. Control point placement A simple user interface was provided to enable placement of control points on the joints. This was primarily done to investigate the sensitivity of the system to the initial control point positioning, which in an automated system would invariably be the same for the same image. The user placed 16 control points on each image. These joints are show in Figure 4. In addition to placing the control points for the joints, the control points for the two wedges are also placed by the radiologists. For each wedge, six control points are placed with four at the corners and two in the middle. Once all the control points are placed, the remaining steps of the generalized algorithm stated above are carried out autonomously. The middle phalange or cortex control points are computed automatically and are located at the middle of straight line connecting the two joints, one above and one below the middle phalange. The diameters of the circular regions of interest placed around each joint are computed proportionally to the length of the fingers. Let ji be the control points placed at the joints and mi be the control points placed at the cortex of the phalanges. The distance di between joint ji and ji+4 is given by: (1) In the equation above jix and jiy are the x and y coordinates of the control point j respectively. Also i and i+4 are indexes for two joints on the same finger, as only the joints on the fingers are used in this work and not the thumb. Using the distance computations above the radius rij for the Region of Interest (ROI) placed at the joint ji and rim for the ROI placed around the cortex mi is given by: i

(2)

and (3)

The resulting ROIs are larger around the joints and are smaller around in the middle phalange, yet they all cover bone and soft-tissue evenly. 3.2. Bone ridge delineation Initially the entire hand image is equalized using the gray scale distribution of the region of interest around 1st MCP. This procedure enhances the bone areas and soft-tissue areas of the hand and suppresses any other details. This provided an image with the best contrast between bone and soft tissue regions. Once the image is equalized a standard Sobel gradient detection algorithm is applied. The resulting image has well defined ridges of the bone along with other edges that represent the bone trabecular structure.

For each middle phalange control point mi, the bone ridges are identified by traversing the finger axis formed by connecting ji and ji+4, in a direction perpendicular to that of the axis, as shown in Figure 2. The finger anatomy is very well defined in this region. A pixel is defined to be on the ridge, if at that each pixel the intensity ranges of the edge map and of the original image, is maximal compared to all other pixels between this one and the one of the finger axis.

Figure 2. Computation of a point on the bone edge. Let rk be the points on the ridge corresponding to a set of points m k that are in a neighborhood N of the point mi and on the finger axis. Then the average of the distance between mk and mk is used to define the bounds within which a boundary tracing algorithm can be used to track the rest of the bone ridge for this section of the finger. The system follows the shaft of the finger from the middle phalange to the joint above and scans perpendicular to this direction and identifies the ridge points. The same procedure is repeated for the part that connects the middle phalange to the joint below. At the end of these procedures an initial estimate of the finger’s bone boundary is detected. The next step is to scan the boundary points and discard any outliers. Owing to the smoothness of the bone contours, a 7th order Bezier Spline is fit to the contour data. Once this is completed, the resulting ridge contours are presented to the radiologist. At this point the radiologist is also provided with a tool to “fix” any severe errors that are observed in the bone outlines. These errors are generally found close to the joint, where the edge distribution is quite noisy. This simplistic approach provides a very fast and accurate mechanism to generate the bone boundary outlines. Perhaps a more sophisticated algorithm can be employed, at the expense of longer execution time, to further improve the boundary in the vicinity of the joint. However the results obtained on the normal population suggest that this method is sufficient to provide a high degree of reproducibility. Once the contours are accepted, the system automatically segments the bone region and the soft-tissue regions using the outlined contours and provides an estimate of the average density of each of these regions in units of equivalent density computed from the calibration wedges. 3.3. Wedge profile computation The two reference wedges imaged along with the hand provide an equivalent density value for each ROI on the hand. The Acrylic wedge is used to compute the soft-tissue density while the Aluminium wedge is used to compute bone density. In order to obtain reliable estimates of the equivalent densities an average profile is computed for each wedge. Let Lk be a vector of pixel intensities across a line inside

the wedge, parallel to the long edge of the wedge. The average profile for the wedge is given by

Where w = Acrylic or Aluminium (4) Once the average profile is computed a linear regression model is computed to represent the average wedge profile as a function of the height of the wedge. Using this regression model, for any given gray value an equivalent height on the wedge can be computed, and this can be in itself associated to the equivalent density of the wedge for that gray value.

Figure 3. Various stages of the bone delineation step. From left to right: original image, two joint and one middle phalange control points, histogram equalized image, Sobel gradient edge map, edges computed along with 7th order Bezier curve fit, segmented bone and soft tissue regions. 4. EXPERIMENTAL RESULTS A small study, involving 10 normal subjects, was conducted at UCSF to determine the reproducibility and accuracy of the technique being developed. The average age of the patients was 47.1 years. The youngest and oldest subjects were 32 and 58 years old respectively. Left and right hands of each subject were imaged twice. For one of the subjects during the first acquisition one set of wedges was employed, and during the second acquisition a different set of wedges was required. This was done to observe any variability introduced by changing the wedges. Two distinct users, one radiologist and one technician analyzed the set of 40 images. While the radiologist was a trained expert, the technician was trained only on the use of the system. In Figure 6a and 6b the final bone ridge outlines and segmentation results for a left hand are shown. As the results for left and right hands were similar, only results for the left hand are presented. The coefficient of variance (CV) was computed as follows23: (5) The system used in this work was developed using the NIH-Image/J software. A set of “plugin” modules were written for the Image/J package, and some customization of that source code was done to accommodate multiple region-ofinterests. The equalization and edge detection routines described in section 3 are all part of the standard ImageJ package. Intra-reader variability was accessed by comparing the results of each reader for reading each ROI and also per patient. Figure 4a and 4b show the CV for each ROI and each patient respectively for the expert reader. Inter-reader variability was accessed by comparing the results of one reader against that of the other. Figure 5a and 5b, show the inter-reader CVs for each ROI and for each patient respectively. In each of these charts, CV-Softav and CV_Boneav represent the CVs computed on

average gray-scale measurements on soft tissue and bone regions in the segmented ROIs. Meanwhile, the CV_Soft_d and CV_Bone_d represent the corrected values of CV_Softav and CV_Boneav using the Acrylic and Alumynium wedge profiles respectively. As an overall estimate for any given patient, the average CV for an individual reader is 3.0% and the average CV for interreader accuracy is about 4.0 %. Figure 5b. Left hand inter-reader CVs per patient. Each set of bars in both of the charts above is grouped from left to right as CV_Softav, CV_Bone_av, CV_Soft_d,

CV_bone_d. Figure 6a. A sample left hand with all the bone ridges outlined by the system and the two wedges outlined by the radiologist.

Figure 6b. Segmentation result of the hand in Figure 6a.

6. MULTIPURPOSE PACKAGE FOR THE DOCTORS (Our Contribution in Digital Image Processing) This is a project we are working at. This product will be useful for the doctors working in various specializations. We have organized our project in the following ways:-

1. Goal of project: Medical imaging is a field which researches and develops tools and technology to acquire, manipulate and archive digital images which are used by the medical profession to provide better care to the patients. The goal of our project is to build a package useful for doctors working in various specializations. This product is aimed to enhance the image size by taking the image in black and white mode. Thus aimed to identify and enhance the existence of any flaws in the bones, pimples on face and even the spots and marks on face etc. 2. Introduction to important terms: Processing of digital images include operations involving digital images such as acquisition, storage, retrieval, translation, compression, etc. • Image Acquisition: image is acquired and brought into the system (digital camera, CAT scan) usually requires preprocessing, e.g., scaling, sampling • Image Enhancement: image is made clear to the user by enhancing some of the features of the image this is a subjective operation (it looks good) • Image Restoration: image is improved- this operation is objective (e.g., noise removal) • Color Processing: image colors and color transformation e.g., from display color space (red green and blue) into hardcopy printing space (cyan, magenta and yellow). • Compression: image archive size is reduced (storage, transmission) error free, and error prune compression • Segmentation: image is partitioned into features (e.g., boundary of objects) • Representation: extract features are stored outside the image • Recognition: image objects are being identified (e.g., liver, kidneys, spine) 3. Project definition We are planning our project to be very useful for the doctors working in various specializations, in spite of their illiteracy to the computer. Our package will allow the doctors to convert the grey scale image into the binary image and identify the flaws. A very user friendly GUI support will be there. First the image will be converted into the pixel numbering and then the numbering will be used for the binary image generation. For this we will be using the following methods1. Binary image by median: the median value of the pixel values is identified and the values that are greater than the median are given high intensity and that having less than that will be given the low intensity. Thus we will obtain the binary image which will be used for enhancing techniques. 2. Binary image by Threshold: The threshold is identified by random number picking or the average of the pixel values is identified and that is used for the further processing. 3. Grey level image with 32 different levels: The range for the levels from 0 to 9 and A to V are given and the pixels are given values according to the regions into which they fall into. And an image is generated which is having a less reflection compared to the original high resolution grey levels image. 4. Code for enhancing the image size: The code we have generated will take the values of pixels from a file of pixels and uses them to generate the binary image. Here we have followed the threshold method and the threshold value is identified accordingly.

The code useful in c language is … //This is the module for converting the grey level image to a binary image by threshold method. #include<stdio.h> #include #include main() { int i,j,th,gd,gm,bin; int image[64][64]; long int sum=0; char ch; float avg; FILE *fp; gd=DETECT; initgraph(&gd,&gm,””); fp=fopen(“d:\lin.txt”,”r”); if(fp==NULL) printf(“File Not Open”) for(i=0;i<64;i++_ for(j=0;j<64;j++) { ch=getc(fp); image[i][j]=ch; sum=sum+ch; //Sum of grey levels } avg=(ing\t)(sum/(64*64.0));//Threshold generation for(j=0;j<64;j++) { if(image[i][j]>avg) bin=0; else bin=15; putpixel(i+20,j+20,bin); } getch(); closegraph(); fclose(fp); }

5. DISCUSSION AND CONCLUSION In this paper a radiographic absorptiometry based methodology was presented to accurately and reliably estimate the density of soft-tissue and bone. The results presented here focused on the reproducibility of such measurements in a normal subject in-vivo study. One aspect to note is that one of the readers in this work was a

technician. This suggests that in a routine clinical setting, there would be no need to involve a radiologist to carry out such a procedure. A trained technician would be able to provide estimates as reliable as an expert reader would. The results presented here are preliminary and focused only the reproducibility aspects of the technique. It should be noted however, that despite having manual inputs in the control point placement stage, the measurements were highly reproducible. While there is already work done to automate the placement of the joint control points, further work includes automating the wedge profile control point placement. By taking these steps it is expected that the reproducibility of the system will be further enhanced. It is also not clear if there is a need for soft-tissue density correction to be applied to the bone density estimates. This is because the density values of the bone include some fraction of soft-tissue densities as well. Most existing correction techniques require the use of a model for the fingers, and that in itself can lead to introducing more errors in the density estimate. The sensitivity of the technique in quantifying changes undergone by a subject during the early stages of the disease is also not yet well defined. This technique is being applied towards monitoring early stage rheumatoid arthritis patients in an ongoing clinical trial. Results obtained from the clinical trial data should provide a better understanding of such sensitivity parameters. REFERENCES 1. Fries J.F., Block D.A., Sharp J.T. et al. Assesment of radiologic progression in rheumatoid arthritis. A randomized, controlled trial. Arthritis Rheum 1986;29;1-9 2. Sharp J.T., Wolfe F., Mitchell D.M., Bloch D.A., The progression of erosion and joint space narrowing scores in rheumatoid arthritis during the first twenty-five years of disease. Arthritis-Rheum. 1991; 34; 660-8 3. Van Leeuwen M.A, van Rijswijk M.H, vander Heijde D.M, et al. The acute-phase response in relation to radiographic progression in early rheumatoid arthritis: a prospective study during the first three years of the disease. Br J. Rheumatol 1993; 32(suppl. 3); 9-13 4. Eberhardt K.B, Truedsson L, Pettersson H., et. al. Disease activity and joint damage progression in early rheumatoid arthritis: relation to IgG, IgA, and IgM rheumatoid factor. Ann Rheum Dis 1990; 49; 906-9 5. Van Leeuwen M.A, vander Heijde D.M, van Rijswijk M.H. et al. Interrelationship outcome measures and process variables in early rheumatoid arthritis. A comparison of radiologic damage, physcial disability, joint counts, and acute phase reactants. J. Rheumatol 1994; 21; 425-9 6. Makisara G.L., Makisara P., Prognosis of functional capacity and work capacity in rheumatoid arthritis. Clin. Rheum. 1982;1:117-25 7. Nørgaard, F., Earliest roentgenological changes in polyarthritis of the rheumatoid type: rheumatoid arthritis. Radiology, 1965;85: p. 325-29. 8. Martel, W., J.T. Hayes, and I.F. Duff, The pattern of bone erosion in the hand and wrist in rheumatoid arthritis Radiology., 1965;84: p. 204-14.