Cbir And Remote Clinical Diagnosis

Content Based Image Retrieval and its Application in Remote Clincal Diagnosis and Healthcare Suyog Dutt Jain Dept. Of Computer Science & Engineering MIT, Manipal

Before Starting..  Center for Soft Computing Research, Indian Statistical Unit, Kolkata  Department of Computer Science & Technology, MIT Manipal  Dr. Niranjan U C , Adjunct Faculty, Dept. Of ECE, MIT Manipal  All my Teachers  Friends

"To Dream About Something Is the First Step Towards Achieving It"

Background  Content Based Image Retrieval [CBIR] : A topic of research from decades  Its time to develop some domain specific applications using CBIR  Application of CBIR in Healthcare Domain  Global Availability and Ease of access : Internet Based System

Introduction  What is Content Based Image Retrieval [CBIR] ?  What is the need for CBIR?  How it Works? Feature Extraction Feature Matching

Objectives of the Work  Identification of potential users of the application.  Study of the existing techniques for Content Based Image Retrieval.  Maintenance of a database of actual medical cases and images.  Deciding upon Image Features to be extracted and Similarity Metric.  Make the system web deployable.  Authorized access to the system.  Web Based efficient Graphical User interface

Overview of the sotwares used  MATLAB R2006B  MATLAB Builder for Java  Java Servlet API  MySQL Database Mangagement System

System Architecture DATABASE SITE

COMMUNICATION OVER IP

DATABASE SERVER SERVER SIDE REMOTE DISTRIBUTED DIAGNOSIS SERVER MACHINE

COMMUNICATION OVER IP

SERVER SIDE CLIENT SIDE

System Architecture

System Architecture CBIR MODULE

JAVA SERVLET USER AUTHETICATION USER MANAGEMENT QUERY HANDLING

MATLAB FEATURE EXTRACTION

FEATURE COMPARISON RESPONSE HANDLING DATABASE HANDLING

Proposed Method: Intensity Based Retrieval The Distribution Function cdf (i) upto the gray level ‘i’ is given by

Where, h (j) is the normalized histogram at gray level j nj is the number of pixels with gray level j M.N is the size of the image The CDF contains the same information as the histogram but in redundant way

Proposed Method: Intensity Based Retrieval  CDFs at different grayscale intervals is modeled with lines.  Each of these models has two parameters viz. slope 'a' and intercept 'b‚

(a)  Estimation of the slope ‘a’ and the intercept ‘b’ by the least squares method

(b)

Proposed Method: Intensity Based Retrieval Solution of equation (b) is:

Where,

Proposed Method: Intensity Based Retrieval  Similarity Metric

Eintensity = adiff + bdiff adiff = (|a1 (q)-a1 (d)| + (|a2 (q)-a2 (d)| +……… (|ap (q)-ap (d)| bdiff = (|b1 (q)-b1 (d)| + (|b2 (q)-b2 (d)| +……… (|bp (q)-bp (d)| Where, a1(q), a2(q)...ap(q) and a1(d), a2(d)...ap(d) represent slopes b1(q), b2(q)...bp(q) and b1(d), b2(d)...bp(d) represent intercepts

Intensity Based Retrieval: Results

CDF fitted with p=4

Intensity Based Retrieval: Results

CDF fitted with p=8

CDF fitted with p=16

Intensity Based Retrieval: Results

CDF fitted with p= 4

Intensity Based Retrieval: Results

CDF fitted with p=16

CDF fitted with p=16

Proposed Method: Shape Based Retrieval  Edge Detection  Edges: Areas with strong intensity contrast i.e. a jump in intensity from one pixel to the

next

 Reduces the amount of data and filters out useless information, while preserving the

structural properties of the image.

 Gradient method  Detects the edges by looking for the maximum and minimum in the first derivative of the

image.



Laplacian method  Searches for zero crossings in the second derivative of the image to find edges.

Proposed Method: Shape Based Retrieval

Signal showing edge in an image

Signal showing the First Derivative of the Original Signal

Signal showing the Second Derivative of the Original Signal

Proposed Method: Shape Based Retrieval  Sobel Edge Detector  Performs a 2-D spatial gradient measurement on an image.  Uses a pair of 3x3 convolution masks, one estimating the gradient in the x-direction

(columns) and the other estimating the gradient in the y-direction (rows).

 Sobel Masks

 Applied separately to obtain gradient components in each orientation

Proposed Method: Shape Based Retrieval  Gradient Magnitude

 Approximate Faster Magnitude

|G|=|Gx| + |Gy|  Gradient Image  Edge Histogram  Eshape

Shape Based Retrieval: Results

Original Image

Gradient Image

Shape Based Retrieval: Results

Edge Histogram

Shape Based Retrieval: Results

Shape Based Retrieval: Results

Shape Based Retrieval: Results

Shape Based Retrieval: Results

Proposed Method: Texture Based Retrieval  Grey Level Co-occurrence Matrix (GLCM)  Tabulation of how often different combinations of pixel brightness values (grey levels)

occur

in an image

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

0

0

1

1

0

0

1

1

0

2

2

2

2

2

3

3

Proposed Method: Texture Based Retrieval  Grey Level Co-occurrence Matrix (GLCM)

0

0

1

1

0

0

1

1

0

2

2

2

2

3

Image Intensity Distribution

0

1

2

3

0

2

2

1

0

1

0

2

0

0

2

0

0

3

1

3

0

0

0

1

2

3

GLCM

Proposed Method: Texture Based Retrieval  Normalization of GLCM

Where, Vi,j is the (i , j) element of the GLCM Pi,j is the (i , j) element of the Normalized GLCM

Proposed Method: Texture Based Retrieval  Calculation of Texture Features from GLCM Contrast [ T1 ] Dissimilarity [ T2 ] Homogeneity [ T3 ] Angular Second Moment [ T4 ] Entropy [ T5 ]

 Feature Vector F = [ T1,T2,T3,T4,T5]  Similarity Metric Etexture = | Fd – Fq | Where. Fd = Feature Vector of Database Image Fq = Feature Vector of Query Image

Proposed Method: Similarity Metric  Overall Similarity Metric

Efinal = (Etexture + Eintenstiy + Eshape)/3  Efinal is used to sort similar images  Similar past cases are retrieved

Experimental Results & Discussion

Query Image1

Experimental Results & Discussion

Query Image2

Experimental Results & Discussion

User Authenticatio n Interface

Experimental Results & Discussion

Query Submissi on Interface

Experimental Results & Discussion

Matching Cases Interface

Fig. 3 User Interface for viewing similar cases retrieved by the diagnosis server

Response Time Analysis

Average Response Time = 29.28 seconds.

Conclusion  CBIR & its application in healthcare domain  Web based architecture for CBIR.  Global availibility and ease of access.  Easily expandable to include modern CBIR algorithms  Can act as a prototype for other domain specific applications

Publication from the work Suyog Dutt Jain, Dr Niranjan U C, “Distributed Framework for Remote Clinical Diagnosis with Visual Query Support”, 5th IEEE International Conference on Information Technology and Application in Biomedicine, China, May 2008.

Bibliograph & References [1] Andriole KP. “Addressing the Coming Radiology Crisis” The Society for Computer Applications in Radiology Transforming the Radiological Interpretation Process (TRIP) Initiative. White Paper (http://www.siimweb.org). November 2005. [2] Antani S, Long LR, Thoma G. “Content-based image retrieval for large biomedical image archives”. Proc. 11 World Cong. on Med Info (MEDINFO) 2004, pp. 829-33. [3] http://www.mypacs.net/ [4] Flickner, M et. al. “Query by image and video content: the QBIC system” IEEE Computer 28(9), 23-92, 2001 [5] Kato, T. “Database architecture for content-based image retrieval”, Image Storage and Retrieval Systems. Proc SPIE 1662, 112–123, 1992. [6] Swain, M. J., and Ballard, D. H., “Color indexing for Content Based Image Retrieval” International Journal of Computer Vision 7(1):11–32, 1991.

Bibliograph & References [7] Gonzalez, R. C., and Woods, R. E., Digital image processing, 2004 2nd Edition, pp 94–103. [8] Stricker, M., and Dimai, A., “Color indexing with weak spatial constraints”. In: Storage and Retrieval for Image and Video Databases IV. Proc SPIE 2670, 29–40, 1996. [9] Manjunath K N, Niranjan U C, “Linear Models of Cumulative Distribution Function for Content Based Medical Image Retrieval”, Appears in Proc. Of 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, Shanghai, China, 2005 pp.6472-6475 [10]Robert M. Haralick, K. Shanmugam, “Textual Features for Image Classification”, IEEE Transactions on Systems, Man, and Cybernetics, VOL. SMC-3, No.6, November 1973 pp.610-621