A Concept Based Image

A Concept-Based Image Retrieval with Concept Pattern Predicates J. D.

Yang

In this paper, we propose a concept-based image retrieval performed by concept pattern predicates which can capture the pattern of concepts buried in iconic images (or simply images)[1][2]. Concepts may be viewed as an aggregation of constituent concepts or primitive objects in an image arranged according to specific patterns. Primitive objects are objects labeled by indexing. For example, in Figure 1, the composite pattern of a concept, ‘office room’ may be described by its configuration involving a ‘study room’ and a ‘reception set, each of which is in turn a concept. ‘Table’, ‘sofa’,

‘bookshelf’, etc. are primitive objects labeled manually or

automatically.

computer

StudyRoom

DeskChair sofa

ReceptionSet

table sofa

OfficeRoom StudyRoom ReceptionSet bookshelf

StudyRoom

DeskChair

ReceptionSet sofa

table

sofa

Figure 1 Two possible patterns of an office room A query may be given as a concept such as ‘office room’ or

‘study room’ <south> ‘reception

set’. An image is therefore an answer if its configuration satisfies the pre-defined patterns of the query. 1. Pattern predicates Definition 1 Let D={east, west, south, north, southeast, …, northeast} and let IDB be a set of images, p be an image in IDB and Op be a set of primitive objects in p. Then p is indexed by a set of triples p.T of the following form. p.T ={ t | t = where r12 ∈ D and o1, o2 ∈ Op }. Definition 2 Let c be a concept. Then the components and configuration of c is referenced by c.comp and c.config respectively. Definition 3 Given a set of concepts Cc, a concept pattern predicate CP is defined as follows. 1.

CP : Cc × IDB → {true, false} is a concept pattern predicate.

2.

If CP1 , CP2 , …, CPn are concept pattern predicates, then

1

CP(C, P)=CP1 (C1, P)CP2 (C2 , P) CP3 ⋅ ⋅ ⋅ CPn-1 (Cn-1, P ) CPn (Cn, P) is a concept pattern predicate where r12, … r(n-1) n ∈D’= D ∪ { Above, Below, Around, Leftside, RightSide}.. 3.

Nothing else is a concept pattern predicate.

Definition 4 A special concept pattern predicate called index pattern IX : Cc× IDB → {true, false} is defined as follows. IX(c, p) is true if

c.config ⊆ p.T.

Definition 5 Let CP and CP1 are concept pattern predicates. Then a concept pattern rule is given by CP(C, P) ← CP1 (C1, P) :[action(C, C1, P)] where action is a procedure to perform semantic actions associated with CP(C,P). Definition 6 An image information Retrieval system IIR is defined as follows. IIR = < IDB, IKB, IE, Q > where IDB is a set of images, IKB is a set of concept pattern rules, IE is an inference engine processing a query and Q is a set of queries. Queries are given as concept pattern predicates. Definition 7 An answer for a query Q, || Q || = { p | IDB ∪ IKB

|| Q || is defined as follows.

|−IE

Q(*, p)}

Example 1 IKB: OfficeRoom(or,P) ← Study Room(sr,P) south ReceptionSet(rs, P) | Study Room(sr, P) east ReceptionSet(rs, P) :[ dir(sr, rs, P) = east; or.comp={sr, rs}; or.config = {<sr, east, rs>}] StudyRoom(sr, P) ← DeskChair(dc, P) Above DeskChair(dc, P) Above

computer bookshelf

:[ dir(dc, bookshelf, P) = Above.value; dc.comp={dc, bookshelf}; dc.config = {};] ReceptionSet (rs, P)← IX(rs , P) :[rs.comp={table, sofa}; rs.config={}; rs.config ⊆ P.T] DeskChair (dc, P)← IX(dc, P) :[ dc.comp= {desk, chair};dc.config= {<desk south chair>}; dc.config ⊆ P.T]

In the above concept pattern rules, the corresponding semantic actions are coded in :[]. They are performed whenever the rule is executed. Dir(sr, rs, P) is a function to determine a new direction between ‘StudyRoom’ and ‘ReceptionSet’ by using sr and rs together with P. A similar algorithm to implement the function can be found in [4]. Figure 2 shows an image p1, followed by Figure 3 which depicts a query processing to include it in an answer set.

2

or

b

sr

rs

d t

s

c dc

OR: OfficeRoom, SR: StudyRoom, DC:DeskChair, RS:ReceptionSet,

Figure 2 An answer image p1 OR(or, p1) :[dir(sr, rs, p1 ) = east ; or. comp = {sr, rs} or. config = {<sr, east, rs>}]

SR(sr, p1)

RS(rs, p1)

east

:[dir(dc, b, p1) = north; sr.comp = {dc, b}; sr.config= {};]

DC(dc, p1)

north b

t

:[rs .comp ={t,s}; rs .config = {t2}]

Around sofas

:[dc.comp= {d,c}; dc.config= {t1} ]

east
south

c>

s

Around.value=east

t2 =

t1 =

IX(rs, p1), t2 ⊆ p1.T

IX(dc, p1), t1 ⊆ p1.T

Figure 3. The processing of the query ‘office room’ for obtaining p1 as an answer References 1. C. C. Chang and S. Y. Lee, “Retrieval of Similar Pictures on Pictorial Databases, ” Pattern Recognition, 1991. 2. S. K. Chang et al., Intelligent Image Database Systems, Vol. 5, World Scientific, 1996. 3. H.J. Yang, H. Y. Kim and J. D. Yang, “Design and Implementation of COIRS(a ConceptBased Image Retrieval System,” Visual’99, The Netherlands, June, pp. 391-398, 1999. 4. J. D. Yang and H. J. Yang, “A Formal Framework for Image Indexing with Triples: Towards a Concept-Based Image Retrieval,” Inter. Journ. of Intell. Sys., Vol. 14, No. 6, 1999. 5. J. D. Yang, “An Image Retrieval Model Based on Fuzzy Triples,” Revised for Fuzzy Sets and Systems, “http://jiri.chonbuk.ac.kr/~jdyang,” 1999.

3