My research interests include multimedia computing (2005-present), biometrics (2002-2005), and software engineering (1999-2002). This page mainly describes my recent work on multimedia computing.


1. 2006-2007 Investigation on Various Ontological Similarity Measures

• I select the topic of concept-based video search this year and begin by experimenting various ontological measures (such as Roda, Hist&St-Onge, Resnik, Wu&Palmer, Leacok&Chodorow, Jiang&Conrath, LIN) for query-to-concept mapping. Both our theoretical analysis and experimental results confirm that Wu&Palmer (WUP) is the best one for this task. A breif analysis can be found here .

2. 2007-2008 Ontology-enriched Semantic Space

 • We propose the construction of an ontolgy-enriched semantic space (SS) to measure semantic similarity between user queries and concepts. In contrast to conventional ontolgical similarity methods which perform the reasoning by simply traversing the ontology graph, SS transforms the graph structure of an ontology into a linear space in which every concept has a vector representation. The semantic relationship between two concepts can be reasoned by simply computing cosine similarity between their concept vectors. The similarities in SS are globally determined because every concept vector has been formed by embodying the concept's ontological relationships to a set of selected "reference" concepts which are evenly distributed in the ontology graph. Furthermore, the way of mining inter-concept relationship by investigating concept vectors is more computationally convenient than traversing concept nodes in an ontology graph. More importantly, the construc tion of SS will result in a set of "reference" concepts which optimally cover the space. We argue that the detectors of these concepts have comparably higher generation power than those of other concepts and thus should be given higher priority in concept development.

Two papers are related to this work.

[1] Xiao-Yong Wei, Chong-Wah Ngo, "Ontology-enriched Semantic Space for Video Search", ACM Multimedia 2007 (MM'07), Augsburg, Germany, Sep. 2007

[2] Xiao-Yong Wei, Chong-Wah Ngo, Yu-Gang Jiang, "Selection of Concept Detectors Using Ontology-Enriched Semantic Space", IEEE Transaction on Multimedia, Vol. 10, no. 6, 2008


Potential extensions:

1. Use non-linear assumption to contruct the space.
2. Use web sources to enrich the space.

3. 2008-2009 Context Space

• Similar to SS, we propose the building of a vector space, namely context space (CS), to mine information of contextual or co-occurrence relationship. Constructed following the some procedure as SS, CS inherits the advantage of global consistency over conventional correlation measures which only provide local view of correlation among a few concepts. CS has two successful applications in concept-based search and context-based concept fusion, which have been presented in following two papers respectively.

[1] Xiao-Yong Wei, Chong-Wah Ngo, "Fusing Semantics, Observability, Reliability and Diversity of Concept Detectors for Video Search", ACM Multimedia 2008 (MM'08) Vancouver, Canada, Oct. 2008

[2] Xiao-Yong Wei, Yu-Gang Jiang, Chong-Wah Ngo, "Exploring Inter-concept Relationship with Context Space for Semantic Video Indexing", ACM International Conference on Image and Video Retrieval (CIVR'09), Santorini, GR, July 8-10, 2009


Potential extensions:

1. Tags in seem a good source to extend the context space to web-scale.

4. 2008-2009 Concept-based Video Search by Semantic and Context Reasoning

• With SS and CS available, we propose a novel multi-level fusion strategy for concept-based video search to combine detectors. The strategy for answering queries considers different aspects, including semantics, context, reliability, and diversity of detectors. Benefting from the advantages of global consistency and computational convenience of the spaces, the number of appropriate detectors is adaptively determined by jointly reasoning inter-concept semantic and contextual relationship in SS and CS. In the fusion step, detectors are combined hierarchically where each level of the fusion emphasizes one aspect of the detectors. This work has been presented in paper:

[1] Xiao-Yong Wei, Chong-Wah Ngo, "Fusing Semantics, Observability, Reliability and Diversity of Concept Detectors for Video Search", ACM Multimedia 2008 (MM'08) Vancouver, Canada, Oct. 2008

Potential extensions:

1. Consider causation between concepts can make the mapping and fusion more precise.
2. Consider frequency of concepts.


A demo can be downloaded here, which summarizes our work in 2, 3 and 4 (the Best Demo Award in ACM HK SRC-Day).


Some examples of the reasoning flows in this demo are listed as follows. (The pictures are dynamically created after the queries are issued.)

Example 1: Find a vehicle on road.


Example 2: Find boat in river

Example 3: Find an airplane in sky






Free Website Counters
Free Website Counters