28-03-2014, 12:52 PM
Learn to Personalized Image Search from the Photo Sharing Websites
Abstract
Increasingly developed social sharing websites, like
Flickr and Youtube, allow users to create, share, annotate and
comment medias. The large-scale user-generated meta-data not
only facilitate users in sharing and organizing multimedia con-
tent, but provide useful information to improve media retrieval
and management. Personalized search serves as one of such
examples where the web search experience is improved by
generating the returned list according to the modified user search
intents. In this paper, we exploit the social annotations and
propose a novel framework simultaneously considering the user
and query relevance to learn to personalized image search. The
basic premise is to embed the user preference and query-related
search intent into user-specific topic spaces. Since the users’
original annotation is too sparse for topic modeling, we need
to enrich users’ annotation pool before user-specific topic spaces
construction. The proposed framework contains two components:
1) A Ranking based Multi-correlation Tensor Factorization model
is proposed to perform annotation prediction, which is considered
as users’ potential annotations for the images; 2) We introduce
User-specific Topic Modeling to map the query relevance and
user preference into the same user-specific topic space. For
performance evaluation, two resources involved with users’ social
activities are employed. Experiments on a large-scale Flickr
dataset demonstrate the effectiveness of the proposed method.
INTRODUCTION
Keyword-based search has been the most popular search
paradigm in today’s search market. Despite simplicity and ef-
ficiency, the performance of keyword-based search is far from
satisfying. Investigation has indicated its poor user experience
- on Google search, for 52% of 20,000 queries, searchers did
not find any relevant results [1]. This is due to two reasons. 1)
Queries are in general short and nonspecific, e.g., the query of
“IR” has the interpretation of both information retrieval and
infra-red. 2) Users may have different intentions for the same
query, e.g., searching for “jaguar” by a car fan has a complete-
ly different meaning from searching by an animal specialist.
One solution to address these problems is personalized search,
where user-specific information is considered to distinguish the
exact intentions of the user queries and re-rank the list results.
Given the large and growing importance of search engines,
personalized search has the potential to significantly improve
searching experience.
RELATED WORK
In recent years, extensive efforts have been focusing on
personalized search. Regarding the resources they leveraged,
explicit user profile [17], relevance feedback [18], user history
data (browsing log [19], click-through data [20], [21] and
social annotations [11], [8], [4] etc.), context information
[23] (time, location, etc.) and social network [1], [3], [16]
are exploited. For the implementation there are two primary
strategies [24], query refinement and result processing. In the
following we review the related work by the strategy they
used.
Multi-correlation Smoothness Constraints
Photo sharing websites differentiate from other social tag-
ging systems by its characteristic of self-tagging: most images
are only tagged by their owners. Fig.4(a) shows the #tagger s-
tatistics for Flickr and the webpage tagging system Del.icio.us.
We can see that in Flickr, 90% images have no more than
4 taggers and the average number of tagger for each image
is about 1.9. However, the average tagger for each webpage
in Del.icio.us is 6.1. The severe sparsity problem calls for
external resources to enable information propagation.
Parameter Setting
NUS-WIDE15 is randomly split into two parts, 90% for
training and testing (denoted as ), and 10% for validation
(denoted as ). The result of annotation prediction directly
affect the performance of personalized search. In our work,
we select parameters according to the performance of anno-
tation prediction.12 There are three sets of parameters for the
proposed RMTF+LDA model. The first three parameters are
the rank of factor matrices, rU , rI , rT . According to [30],
[31], we simply choose the ranks proportional to the original
dimensions | |, | |, | | and set rU = 50, rI = 250, rT = 5.
This guarantee that the same ratio of energies for different
modes are preserved.
CONCLUSION AND FUTURE WORK
How to effectively utilize the rich user metadata in the social
sharing websites for personalized search is challenging as well
as significant. In this paper we propose a novel framework
to exploit the users’ social activities for personalized image
search, such as annotations and the participation of interest
groups. The query relevance and user preference are simulta-
neously integrated into the final rank list. Experiments on a
large-scale Flickr dataset show that the proposed framework
greatly outperforms the baseline.