Open Access Open Access  Restricted Access Subscription or Fee Access

Probabilistic Model for Summarizing Text and Automatically Annotating Images with Keywords

K. Jose Triny, R. Lakshmi

Abstract


Annotation based image retrieval is more feasible to fulfill user requirements in a straight forward manner. Modern multi-media documents are not merely collections of words but can be a vast collection of related text, images and audio references. Images that do not coincide with textual data cannot be retrieved. Analyzing the pictures in large collections is a crucial problem. Search engines on the web retrieve images without viewing their content, simply by matching user queries against thematically collocated textual information which in turn limits the applicability. Methods are proposed to automatically generate captions for a picture from a weakly labeled data. So as to annotate images and generate captions a probabilistic suggestion with abstractive and extractive caption generation model prevails. Indeed, the abstractive model compares favorably to handwritten captions and is often superior to extractive methods. However the system is designed which is been used to realize the features of the images locally and is less grammatical. A phrase-based probabilistic model is framed to generate captions for images. To resolve such criteria images are experimented with global features in thematically co-located documents related to document structure such as titles, section of articles and also to exploit syntactic information more directly.

Keywords


Abstractive Topic Model, Extractive Topic Model, Image Annotation, Text Summarization

Full Text:

PDF

References


P. Duygulu, K. Barnard, J. de Freitas, and D. Forsyth, “Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary,” Proceedings Seventh European Conf. Computer Vision, pp. 97-112, 2002

S. Feng, V. Lavrenko, and R. Manmatha, “Multiple Bernoulli Relevance Models for Image and Video Annotation,” Proceedings IEEE Conf. Computer Vision and Pattern Recognition, pp. 1002-1009, 2004

Y.Feng,M.Lapata, “Automatic caption generation for news images” IEEE Trans. on Pattern analysis and machine Intelligence, vol. 35, no. 4, pp. 797-812, 2013.

L. Ferres, A. Parush, S. Roberts, and G. Lindgaard, “Helping People with Visual Impairments Gain Access to Graphical Information through Natural Language: The igraph System,” Proc.,11th Intl’ Conf. Computers Helping People with Special Needs,Springer pp. 1122-1130, 2006.

V. Lavrenko, R. Manmatha, and J. Jeon, “A Model for Learning the Semantics of Pictures,” Proceedings 16th Conference Advances in Neural Information Processing Systems, 2003

F. Monay and D. Gatica-Perez, “Modeling Semantic Aspects for Cross-Media Image Indexing,” IEEE Transactions Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1802-1817, Oct. 2007.

V. Ordonez, G. Kulkarni, and T.L. Berg, “Im2Text: Describing Images Using 1 Million Captioned Photographs,” Advances in Neural Information Processing Systems, vol. 24, pp. 1143-1151, 2011.

A. Sadeghi and A. Farhadi, “Recognition Using Visual Phrases,” Proceedings IEEE Conference Computer Vision and Pattern Recognition, pp. 1745-1752, 2011.

A.W. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, “Content-Based Image Retrieval at the End of the Early Years,” IEEE Transactions Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1349-1380, Dec. 2000.

R. Socher and L. Fei-Fei, “Connecting Modalities: Semi-Supervised Segmentation and Annotation of Images Using Unaligned Text Corpora,” Proceedings IEEE Conference Computer Vision and Pattern Recognition, pp. 966-973, 2010.

A. Vailaya, M. Figueiredo, A. Jain, and H. Zhang, “Image Classification for Content-Based Indexing,” IEEE Trans. Image Processing, vol. 10, no. 1, pp. 117-130, 2001.

C. Wang, D. Blei, and L. Fei-Fei, “Simultaneous Image Classification and Annotation,” Proc., IEEE Conf. on Computer Vision, pp. 1903-1910, 2009.

Kevin Murphy, Antonio Torralba, Daniel Eaton, and William Freeman, “Object detection and localization using local and global features”, MIT Springer, pp.382-400, Jan 2006

D. Lowe, “Object recognition from local scale-invariant features,” in Proc. 7th IEEE Int. Conf. Comput. Vis., 1999, vol. 2, pp. 1150–1157

Lisin, A. ; Blaschko, M.B., Learned-Miller, E.G. ; Benfield, M.C. “Combining Local and Global Image Features for Object Class Recognition” in Proc. IEEE Conf.Computer Vision and Pattern Recognition, Jun 2005.


Refbacks

  • There are currently no refbacks.