Contest on Arc Segmentation

Arc Segmentation Contest at the GREC2005 Workshop

( new!) Download the scanned test images and their ground truths, and all participants’ results of the arc segmentation contest at GREC2005 from this zipped file. View the test images and their ground truths together at here (thanks to Daniel and Thomas).

We are pleased to announce that the contest on arc segmentation will be held at the GREC2005 workshop, Hong Kong, August 25-26, 2005. This will be the sixth in the series of graphics recognition contests organized by the International Association for Pattern Recognition's technical committee on graphics recognition (IAPR TC10). The first contest, held at the GREC'95 workshop in University Park, PA, focused on dashed line detection [1], [2], [3]. The second contest, held at the GREC'97 workshop in Nancy, France, attempted to evaluate complete (automatic) raster to vector conversion systems [4], [5], [6], [7]. The third contest, held off-line associated with the GREC'99 workshop in Jaipur, India, also aimed to evaluate complete (automatic) raster to vector conversion systems. The fourth (at GREC’01 Kingston, Ontario, Canada) and the fifth (at GREC’03 Barcelona, Spain) focused on arc segmentation.

The contest this time will also on arc segmentation, which is an important and difficult problem in the field of graphics recognition. This contest will test the abilities of participating algorithms/systems to detect arcs from raster images. We plan to develop and use a set of performance metrics based on the published line detection performance evaluation protocol [8] to evaluate and compare the participating algorithms/systems on-line at the workshop site with test data of different quality and complexity. Pre-contest training images and the performance evaluation software are provided before the contest so prospective participants can try their systems and improve them for optimal performance. Test images could be synthesized and/or real scanned images. Although the contest is on arc segmentation, other line types and/or text may also appear in the test images to increase the complexity. We welcome participations of both commercial software with arc segmentation functionality and the academic research prototypes of arc segmentation algorithms. As has been the case with previous graphics recognition contests, we will enable anonymous participation. A participant can choose to remain anonymous either from the outset or in the contest report, which will be published as a paper that evaluates the performance of the various algorithms following an accepted protocol.

The performance evaluation (PE) tool is available and can be downloaded at ArcEval.exe. It is a console application running on Microsoft Windows systems. It accepts as input two vector files (the ground truth vector file and the detected vector file) in the VEC format defined by Dr. Atul Chhabra and outputs the PE results (Dv—the Vector Detection Rate, Fv—the Vector False Alarm Rate, and VRI—the Vector Recovery Rate) define in the line detection performance evaluation protocol [8]. In this tool, we only consider the recognition precision of solid arcs. That is, although there are other classes of vectors, e.g., straight lines, dashed lines, text, in both the ground truth file and the detected file, we only care about Dv for ground truth solid arcs (i.e., how those ground truth solid arcs are detected?) and Fv for detected solid arcs (i.e., how precise those detected solid arcs are?).

If you cannot run this PE tool, you can send me your ground truth file and detected file and I can run the tool on your files and send you back the PE result.

A preliminary test image and its ground truth vector file are also ready for download. We will continue to post more test images and their ground truths on this site.

More test images with various levels of noises and their ground truth vector files are also available. Click here to download the zipped package. The method used to generate these noises is described in a paper.

The contest summary for the fourth contest (on arc segmentation) at GREC’01 is now available here. The test images used in the contest are also available. Download the four synthesized test images and their ground truths from this zipped file. Download the three scanned test images and their ground truths from this zipped file.

The contest summary for the fifth contest (on arc segmentation) at GREC’03 is now available here. Download the three scanned test images and their ground truths from this zipped file.

Any comments/suggestions/questions, please send email to Liu Wenyin.

References

[1] R. Kasturi and K. Tombre (eds.), Graphics Recognition: Methods and Applications, First International Workshop, University Park, PA, USA, August 1995, Selected papers published as Lecture Notes in Computer Science, volume 1072, Springer, 1996

[2] B. Kong, et al., "A Benchmark: Performance Evaluation of Dashed Line Detection Algorithms," in Graphics Recognition: Methods and Applications, Lecture Notes in Computer Science, volume 1072, Springer, 1996.

[3] D. Dori, L. Wenyin, and M. Peleg, "How to win a dashed line detection contest," in Graphics Recognition: Methods and Applications, Lecture Notes in Computer Science, volume 1072, Springer, 1996.

[4] A. Chhabra and I. Phillips, "The Second International Graphics Recognition Contest - Raster to Vector Conversion: A Report," in Graphics Recognition: Algorithms and Systems, Lecture Notes in Computer Science, volume 1389, Springer, 1998.

[5] I. Phillips, J. Liang, A. Chhabra and R. Haralick, "A Performance Evaluation Protocol for Graphics Recognition Systems," in Graphics Recognition: Algorithms and Systems, Lecture Notes in Computer Science, volume 1389, Springer, 1998.

[6] A. Chhabra and I. Phillips, "A Benchmark for Graphics Recognition Systems," in Proceedings IEEE Workshop on Empirical Evaluation Methods in Computer Vision, Santa Barbara, CA, 1998.

[7] I. Phillips and A. Chhabra, "Empirical Performance Evaluation of Graphics Recognition Systems," in IEEE Transaction of Pattern Analysis and Machine Intelligence, Vol. 21, No. 9, pp. 849-870, 1999.

[8] Liu Wenyin and Dov Dori, "A Protocol for Performance Evaluation of Line Detection Algorithms", in Machine Vision and Applications, Special Issue on Performance Characteristics of Vision Algorithms, Vol. 9, No. 5/6, pp. 240-250, 1997.