01-05-2012, 11:03 AM
How to formulate image processing applications
How to formulate image processing applications.pdf (Size: 139.13 KB / Downloads: 35)
1 Introduction
In the last fifty years, a lot of image processing applications have been developed
in many fields (medicine, geography, robotic, industrial vision, ...). Nevertheless,
the problem of formulating such applications in image processing terms has been
little studied. Image processing specialists develop their applications by trialerror
cycles and do not define a real formulation of the problem they consider.
Thus, applications are merely formulated through their solution and not by a
formal definition of the problem they solve.
However a complete and rigorous formulation of the application is essential
towards the goal of designing more robust and more reliable vision systems, and
in order to fix the limits of the application, to favor reusability and to enhance
the evaluation. Such a formulation has to clearly specify the objective and to
identify the range of the considered input data. Unfortunately, formulating an
image processing application is a problem of qualitative nature that relies on
subjective choices. Hence, an exhaustive or exact definition of the problem does
not exist. Only an approximative characterization of the desired application behavior
can be defined [1].
Our approach tends to acquire the user’s business knowledge and map it to
an image processing one in order to find a solution. Hence, we propose to study
the formulation of image processing applications through the design of a system
that allows inexperienced users to build their applications by interaction. In this
paper, we present a brief state of the art on image processing application formulation
(Section 2) and describe the global system we are developing (Section 3).
Then we propose a model for the formulation of image processing applications
and its formalization through an ontology (Section 4). Finally, the interaction
part of the system is explained (Section 5).
2 A. Renouf, R. Clouard, M. Revenu
2 State of the Art
Attempts to formulate image processing applications were conducted around
knowledge based systems such as LLVE [2], CONNY [3], OCAPI [4], MVP [5] or
BORG [6]. A priori knowledge on the application context (sensor effects, noise
type, lighting conditions, ...) and on the goal to achieve were more or less implicitly
encoded in the knowledge base. The formulation of the application is
reduced to the choice of a transformation objective in a list of predefined image
processing tasks which are well known. Therefore they make the saving of a
complete and explicit formulation of the image processing application.
Among these works, an original approach was proposed by J. Hasegawa et
al. [7]. The formulation is given by the user through a sample figure as a sketch.
This kind of formulation is interesting because it is objective and its nature is
quantitative. Nevertheless its scope is limited to a restrictive subset of segmentation
or detection goals and it does not really take into account the input images
variability.
More recent approaches bring more explicit modelling [8][9][10][11] but they
are all restricted to the modelling of business objects description for tasks such as
detection, segmentation, image retrieval, image annotation or recognition applications.
They use ontologies that provide the concepts needed for this description
(a visual concept ontology for object recognition in [9], a visual descriptor ontology
for semantic annotation of images and videos in [12] or image processing
primitives in [8]) or they capture the business knowledge through meetings with
the specialists (e.g. use of the NIAM/ORM method in [10] to collect and map
the business knowledge to the vision knowledge). But they do not completely
tackle the problem of the application context description (or briefly as in [9])
and the effect of this context on the images (environment, lighting, sensor, image
format). Moreover they do not define the means to describe the image content
when objects are a priori unknown or unusable (e.g. in robotic, image retrieval
or restoration applications). They also suppose that the objective is unique and
known beforehand (to detect, extract or recognize an object with a restrictive
set of constraints) and therefore they do not address its formulation.
3 A System to Formulate Image Processing Applications
Our work is part of a global project which aims at developing a system that
automatically generates image processing softwares from user-defined problem
formulations. This system is composed of two sub-systems (Fig. 1): a formulation
system for image processing applications which is the focus of our work, and a
planning system for image processing tasks [6]. At the moment, the system handles
only one image processing task at a time. If a real application needs several
tasks, the system is used as many times as there are tasks. It is also restricted to
image processing tasks. It considers only image-to-image transformations without
any interpretation of the image content. The user defines the problem with
the terms of his/her domain by interaction with the user layer of the formulation
system. This part of the system is a human-machine interface which uses
How to formulate image processing applications ? 3
image processing plan
Construction of the Ontology
Processing
Image
BORG
Planning System
for
Image Processing Tasks
in an image processing problem
Transformation of the user’s formulation
by interaction with the user
Formulation of the application
Formulations
Image Processing Applications
for
Formulation System
HERMES
Domain
Ontology
Image Processing
Layer
User Layer
Images
Application
Fig. 1. The system architecture of the project.
a domain ontology to handle the information dedicated to the user. It groups
concepts that allow the users to formulate their processing intentions and define
the image class. Then the formulation system translates this user formulation
into image processing terms taken from an image processing ontology. It uses a
set of rules to ground the symbols given by the user which are terms of his/her
domain and concepts of the domain ontology to image data which are concepts
of the image processing ontology (see Section 5.3). This translation realizes the
mapping between the phenomenological domain knowledge of the user and the
image processing knowledge.
The result of this translation is an image processing request which is sent to
the planning system to generate the program that responds to this request. This
cooperation needs the two sub-systems to share the image processing ontology.
Then the formulation system executes the generated program on test images
and presents the results to the user for evaluation purposes. The user is responsible
for modifying the request. This process is repeated until the results are
validated.
4 Formulation Model for Image Processing Applications
We propose a complete model for the formulation of image processing applications
[13]. This model allows to identify and organize the relevant information
which are necessary and sufficient for the planning system (or an image processing
specialist) to develop a convenient solution. It covers all image processing
tasks and is independent from any application domain. In this section, we present
a brief review of this model formalization through an image processing ontology
(notice that this ontology tackles the problem of the formulation from the image
processing experts point of view). This ontology is composed of two parts: the
objectives specification and the image class definition. It is written in OWL DL
which is based on description logics, defines 300 concepts, 23 roles and 183 restrictions.
During the translation step, the formulation system instantiates this
4 A. Renouf, R. Clouard, M. Revenu
* *
*
*
*
*
0..1
0..1
0..1
constraint
Control
Constraint
Performance
Criteria
Quality
Criteria
has
value Value
Numeric
Value
Symbolic
Value
has
task task criteria
Criteria
to Optimize
has−
error
has−
error
Level of
Detail
has
level
Elements
to include
include
exclude Elements
to exclude
has
object
Acceptable
Error
has
1
Objective
Object
Business
has
Fig. 2. CML representation of the objective specification model.
ontology and sets their properties using an OWL API in order to represent the
formulation of the user at the image processing level.
4.1 The Objective Specification
This specification relies on the teleological hypothesis [13]. This hypothesis leads
us to define an image processing application through its finalities. Hence, an
application objective is represented by a list of tasks with related constraints
(criteria to optimize with acceptable errors, levels of detail with acceptable errors,
elements to be excluded and included in the result) and a list of control
constraints. We present on Fig. 2 the formalization of this model using the CML
formalism of CommonKADS which covers the ontologies specification [14]. This
figure represents the highest abstraction level of the objective specification part
of the image processing ontology and does not show all its concepts. A concept
like Task is the root of a hierarchic tree where nodes are processing categories
and leaves are effective tasks. Processing categories belong to reconstruction,
segmentation, detection, restoration, enhancement and compression objectives.
Effective tasks are extract object, detect object, enhance illumination, ...
To illustrate the use of this model, we take the example of a cytological application
[15] where the user wants to extract serous cell nuclei. In Table 1, we
present the formulation of the application objective which is obtained from the
instances (and their properties) created by the formulation system in the image
processing ontology. The terms (task, extract object, criteria to optimize, ...) are
concepts of the ontology and terms in italic are instances created by the system
(object1 is an instance of the concept business object, num1 is an instance of the
concept interval).