Introduction
The Data Management Component (DMC) is a set of
interfaces handling data persistence for Planck DPC's
(in the IDIS framework). It will be used by both
consortia (HFI and LFI).
Since the DMC is a set of interfaces, different
persistence mechanisms can be implemented in a transparent way. This
insures some independance from the market (especially from the currently
highly volatile market of object databases, O2 being virtually dead, Object
Design appearing as the new leader as of June 2000).
Even if the two consortia choose the same implementation, which is not
required, care must be taken to preserve this independence as the
DMC interface is evolved.
Object persistence mechanisms
The DMC should provide object persistence, i.e. the
fact that the internal state of an object is preserved after the end of
the life of the application that created or modified it. The
DPC's will use this object persistence facility to store
data, like maps, TOI's, calibration information,
pointing information, etc.
Data persistence can be achieved in several ways. For L2/L3
DPC's, one should keep in mind that
- DPC software should be independant of the
DMC implementation. In particular, it should be able to
run, for testing and development purposes, without any
DMC implementation (or a basic flat file
implementation).
- Most objects handled during DPC processing will
be transient. Only input data and final products of a task will be
tagged persistent. Too tight a coupling between classes handled by
DPC software and DMC should be
avoided.
One way to achieve this is by using the Bridge design pattern [Gamma 1995], to
decouple interface and implementation inside the DMC.
The DMC should probably be kept as a separate package providing
persistence services, the science objects and classes having no explicit
dependency to the DMC.
The DMC interface should moreover make minimal
assumptions on the implementation layer functionalities.
Data structure details, and opaque persistence
DPC software will need to store complex objects, which
structure can evolve. The precise internal data structure does not need to
be handled by the DMC (semantically, although the DMC needs of course to know
how the structure is built in terms of basic data types).
This facilitates schema evolution, since this provides skill separation
between precise data structure definition (a physicist task) and data
persistence (a database specialist task).
This also keeps the DMCI simple: it is a low-level
software layer, that should not depend on higher-level (and thus more
volatile) concepts.
Persistent data should be associated with tags and flags on which
DMC requests can be based. A history of each object
should also be kept, with support for versioning. Finer-grained searches
should be handled by higher-level software layers.
Use cases
The data model specifications document should be
complemented by use cases. A list of the kind of requests on each
type of data will help determine the
information that need to be
accessible by the DMC, i.e.
which classes of the
data model have to be known by the
DMC, and which
tags should be associated with them.
Requirements summary
The DMCI should implement the
following requirements:
- The DMCI should
provide a set of interfaces to handle persistence on any kind of
data.
- Software that uses DMCI facilities
should be able to run without those facilities.
- Making data
persistent should not imply a hard-coded dependency between the data
definition (in C++ or Java) and the DMCI.
Persistence should in particular not be obtained by heritage, but by
mechanisms like bridges and delegation.
- The DMCI should make persistent all basic data types
(integers, floating point numbers, strings, and n-dimensional arrays
of such data), as well as aggregates of those basic types. It should
provide an API to define new data types as
aggregates of already defined types. It should provide an
API to define searchable fields and tags
associated to data.
- The DMCI should associate
to any persistent data the following characteristics: name, creation
date, modification date, creator. It should handle version history:
several versions of an object can be stored and retrieved.
- The DMCI should provide an API to
locate an object based on searcheable fields and tags, and on version;
to retrieve and store an object; and to lock an object while it is
being processed.
The DMCI could provide
more functionalities for
fundamental data structures like
TOI's and maps. This
remains to be defined,
keeping in particular in mind that, for instance,
unforeseen pixelisation schemes could be used and that this should neither
break the DMC nor imply major developments in a low
software layer.
Revisions
- 0.1 Éric Aubourg, 07/07/2000
- 0.2 Éric Aubourg, 10/07/2000