This chapter provides a high-level, feature-oriented overview of the nevisDataPorter component and provides information about the context in which nevisDataPorter can be used.
nevisDataPorter can be used to export data from a data source to a data sink keeping source and sink in sync. It can also be used to do batch data transformations as part of data migration tasks. The following picture gives a coarse overview of nevisDataPorter's architecture.
A DataSource is used to retrieve objects (e.g., from a database). Each object retrieved by a DataSource is then passed down a chain of DataFilters. Each DataFilter may create Attributes for the object to be exported. The resulting object is then passed to a DataSink which stores the object (e.g., in LDAP directory).
nevisDataPorter was designed in line with the following design goals:
- Flexible and extensible data migration framework.
- May run within a Java EE application server or standalone as a server or command-line application.
- Flexible configuration language.
- Detection of configuration errors during initialization.
When exporting data from a source to a sink, the characteristics of data entities may vary on the data source and data sink. To address this mismatch, data transformations need to be applied. The following transformation-demands need to be addressed:
- An entity is a piece of data to be provisioned from a data source to a data sink.
- An entity on the source corresponds to exactly one entity on the sink.
- Multiple entities on the source correspond to one entity on the sink.
- An entity on the source corresponds to multiple entities on the sink.
- A named subset of the data of an entity as denoted as an attribute.
- An attribute on the source corresponds to exactly one attribute on the sink.
- Multiple attributes on the source correspond to one attribute on the sink.
- An attribute on the source corresponds to multiple attributes on the sink.
- An attribute on the source needs a different encoding on the sink.
Data Source Implementations
The following data source implementations are available:
- The BeanDataSource uses Java objects as data sources.
- The CSVDataSource uses CSV-files as its input.
- The JDBCDataSource uses JDBC to access a database.
- The LDAPDataSource uses JNDI to access an LDAP directory.
- The NevisIDMSource uses a web service to access nevisIDM.
- The NevisIDMHistorySource uses the history web service to access nevisIDM.
- The SoapDataSource uses SOAP web services to request data.
- The RestDataSource uses REST-ful web services to request data.
- The XmlDataSource uses XML file as its input.
Data Sink Implementations
The following data sink implementations are available:
- The BeanDataSink adds all objects into a pre-existing list.
- The CSVDataSink appends objects to a CVS-file.
- The EmailDataSink uses SMTP to send e-mail messages.
- The JDBCDataSink uses JDBC to update a database.
- The JmsDataSink uses JMS to send messages.
- The LDAPDataSink keeps an LDAP directory in sync with some data source.
- The ActiveDirectoryUser sink is a derivation of the LDAPDataSink. It implements the special constraints Microsoft Active Directory enforces for user creation.
- The NevisIDMSink uses a web service to update data stored in nevisIDM.
- The NevisAgentDataSink sends scripts to NevisAgent and commands the agent to execute the scripts.
- The SoapDataSink uses SOAP web services to send data.
- The RestDataSink uses REST-ful web services to request data.
- The TextDataSink is used to write plain text files.