Keeping track of information state

In the Data Center Operation ETL solution, just about any piece of information can be in one of four different states at any point in time:

The states are: NEW, CHANGED, DELETED, SYNC_DELETED and SYNCED. In the typical life-cycle, information starts in the NEW-state.

Data Center Operation will pick up the information as new, interpret the information and add it to the Data Center Operation model. Once the information has been read into Data Center Operation, it will be moved to the SYNCED-state.

When the information is updated from the external system, the state of the information should be set to CHANGED. When Data Center Operation synchronizes its information, the changes will be reflected in the Data Center Operation model. If the external system decides to remove a piece of information it should be moved to the DELETED-state. Data Center Operation will subsequently handle cleanup of the information inside Data Center Operation and report back the state SYNC_DELETED. 

The state-system is primarily used to speed up the process of synchronizing data from the staging database to Data Center Operation internal database. Only data/information in the NEW- or CHANGED-state are expected to contain information that is not already known by Data Center Operation, therefore only those are read and synchronized.

Note: information in the SYNCED- SYNC_DELETED- and DELETED-state will not be read by Data Center Operation, and subsequently changes to the information will not be updated into Data Center Operation. 

Tracking Data Information State in Export Data

The export database uses the same strategy for keeping track of the state of data, as the import database, with the one exception, that the data will never be marked as synced.

The red arrows represent state-changes made by the external system, either through direct data manipulation or through a transformation. The green arrows represent state-changes made by Data Center Operation as part of the synchronization of information.