COSOL WHITE PAPER

Data Reconciling & Archiving

This paper is the third in a series summarising lessons learned, to provide insight for executives who are faced with a possible large-scale program of this nature within their own organisation.

Our point of view

Experience shows that one critical success factor for these programs to realise the value in their business cases is data migration. The axiom “garbage in, garbage out” remains as relevant today as it did when computers were first invented. COSOL strongly believes:

Introduction to data migration

Data migration can best be explained as three distinct phases

• Pre-migration: data profiling and remediation (White Paper #1)
• During migration: data standardization and loading (White Paper #2)
• Post migration: data reconciliation and archiving (White Paper #3)

Recapping the key takeaways from the pre-migration:

1. Data will typically be exposed as a major risk and cost in large scale digital transformations due to unknown and/or poor quality.
2. Strong data ownership and data governance is critical.
3. Commence data profiling and data remediating as early as possible.
4. Six data quality dimensions need to be remedied through the data migration.
5. The process of applying remedies to standardise the data for the new system is an iterative process consisting of multiple rehearsals; and
6. An integrated cutover plan is critical to the overall success of the program.

With these insights, this white paper examines the phase post migration where production data is reconciled in the target system(s) and legacy data is archived.

Data reconciling and archiving objective

The objective of this phase is to ensure that relevant, accurate and auditable production data is loaded into the new system in a way that enables new ways of working, and to archive legacy data into a vault such that it can still be accessed should operational, statutory or regulatory needs arise. Refer figure 2.

To archive or not to archive? That is the question…

Hopefully by now the key takeaway of strong data ownership has been addressed, as these business owners are now faced with this very important decision. And their response could enable the value of new ways of working, or they could inadvertently destroy value by retaining the status quo.

Step change – taking the step

Without strong business ownership, project teams are often left to their own devices and will default to bring as much data as possible from the existing system into the new one “just in case”. The issue with this is that the new system is now reporting on business and process performance as it related to the old way of working, which masks the step change in performance between old and new. It may also inhibit the ability to adopt change thereby limiting the value of the transformation project.

The better practice is to bring only the master data and open transactional data needed to support the new way of working. Taking this very important step ensures that you have a “clean” new system, with a new beginning and a new set of KPIs that will clearly show the step change between the old and the new.

Old data is not lost. It is safely archived Into a cost -effective vault and remains accessible should the need arise, and business owners should remain resolute about the archiving decision and not succumb to the ‘easy way out’ by electing to bring unnecessary old data into the new system. Take the step.

Business owners should continually pose the question: if ways of working is new, then why bring “old” data?

Data Reconciling

With the big decisions made on what will be archived and what will be loaded into the new system, it is now time to reconcile the production data to ensure that values have not been corrupted during the migration.

The purpose of the data reconciliation process is to provide the business the confidence that the data they intended to migrate has been migrated, accounting for the remediation of that data throughout the entire migration process.

Data reconciliation is the responsibility of the business data owner. Successful projects engage all stakeholders including business process owners, audit teams and functional data team members.

It is critical that the business understands the lineage of the data from existing systems to the new, and that that journey can be explained and proven beyond the life of the project.

Successful data migration projects engage with business process owners, internal and external audit teams to understand specific statutory, regulatory, and operational requirements for the reconciliation activity.

Without this strong business ownership there is a high risk that:

1. A technical project team will complete a technical reconciliation only of the data migration process, resulting in reconciliation reports like “Records attempted: 100, actual records loaded: 100”.
2. The lineage of the data will be lost in technical scripts that developers understand, but when they leave the building, future audits will be unable to easily grasp the logic of before and after values.
3. (As stated earlier, there is also the risk that too much of the old data is brought across).

To mitigate these risks, the functional consultants shown in figure 2 should prepare reconciliation workbooks for the data owners. These workbooks provide transparency over the migration process, and a functional reconciliation ensuring subsystem to subledger and general ledger integrity is maintained.

An example is where general ledger balances are loaded into a target system as an opening balance, however, there are several sub-system data objects that may update the balance sheet (e.g. inventory loads). In this case the inventory process will update the associated GL balance which needs to take into consideration the way that pricing strategies are applied (moving price average, fixed etc) and stock on hand calculated, as these may result in financial differences between the existing and target systems that must be reconciled.

Data Archiving

With production data reconciled and loaded into the new system ready for go live, attention must turn to the decision of what to do with legacy data. There are two options:

1. Change the old systems to read-only with a limited number of users having access should the need arise; or
2. Migrate the historical data into a “data vault” so that it is “business readable” without requiring the source application to interpret the data.

While the first option may appear expedient, it has several limiting factors.

1. Old software must be retained to read the data. This may have licence, maintenance, and security implications, particularly if the software is old and no longer supported by the vendor.
2. The old software will require old hardware to run on, or a cost to replatform the old software to run new hardware in a data centre or in the cloud.
3. The old software will require people with the skills and knowledge of that software to be retained.
4. Reverting to old software too readily may inhibit the step change; and
5. The old system and data have no linkage to the new system and data. This makes the simple task of looking up what is vendor number 567 in the new system much harder, as it was known as vendor number 123 in the old system.

Placing legacy data in a ‘Vault’ has many advantages over retaining it in old ‘read only’ software and hardware.

The second option addresses all of these shortcomings and is made possible by the fact that the data has gone through the six steps of remediation referred to in the second white paper to get to this point, and as a result, data mapping and value mapping is all in place.

All that is now needed is a data vault with a simple user interface, and the embedded value mapping provides a window into the past for comparison purposes without compromising the performance measures of the new process.

Old software and hardware can now be retired, and skilled personnel can be retrained and refocussed in line with new systems and new ways of working, reinforcing the step change.

Legacy data retention

While a topic in its own right, it is prudent to remind the reader of the importance of a clear cata retention policy. Good data archiving and disposal practices were once driven by the high cost of retention of disk and tape storage facilities. In this digital age, storage has been commoditised and is no longer the cost driver it once was, leading to a situation where more data may be retained than required.

Data is an asset, until it’s a liability.

It is a legal requirement for a business to provide data that the business has retained over a particular matter when requested.

In this modern age of litigation, digital records are discoverable, and hence we strongly recommend that any archiving strategy is coupled with a retention strategy. Retention should consider and comply with the minimum statutory, regulatory, and operating requirements of the business, and data surplus to this ought to be disposed of.

Data reconciliation and archiving takeaways

1. Business owners should continually pose the question: if ways of working is new, then why bring “old” data? Old data masks the step change in performance between old and new. It may also inhibit the ability to adopt change thereby limiting the value of the transformation project.
2. Data reconciliation is the responsibility of the business data owner. Successful projects engage all stakeholders including business process owners, audit teams and functional data team members.
3. Placing legacy data in a ‘vault’ has many advantages over retaining it in old ‘read only’ software and hardware. Old software and hardware can be retired, and skilled personnel can be retrained and refocussed in line with new systems and new ways of working, reinforcing the step change.
4. Data is an asset, until it’s a liability. It is a legal requirement for a business to provide data that the business has retained over a particular matter when requested. Retention should consider and comply with the minimum statutory, regulatory, and operating requirements of the business, and data surplus to this ought to be disposed of.

The final white paper in the series will focus on the practical application of the best practices that have been outlined in white papers one through three. This will Include case studies and client experiences that underpin the key take-aways.