Banking on big data: preparing for the convergence of transactional and analytic information

Banking is an information intensive business: information is used to drive decisions about risk, provide insight into customer needs, behaviour and trends, and assess the overall health and position of the organization.

Banking information can be divided into two main categories: current transactional information and analytic historic information. The first is real time and current, likely more situational and siloed and the latter is likely to have more perspective but may be one day or more old.

These previously disparate worlds are coming together, here’s why:

  • Technology is improving and entire systems and data can run in memory across clustered systems, part of the advantage of separating online from analytic data is to ensure that online performance is maintained.  With ample high speed capacity and new structures this is less of a constraint.
  • Requirements for contextual history are increasing:  Understanding a transaction at a moment in time is less useful than understanding that transaction in the context of previous, similar transactions — this is applicable to fraud, customer service and marketing, anti money laundering and other purposes.
  • The cost of this technology is decreasing:  processing power, disk capacity and other costs are decreasing allowing “parallel supercomputing” type capabilities at an attainable price point.
  • Centrality of data storage eases maintenance:  by storing the data in a single repository it is easier to apply privacy, audit, retention and archival rules.
  • A new category of data is emerging:  grey data.  This is largely unstructured data that is captured from channels and other sources that may or may not be related to structured data.  Organizations are starting to retain this data and mine it for relevance.
  • Big data is proven:  Facebook, yahoo, google all use parallel clustering to drive petabytes of data — this technology is ‘cloud friendly’ and there are several open and commercial packages that can enable this approach.  IBM has an informative research presentation on this topic.
  • Data is increasingly seen as an enterprise asset:  banks are realizing that data has multiple stakeholders and that the enterprise can benefit from the use and management of information.

The default approach to data in the past was retain what you need and discard everything else — the new approach will be to retain everything and discard what needs to be purged for regulatory reasons.

Future core banking platforms will be less concerned with purging and archiving transactional data and more focused on turning that data into useful information.

To prepare for this convergence it is important to have an inventory of the structured and unstructured data sources that may be impacted by this trend and to consider the semantic models that the data can be mapped to.  The Enterprise Data Management council seeks to establish common semantic models for financial instruments and concepts.  Having a common model will be useful when data is aggregated into a common cluster.

The technology is moving very quickly in this area and preparing for convergence is more than being aware of the technology aspects of this trend but considering that any data stored today may need to be aggregated and clustered tomorrow.

Leave a Reply

Skip to toolbar