COGNOiSe.com - The IBM Cognos Community

ETL, App Development and other tools => COGNOS DecisionStream/Data Manager => Topic started by: wyconian on 17 Aug 2011 04:43:21 AM

Title: Delta Strategy for Clobs
Post by: wyconian on 17 Aug 2011 04:43:21 AM
Hi everyone

I'm trying to come up with a 'fast' delta strategy for loading data from a clob field.  Any ideas?

I've tried a minus query transforming the clobs to chars but that takes ages, there is no 'update' date from the source application.

Any suggestions welcomed.

Thanks for your help.

Simon
Title: Re: Delta Strategy for Clobs
Post by: starschema on 18 Aug 2011 10:18:38 PM
One thing you could try, is storing the hash (md5, sha1, etc) of the CLOB field in your target table, you could then pull the business key and hash from the source table, and use this to compare against the hash in the target table, to determine if the field has changed or not.

Using the results of this comparison, you could then pull the full row for only rows where the CLOB has changed.

Depending on your requirements, it may make sense to hash the entire row, rather than just the CLOB column.
Title: Re: Delta Strategy for Clobs
Post by: wyconian on 19 Aug 2011 08:57:38 AM
Thanks for the advice, I'll check it out  :)