DS losing rows

[eknight]

The system I'm working on has the weird phenomenon that about once a month one random row is not processed. I'm slowing going crazy trying to find the error in any of our SQLs or data and can't find a thing. The only common variable is the rows are lost on days when the system has an unusually large load.

I'm slowing becoming convinced that when DS is forced to do some sort of memory management due to low resources there is an off-by-one error in DS somewhere that is causing the missing row problem. I'm using DS version 7.1.778.0 which is from 2004 and no longer supported.

Has anyone else ever heard of a problem like this in DS or experienced something similar?

[MFGF]

Hi,

DS will only do memory management if you have set a working memory limit on the build, or if you have implemented memory breaking. Both of these are only really valid when a build is either set to detect duplicates (reject or merge) and/or to aggregate measures up the levels of a hierarchy. If you are doing neither, your build will read, process and deliver a row at a time, and will have a quite small memory footprint.

Have you managed to identify which row is not being processsed? Is it always the first? If you're unsure, you could perhaps add a derivation to the build that uses a logmsg() function to write the key of each row to the log file as the build is progressing? Might help you to pin down where the loss is occuring?

As you say, DS7 is no longer in support, so you're not going to be able to get help from the IBM support desk. Is there any reason not to upgrade to a newer version and get back into support?

Cheers!

MF.

[eknight]

Actually we're using the reject/merge features pretty heavily. Are there flags that are set in order to use the memory limit or memory breaking? Is there anyway to check if we're using this.

One of the new proects is using DataManager finally but there are no plans to remake the legacy system in a new version and management is very resistant to larger software upgrades. Also I'm pretty sure that DS7 being out of support means it's license free ;-)

[MFGF]

Hi,

If you go to the properties of your build and on to the Memory tab, you will see a value in the "Working memory limit (Mb)" field if a limit has been set. Otherwise Data Manager will not be managing the amount of memory it requests. If you go to the Breaks tab, you will see items in the right-hand "Break on" pane if memory breaking has been defined.

Not sure whether a product being out of support means a free-for-all on licensing. I'd strongly doubt it, to be honest. If IBM happen to implement a Software License Review I imagine it will cover all IBM software in use in your organisation, whether supported or not.

Cheers!

MF.