Is it possible to output an ixf formatted file from a DS build?

[eknight]

The day to day of our system runs in a "delta" mode where only the recently changed rows are processed and updated. However every now and then a bug is fixed which requires all of our historical data to be reprocessed. This is turning out to be a severe performance bottle neck, especially as the amount of data continues to grow. The DB2 load output option seems like an ideal solution but unfortunately DS seems only able to use the DEL file format which breaks any unicode strings.

Outputting a IXF file, even if DS doesn't do the subsequent load itself, would be ideal.

On a side note, what is the best option for occasionally having to process ~20-100 million rows? Our usually throughput size is on the order of tens of thousands of rows, so a massive scaling of the hardware is undesirable.   

[MFGF]

Data Manager doesn't create IXF files, unfortunately. The bulk loader options create plain text files.  Sorry - it's probably not what you wanted to hear.

For occasional large loads, it's tricky to say how best to handle them, as many factors could come into play. Some businesses rent temporary cloud space, but I guess your biggest issue would be the throughput of data if the cloud is external.

Regards,

MF.

[eknight]

Sorry - it's probably not what you wanted to hear.

As long as it's the correct answer then it's exactly what I wanted to hear. At least I can cross IXF import off my list  ;)

At the moment we're looking at a something similar to your suggestion. Basically having a separate machine that crunches the data and then manually moving it over with export / import. This way we get a reasonable turnaround time and we don't strangle the primary machine for everyone else. It's not a very nice solution though, especially because any automation will need to be ksh scripts that we write ourselves.

If you come across any solutions for something like this, I'd love to hear it.

Cheers,

[ROBER82167]

The DB2LOAD delivery module should be able to handle Unicode strings. Set the "Data File Encoding" Module Property to "Unicode (UTF-8)".

[eknight]

The DB2LOAD delivery module should be able to handle Unicode strings. Set the "Data File Encoding" Module Property to "Unicode (UTF-8)".

It doesn't work because the DEL format is hard coded in the DB2LOAD delivery module. If I could change the format to IXF it'd be fine but that's not currently possible. The DEL format can break fix length unicode strings.