IQD file processing

[vivek_cyno]

The IQD file read 3000 records per second from oracle. I feels that it could be fast. We want to know what's the maximum number of records that an IQD can read.Can anyone assist.

[cognostechie]

There is no such thing as maximum or minimum for this. It depends on the SQL inside your IQD file. If it's a straight SQL without involving any joins, it would be fast. I have had those kind of IQDs reading 12000 records per second. If the SQL involves lots of joins and/or other derivations, it could be slower than reading 3000 records per second. That's the major idea behind splitting the IQDs for different dimensions and facts.

and this is the wrong forum for this. Moreover, you posted in multiple forums !

[nmcdermaid]

I don't agree. The OPEN phase in the transformer log is when Oracle performs all the joins and gets the data ready to send. the READ phase occurs after the OPEN phase is complete and is purely copying data from server to client.

Query complexity, indexes etc. affect the OPEN phase.

Network, driver configuration, hardware etc. affect the READ phase.


You can get a lot of improvement in the read phase in Oracle by changing the fetch buffer entries in the COGMDMOR.INI file

Improving the OPEN phase requiers tuning of your queries / database.

[cognostechie]

So you are saying what I am referring to is speeding up the OPEN phase ? So what is it that you don't agree with ? Even if you change the settings in the INI file that you mentioned, that improves only the READ phase so the problem with the OPEN phase still exists unless you spilt the queries into simple ones.

[nmcdermaid]

I guess it depends whether 3000 records per second is taken over the OPEN + READ phase or just the READ phase. I took your comment to refer to the READ phase when in fact the join complexity has no impact on the read phase.

Anyway I think we are splitting hairs - I wasn't trying to be rude, just clarify some things.