If you are unable to create a new account, please email support@bspsoftware.com

 

Impromptu (with Distinct) clause and Transformer - Performance and duplicattes

Started by LarryN, 17 Feb 2011 12:47:15 PM

Previous topic - Next topic

LarryN

I have a client running Imp 7.5 against a SQL Server 2005 (on Win 7), a 1-table View. When I run the View in SQL Server, I only get one row per ticket_No field. Total 900,000 rows

When I run the IMR, I get three duplicate rows per Ticket_No field/column, save as IQD and run Transformer - Very fast - but triple the measure values (if summed)

- so I use the remove Duplicate Rows option in IMP, run the IQD in Transformer, and now it takes 5 minutes to complete 1000 rows - this is unacceptable.

Does anyone know of a performance issue or a tip to get Transformer to read "Distinct" IQD files faster?

Thanks


MFGF

If you open the IQD in Notepad, how has the SQL query been defined?  Is there anything in there which you can see would cause the duplication?

MF.
Meep!

LarryN

The IQD shows that the "Ticket_No" is only coming from the source SQL Server View table. I will have to verify that the column in the View is actually the master column and not the detail one, but I doubt that since, as mentioned, when running the View in SQL Server - I do not get the duplication of rows?????

Also when running the SQL from the IQD separately - no duplication (more question marks???)

Thanks - I'll Keep looking