How do we read txt or CSV source files sequentially in Data Manager

[SSNCOG]

Hi All,

I have 5 identical txt or csv files and I will have to use then as source for my staging table.I have created 5 data sources to merge the data from these files and its working fine.
But is there any way we can read these files one after other with single connection.

Any suggestions and help in this regards are greatly appreciated.

Thanks,
SSNCOG

[UseCog]

I can only think to create 5 builds and then run it sequentially in a job stream... does it work for you. Or, are running after this requirement due to some other specific reason?

[SSNCOG]

Hi,

Thanks for reply.

I used 5 data sources in same build to merge the data and its works fine.
I didnt try for 5 build for five source files.I am sure it will work.
But my requirement is I will place 5 files in a folder and I want DM to read these files one by one using a single build.is it possible in DM 8.4?

Thanks,
SSNCOG

[UseCog]

I think you can do this by creating a def file with file name aaa*.csv and move the file from 'the' folder to another after when this is loaded(can use system command). I havent tested this, but I feels positive here.

[SSNCOG]

Hi,

Thank you so much for your help.Its working fine now.

I need one more help, is there any way we can capture the file name along with the incoming records?

Thanks,
SSNCOG

[UseCog]

you can store all file names in a table using the system command and then do the same again after moving the loaded file and find the missing one... this is not looking a direct one. But it may work if you are trying to house keep the loaded files name.