Data manager 10.2 Audit Variables

[mwilliams]

I'm creating new Jobstreams within Data Manager 10.2.1.  When a Build fails, I email accordingly with the details of what failed.  I'm trying to use the Audit Tables, specifically dsb_audit_msg_line.  The $DS_AUDIT_ID variable is set at the start of the Job, and is the AUDIT_ID for that job.

However, if I've got 5 Sub-Jobs within, each Sub-Job increments that AUDIT_ID, and uses a different variable $DM_COMPONENT_AUDIT_ID.  So if $DS_AUDIT_ID is set at '1400510', the 3rd Sub-Job would have an ID of '1400513'.  The Log shows $DM_COMPONENT_AUDIT_ID each time with the incremented number, and that is the number that shows in the AUDIT Table dsb_audit_msg_line. 

My problem is trying to report the error messages in an email, and not being able to use the $DM_COMPONENT_AUDIT_ID variable.  It is not available to me.  $DS_AUDIT_ID is available to me all day long, but it doesn't help me when I'm 10 levels deep.  I could calculate what the AUDIT_ID should be, but depending on various factors, such as other Jobs running at the same time, that would be a total guessing game.

Can anyone help with this?

Matthew Williams

[MFGF]

I'm creating new Jobstreams within Data Manager 10.2.1.  When a Build fails, I email accordingly with the details of what failed.  I'm trying to use the Audit Tables, specifically dsb_audit_msg_line.  The $DS_AUDIT_ID variable is set at the start of the Job, and is the AUDIT_ID for that job.

However, if I've got 5 Sub-Jobs within, each Sub-Job increments that AUDIT_ID, and uses a different variable $DM_COMPONENT_AUDIT_ID.  So if $DS_AUDIT_ID is set at '1400510', the 3rd Sub-Job would have an ID of '1400513'.  The Log shows $DM_COMPONENT_AUDIT_ID each time with the incremented number, and that is the number that shows in the AUDIT Table dsb_audit_msg_line. 

My problem is trying to report the error messages in an email, and not being able to use the $DM_COMPONENT_AUDIT_ID variable.  It is not available to me.  $DS_AUDIT_ID is available to me all day long, but it doesn't help me when I'm 10 levels deep.  I could calculate what the AUDIT_ID should be, but depending on various factors, such as other Jobs running at the same time, that would be a total guessing game.

Can anyone help with this?

Matthew Williams

Hmmm. A bit of a conundrum. You could perhaps define a new variable at the the beginning of the top-level jobstream, and have each sub-job update it with the contents of $DM_COMPONENT_AUDIT_ID. If one of the sub-jobs fails and your main jobstream then drops into a leg where $RESULT is false, you could have an email node there which interrogates the contents of your manually defined variable to see what the last value of $DM_COMPONENT_AUDIT_ID was? Would that work for you?

Apologies if I'm misunderstanding - I'm always a bit slow - especially on Fridays! :)

MF.

[mwilliams]

I have this issue resolved, now.  MF, thanks for your suggestion.  Using a second variable for the same item helped me figure out the real issue, which for me is a 50/50 shot in the email portion. 

The solution was in 2 parts.

1)  The variable $DS_AUDIT_ID I mentioned is the correct one to use.  It will change depending on which level you tap into it.  So, while I saw the $DS_AUDIT_ID variable being set at the beginning job level, it's also used in the Sub-Job levels.  In the Log files, I saw the variable $DM_COMPONENT_AUDIT_ID being set at each step, so I assumed that was the one to use, but I was not able to access it.  It appears that variable is only used for Data Manager, most likely in reference to $DS_AUDIT_ID.  That threw me off track for a little while.

2)  Using the $DS_AUDIT_ID variable to show me the error message in an email was the next step.  Inside the email, for all other variables I had, such as $BUILD_NAME, $S_JOB_NAME, etc... I needed to use the Curly Brackets.  For instance, "ETL Failed - {$BUILD_NAME}".  So naturally, I assigned all the other variables with the Curly Brackets.  All the Tests on the nodes previous were telling me the variables were being populated properly, they just weren't showing up on the email properly.  After banging my head for what seemed forever, I got the bright idea to try one of the variables without the Curly Brackets.  For whatever reason, that worked!

So I now have an email with brackets around some variables, and some without.  There doesn't seem to be any rhyme or reason, as the variable type are the same char(8000).  But I now know if it doesn't work with the Curly Bracket, I'll use it without. 

After a week of banging my head on this issue, hopefully this post can be of use to someone else.

Matthew Williams

[MFGF]

I have this issue resolved, now.  MF, thanks for your suggestion.  Using a second variable for the same item helped me figure out the real issue, which for me is a 50/50 shot in the email portion. 

The solution was in 2 parts.

1)  The variable $DS_AUDIT_ID I mentioned is the correct one to use.  It will change depending on which level you tap into it.  So, while I saw the $DS_AUDIT_ID variable being set at the beginning job level, it's also used in the Sub-Job levels.  In the Log files, I saw the variable $DM_COMPONENT_AUDIT_ID being set at each step, so I assumed that was the one to use, but I was not able to access it.  It appears that variable is only used for Data Manager, most likely in reference to $DS_AUDIT_ID.  That threw me off track for a little while.

2)  Using the $DS_AUDIT_ID variable to show me the error message in an email was the next step.  Inside the email, for all other variables I had, such as $BUILD_NAME, $S_JOB_NAME, etc... I needed to use the Curly Brackets.  For instance, "ETL Failed - {$BUILD_NAME}".  So naturally, I assigned all the other variables with the Curly Brackets.  All the Tests on the nodes previous were telling me the variables were being populated properly, they just weren't showing up on the email properly.  After banging my head for what seemed forever, I got the bright idea to try one of the variables without the Curly Brackets.  For whatever reason, that worked!

So I now have an email with brackets around some variables, and some without.  There doesn't seem to be any rhyme or reason, as the variable type are the same char(8000).  But I now know if it doesn't work with the Curly Bracket, I'll use it without. 

After a week of banging my head on this issue, hopefully this post can be of use to someone else.

Matthew Williams

Glad you got is sorted. Referencing a variable in braces {} means you are referring to the variable value at initialization (ie the beginning of the process) whereas without braces you are referring to the variable value existing at that point in the processing.

Cheers!

MF.