I want to talk about metadata, SharePoint Workflow History logs, and how to use this data for different purposes. People underestimate this part when thinking about workflows, and just focus on how to do the workflow logic.
I usually classify the log data the comes out of a workflow into two types: Audit data and Debug data.
Audit data is the data that auditors ask for in order to validate the process integrity. If you have a solid change control policy in place, and you have a workflow in SharePoint to control a process, auditors usually ask for proof in form of workflow logs. For example, if you have a workflow to control the process of creating service account in active directory, auditors will come and will ask you for a proof that these accounts are created in active directory after passing through a workflow approval cycle. To do that, you can code your workflow to generate audit log data to be exported and shown to the auditing team once requested.
Let me give you another example. I had a solution that is used to track new changes in a data center. To do that, I created a list in SharePoint and anyone can submit a change request that passes through couple of approvals. Auditors and security team will come every quarter and ask: “Give us a proof that any change in the data center is logged and passed through approval”. IT Admins will then go and export the Audit logs as a proof. Auditors also require audit data to be available for a full year for any change happening in the data center.
Audit data is used mainly for security reviews and should have long retention (one year for example). The main purpose for such log is only for auditing and proof that an approval cycle is in place to control the process actions.
On the other hand, SharePoint workflow logs can be Debug Logs. Debug logs are used mainly for the team working on writing and supporting the workflow. If you write a complex workflow, you would like to through couple of logs at certain points of the workflow life cycle to track what the workflow is doing and if it is working as expected.
Suppose you have a workflow that calls a web service, sends couple of email notifications, and start approval requests. For you, it would make sense to generate a log before calling the web service, and after you got the web response back, and perhaps log the response code, to make sure the web service call is working just fine. Also, you could log that you are trying to send email notifications now, and that a task is generated for this person waiting for his approval. All these logs are only used for you as a workflow programmer to track the workflow status and action at different points of execution.
Usually debug logs will have a very short retention, as you only want to keep such logs for a month in most cases.Auditors and security team do not care about these log messages.
In the next parts, I will start talking about how to plan these logs and why I personally do not like to use the built in workflow history for this purpose.