Have you ever faced that issue? When your workflow is running smoothly and out of nowhere it gets suspended? Then when you hover the “i” icon to get more details you are informed, that “The workflow instance was too large to persist”? Well I had. Here is what I found and how I recommend to solve it.

[tds_note]   The described scenario and error is related to the InfoPath Form used as the Form Library item, not when used to design a list or library form.[/tds_note]

I faced the problem when I was working  with Nintex for Office 365. Client thought, that it was caused by the workflow size (it had like 95 actions). On top of that there was an InfoPath form that was being saved into the form library. The form itself was quite large, it had 3 pages, a lot of fields. 

The workflow continuously was running into “Suspended” claiming that it is too large to persist, showing that sort of reasons:

RequestorId: 86e24769-e969-3395-0000-000000000000. Details: The workflow instance was too large to persist (over 9801 kilobytes uncompressed, over 6830 kilobytes compressed). The maximum persistable workflow instance size is 15360 kilobytes uncompressed and 5120 kilobytes compressed.

What made me really curious was the fact, that workflow itself, when uncompressed had 1,2MB (source: http://help.nintex.com/en-US/O365/o365/O365WorkFlow/BestPractices/BP-Determinefilesize.htm) so still there was a lot of free MB before reaching the limit. The form, with attachments, was not exceeding 1MB either. But anyway – what has the form in common with the workflow size? Well, read on 🙂

My investigations

I gave it a try. I split the workflow into 4 smaller, the size of the main, approval workflow decreased to 430KB. But even though, it was running very slow and when I added a 2,5MB attachment to the InfoPath form (so that the form has like 3,6MB) the workflow gets suspended again.

I also tried to create a small (15 actions, circa 100KB size) workflow, working on the library, but not touching the contents of the form itself. And to my biggest surprise the workflow started and ended extremely fast without getting suspended.

That makes me wonder…

First – the InfoPath. You know, that when you use it as the form in a Form Library this is just a pure XML. Everything it “is” is defined in the XML – all images you add to the layout, formatting, colors – all is saved in that single XML.

When you add a file control, that allows to attach a file to the form, the uploaded file itself also gets converted and encoded as a Base64 string and… guess what? Yes! It gets saved inside the XML structure 🙂

[tds_note]    In SharePoint Online the limit for a single InfoPath form, together with attachments is 5MB and you cannot change it.[/tds_note]

Then the workflow. I realized that the major difference between my test workflow and the production ones are the actions used for querying and updating the XML (InfoPath XML of course) – the “Query XML” and “Update XML” actions. My initial guess was that to execute them Workflow Manager has to “load” all the XML from the form into the workflow instance.

After the contents are loaded, usually it has to find a specific node, using xpath query and then execute a defined action for that node. If that was just a simple XML, that could probably be quite fast, but when it comes to a structure from a large form, filled up with Base64 encoded strings that makes a really large data set, which processing time is longer. I suppose that the moment, when the Workflow Manager serializes that data and stores it to the database is when it exceeds the threshold (leads). Can’t say for sure, it’s a guess only, but that would explain to me why a 1MB workflow + 1MB form exceed the 5MB threshold.

Conclusions

It turns out that no matter what is done with the Nintex Workflow, this is not it’s fault. I think this is a fault of the SharePoint Workflow Manager engine, that handles the XML processing. And as long as the InfoPath form is used for that purpose, process will possibly get into issues. LEave a comment if you have other hints.

How to solve it?

• In SharePoint on-premise, where you are the master of the SharePoint built-in thresholds, you can simply increase that limit by running a PowerShell command:

  Set-WFServiceConfiguration -ServiceUri http://workflowmanagerurl:12291/ -Name “WorkflowServiceMaxInstanceSizeKB” -Value NEW_LIMIT_IN_BYTES

  [tds_info]    After setting the new limit, you have to restart the ‘Workflow Manager Backend’ windows service so that your changes will be reflected.[/tds_info]

• In both, SharePoint Online and SharePoint on-premise, it’s worth to map fields from InfoPath to SharePoint columns. You do that in the “File –> Publish –> SharePoint Server” window. Then you need to select a column, and check the “Allow users to edit data in this field …” option:

  

  That way, once you change a value of the column from your Nintex Workflow, the value in the form will get updated as well, but you don’t need to use Query and Update XML actions. In my opinion this should be done with all fields that you need to query or update in the form.

  What is important – you can make that sort of binding only with simple fields. No “Containers”, “Lists” and data from inside of them can be bond. This also applies to “Person/Group” field. So it is very limited, but still worth trying. 

• In both, SharePoint Online and SharePoint on-premise, if you have license for Nintex Forms, really use it. The biggest advantage is that it doesn’t keep any XML inside the form, it saves all data directly to lists or splits the XML into single columns. Ex. each repeating section is saved as a single column, so there is much less data to process. The “Form Data” (NFFormData in Nintex for Office 365) field, that holds all the form structure as XML is saved into Nintex DB so there is again no need to load a lot of data into workflow instance.

What is your experience? Please share your findings in comments.