This blog shows an example of a BizTalk Flat file schema with “Suppress Empty Nodes” set to Yes. In addition we demonstrate using this in an Azure Flat File Decode Logic App action. The XML from the Logic App action is not he same as you get from the equivalent BizTalk decoding. Finally, I argue it’s a Logic App bug and request a fix.
The evidence
Previously I have been recording my journey moving BizTalk solutions to Azure and continue this here. One of the tenets of my plan is that we can use the existing BizTalk schemas and BizTalk map XSLT. We use an integration account for this. Specifically most of the maps in our BizTalk exchange are very old, lacking unit tests and documentation. For this reason we do not want to revisit the logic and business rules in the maps at this stage.
Thus our proof of concepts test that Azure can use them by simply importing the schemas and maps. Up to the present time our experience with this approach is good. Creating Logic Apps to replace the BizTalk functionality has been quick and easy, not in small part because we could migrate the BizTalk schemas and maps without little fuss.
Surprisingly importing one flat file schema did not give the result we expect. Firstly a logic app triggers when a blob receives an invoice flat file. Secondly the “get blob content using path” action retrieves the file. Thirdly the Flat File decoding action converts the flat file to XML. Unfortunately the output of the decode action is an empty xml as shown below.
After much gnashing of teeth I found a workaround. Changing the “Suppress Empty Nodes” on the BidOneInvoiceExtractV2 schema from YES to NO, gave XML content.
It is important to realize that this XML output is not the same as that from BizTalk Server. Significantly, it contains attributes that are empty, the one from BizTalk Server does not. BizTalk Server uses the flat file disassembler pipeline component to decode the flat file and there are no empty attributes.
Using Suppress Empty Nodes
Sandro Pereira describes this clearly in Teach me something new about Flat File“.
David Burg issues a warning. When using Logic Apps Flat File Decode you should beware suppress empty nodes unintended effect. He summarizes by saying
The flat file schema property ‘suppress_empty_nodes’ is described as “Indicates whether or not to remove empty XML nodes after the parser generates XML instance data.” in the documentation https://msdn.microsoft.com/en-us/library/aa559329.aspx
He ends by saying as appropriate change it (suppress_empty_node) to No.
Is it appropriate?
Steps to Reproduce the error
Before answering this questions let’s give a simple example so you can study this in situ for yourself. You can download the BizTalk Schema and test file from here. In the first place lets consider the flat file shown below.
The output after decoding should be;
Notably the optional DeliverTo attribute is missing and the first detail line and the second detail line line is missing the optional DeliveryDate.
Next let’s set up an Azure Logic App to decode the flat file into XML. Firstly load the SimpleInvoiceFFTest.xsd into an integration account.
Secondly create a logic app and add the integration account to the properties.
My logic app is not finished yet but it looks like the one in the introduction. Thirdly, in the Flat file decoding action select the SimpleInvoiceFFTest in the dropdown. Finally after saving, submit the SIMPLEINVOICETEST.txt file to the logic app trigger and run it.
In this case the output is empty XML.
<?xml version=”1.0″ encoding=”utf-8″?>
<INVOICE xmlns=”http://SimpleInvoiceFFTest.SimpleInvoiceFFTest2″></INVOICE>
In contrast, after editing the schema, changing the Suppress_Empty_Node schema property to NO and uploading this to the integration account, the resultant XML is now
<?xml version=”1.0″ encoding=”utf-8″?>
<INVOICE xmlns=”http://SimpleInvoiceFFTest.SimpleInvoiceFFTest2″>
<WDBHDR SenderCode=”BID-WDB” ReceiverCode=”CONTOSO” DocumentType=”INVOIC” xmlns=””></WDBHDR>
<WDBDTL LineWarehouse=”4060″ InvoiceNumber=”I45194955″ LineNo=”1″ ItemCode=”158540″ Description=”CLAMS DINNER PACK FOAM” DeliverTo=”” DeliveryDate=”2019-03-06T10:58:09.5406038+13:00″ xmlns=””></WDBDTL>
<WDBDTL LineWarehouse=”4060″ InvoiceNumber=”I45194955″ LineNo=”2″ ItemCode=”107532″ Description=”PAPER GREASEPROOF HALF 400X330MM SHEETS” DeliverTo=”11 Contoso place” DeliveryDate=”” xmlns=””></WDBDTL>
<WDBTRL LineCount=”2″ xmlns=””></WDBTRL>
</INVOICE>
In the final analysis it is clear that the Azure Flat File Decode Action cannot handle schemas that have the suppress_empty_node set to Yes.
Why is Suppress Empty Nodes important?
In view of the above, is changing suppress_empty_nodes to ON, an acceptable workaround. I think the answer is a resounding no because this is common schema pattern that BizTalk developers use for Flat Files. Moreover, one of the selling points of using integration accounts is that it enables easy migration of BizTalk solutions to Azure. If you cannot import BizTalk schemas like the one above then you have some extra work to do. Furthermore I think that this is a bug in the Azure Flat File Decode Action that should be fixed.
Notwithstanding let’s consider what happens if we change suppress_empty_nodes from YES to NO.
Firstly if you have an optional attribute the empty string might be invalid because of the schema data type. For example our schema has an optional datetime which fails validation with “The ‘DeliveryDate’ attribute is invalid – The value ” is invalid according to its datatype”
Secondly an optional attribute with a string datatype might fail validation if there is a restriction on the schema. For example our schema validates the length of the DeliverTo string and fails with “The ‘DeliverTo’ attribute is invalid – The value ” is invalid according to its datatype ‘String’ – The actual length is less than the MinLength value”.
Thirdly down stream maps might fail if an empty string is transformed. It would be rather annoying and time consuming to revisit all the logic in our old maps.
Given these three points changing suppress_empty_nodes to YES is not a good workaround and the Azure Flat File Decode Action should be changed to support this schema property
Conclusion
To summarize I have shown that the Azure Flat File Decode Action does not support the schema property suppress_empty_nodes. Specifically empty XML is created. Additionally I am logging a support ticket with Microsoft to get this fixed.
