A colleague of mine came to me with this problem yesterday and it had me beat for a while. I asked him to validate the map inside visual studio to see what the resultant XSLT looked like. To our horror we discovered that the resultant XSLT was about 48Mb. Now we knew what the problem was but how do you solve it.
If we removed all the links from the map and validated it again the resultant XSLT was still about 48Mb. On opening the XSL we found it was full of default templates. The schema that we where mapping to imported a lot of other schemas. One imported schema in particular added a lot to the default XSL that was generated. It was an enumeration of all the country codes in the world and this type was used in many places on our schema.
How do you stop default generation of XSLT if their a no links in the map? Change the “GenerateDefaultFixedNodes” from “Yes” to “No” as described by Brett ( http://biztalktalk.wordpress.com/2009/06/05/biztalk-mapping-fixing-huge-maps/ ). As promised the resultant XSLT decreased to about 10Kb and above error went away. Thanks Brett.