Selection load slower with HeaderFooters

[softpromark]

TextControl.NET v14 SP2

As you likely remember by now, we use your control to process a homegrown markup language that provides support for If's, ForEach'ing, field codes, and such. I have a test document with just this in it:

<FOREACH {{SomeBusinessObjectCollection}}>
Fee: {{.SomeField}}
</FOREACH>

Simple enough. At a basic level, when my parser hits the FOREACH it marks the start position of the block to copy, then when it hits the closing token it "snapshots" the content between the tokens using Selection.Save(out byte[], InternalFormat). The parser restarts at the start of the block for the first business object in the targeted collection, and evaluates the field code during this pass. When it hits the closing token again, it pastes in a copy of that snapshot we took using Selection.Load( byte[], InternalFormat) and we continue on for the 2nd object in the collection. We'll continue pasting and parsing until we've exhausted every object in that collection. Works great...

But here's the catch, and it doesn't make sense. If the document being parsed has 0 HeaderFooters (across sections or whatever) its lightning fast iterating over dozens of business objects. The byte[] is size 1250. But when I run the same doc - but with a Header and a Footer in it - it takes 10 times longer to perform each Load operation and the byte[] size is 2504. If there's a Header, Footer, FirstPageHeader, and FirstPageFooter, it takes 20 times longer to perform each Load operation and the byte[] size is 3204.

I've profiled this and the entire increase in time is spent on that Load call. Empirically, a Load with 4 HeaderFooters in place can take a few tenths of a second each. Ack! If there are many objects, it can take 10-20 seconds to process, but without any HF's, that same doc runs in less than a second.

I don't get it. The snapshot byte[] should be the same regardless of Headers and Footers since my snapshot selection is the same, no? Its as though the Selection Save and Load operations need to gyrate (much) more simply because of the presence of Headers and/or Footers.

Anyway, I've started to think of workarounds. To that end, I do have a question:

Is it possible to disconnect a HeaderFooter and add it back later? I don't think so, but then I could parse each HeaderFooter, remove each one from the document (but hold references), parse the main document, then add the HeaderFooters back in. However, it doesn't look like I can Remove them and Add them back later. Its too bad, because programmatically removing those HeaderFooters on the fly makes the parsing immediate again. I just can't get them added back in when I'm done.

Any thoughts you have on this would be appreciated! If anything, to explain what HeaderFooters have to do with the little isolated Save/Load I'm attempting out in the main body.
Mark

[softpromark]

So my latest workaround attempt involves ripping through the Sections and taking an RTF snapshot of each HeaderFooter's content as well as other relevant settings like Distance and such, removing all HeaderFooters from the entire document, performing my parse of the main body, then going back through all my snapshots and restoring them into new HeaderFooters that are added at the appropriate locations (based on Section and Type). Its a total hack, but it gets rid of that unexplained delay on every Load() attempt and my markup documents are resolving upwards of 80% faster. I just have to wonder how reliable this is really going to be (e.g. will they always restore the exact same as the original). Ideally, I could have held each HeaderFooter reference, removed them from their collections, and then just wired them back up when I was ready, but apparently the model doesn't work that way.

As an aside, I tried doing that Save/Load stuff from my original post using RTF strings instead of internal format byte arrays but that didn't help at all. The Load() of that little snippet was still 20 times slower with the mere presence of the 4 HeaderFooter elements as compared to processing that same document with none.

Does this sound like the right approach? Can you provide any other suggestions or insight into what I'm seeing?

Thank you,
Mark

[Gunnar Giffey]

Hello Mark,

Thank you very much for posting and the provided information.
I appreciate your testing and I would like to discuss this with our developers.
To be able to do that, I would like to ask you if you could provide me the RTF snippets, that you used for your testing, so I can load string 1, stop the time and load string 2 and measure the time for loading again.
At the moment, I am not sure what is exactly happening when loading a document or RTF without or with headers and footers, but I know that our devs can figure that out.

Now, concerning your workaround:
It should be possible to use arrays to hold the headers and footer, that you remove from the document, but you could also just copy the main text to another TextControl and do the parsing in that new control.

Anyway, I hope your hack works, but I fully agree that this is not a final solution and loading documents with headers and footer should not take 20 times longer.

[softpromark]

Thanks Gunnar. I agree that slipping the main body in and out would be more appealing than doing it by header/footer. However, it would require quite a bit of refactoring on our end because the TextControl instance is being wrapped by another object that stores a lot of related state data significant to the parser and so slipping the control itself in and out is causing 01-240F errors currently (likely due to our wiring being broken).

Anyway, the header/footer approach is working really well ... except for the dreaded "extra line break" at the end of some HeaderFooters when reinserted. I always struggle with those darn things!

For example, I do this to store the data into my own little "HeaderFooterData" class for each HeaderFooter before stripping them off:

private void Configure( HeaderFooter headerFooter )
{
	headerFooter.Activate( );

	Distance = headerFooter.Distance;
	Section = headerFooter.Section;
	Type = headerFooter.Type;

	// snapshot the content
	headerFooter.Selection.Start = 0;
	headerFooter.Selection.Length = headerFooter.TextChars.Count;
	string content;
	headerFooter.Selection.Save( out content, StringStreamType.RichTextFormat );
	Content = content;

	headerFooter.Deactivate( );
}

And I do this to restore:

private void Restore( HeaderFooterData data )
{
	// grab the appropriate Section
	Section section = TextControl.Sections[data.Section];

	// add a HeaderFooter of the appropriate type to the Section
	if ( section.HeadersAndFooters.Add( data.Type ) )
	{
		HeaderFooter headerFooter = section.HeadersAndFooters.GetItem( data.Type );
		headerFooter.Activate( );
		headerFooter.Selection.Start = 0;
		headerFooter.Selection.Length = headerFooter.TextChars.Count;
		headerFooter.Selection.Load( data.Content, StringStreamType.RichTextFormat );
		headerFooter.Distance = data.Distance;
		headerFooter.Deactivate( );
	}
}

Say its processing the Header and all I've got is some bold text - no lists or tables or anything, and no visible paragraph break after "(Continued)" below. In other words, the bottom dashed line when you activate the header in the design view of the document (before parsing) is right up against the bottom of the text.

The headerFooter.Selection.Text is:
"\r\nOrder Number: 2009020001\r\nCommitment Number: 123\r\n\r\nSCHEDULE B - SECTION II\r\n(Continued)\r\n"

But the resultant RTF is this:
"{\rtf1\ansi\ansicpg1252\uc1\deff1{\fonttbl
{\f0\fswiss\fcharset0\fprq2 Arial;}
{\f1\froman\fcharset0\fprq2 Times New Roman;}
{\f2\froman\fcharset2\fprq2 Symbol;}}
{\colortbl;\red0\green0\blue0;\red255\green255\blu e255;\red0\green0\blue0;\red0\green0\blue255;}
{\stylesheet{\s0\itap0\widctlpar\f1\fs24 Normal;}{\*\cs10\additive Default Paragraph Font;}{\s16\itap0\nowidctlpar\f0\fs24 [Normal];}{\s17\itap0\widctlpar\tqc\tx4320\tqr\tx8640\f1\f s24\sbasedon0 Header;}{\s18\itap0\widctlpar\fi-360\li360\f1\fs24\sbasedon0 List;}{\s19\itap0\keepn\widctlpar\qc\tqc\tx4680\f0 \fs24\b\sbasedon0\snext17 Heading 1;}{\s20\itap0\widctlpar\tqc\tx4320\tqr\tx8640\f1\ fs24\sbasedon0 Footer;}{\s21\itap0\widctlpar\sa120\f1\fs24\sbased on0 Body Text;}}
{\*\generator TX_RTF32 14.0.520.501;}
\deftab1134\paperw12240\paperh15840\margl1440\marg t1440\margr1440\margb1440\widowctrl\formshade\sect d
\headery720\footery720\pgwsxn11906\pghsxn16838\mar glsxn1134\margtsxn1134\margrsxn1134\margbsxn1134\p ard\s17\itap0\widctlpar\qc\tx720\tx1440\tx2160\tx2 880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx79 20\tx8640\tx9360\tx10079\plain\f0\fs24\b\cf3\par\t rowd\irow0\irowband0\lastrow\trgaph0\trleft0\trfts Width1\trftsWidthB3\trftsWidthA3\trpaddfl3\trpaddf t3\trpaddfr3\trpaddfb3\clvertalt\clftsWidth3\clwWi dth5040\cellx5040\clvertalt\clftsWidth3\clwWidth50 40\cellx10080\pard\s18\intbl\nowidctlpar\tx720\tx1 440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx64 80\tx7200\tx7920\tx8640\tx9360\tx10079\plain\f0\fs 20\cf3 Order Number: \plain\f0\fs20\cf4 2009020001\plain\f0\fs20\cf3\cell\pard\s18\intbl\n owidctlpar\qr\sl233\slmult0\tx720\tx1440\tx2160\tx 2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7 920\tx8640\tx9360\tx10079 Commitment Number: \plain\f0\fs20\cf4{\txfielddef{\*\txfieldstart\txf ieldtype0\txfieldflags16}{\*\txfieldtext 123{\*\txfieldend}}}\plain\f0\fs20\cf3\cell\row\pa rd\s17\itap0\widctlpar\tx720\tx1440\tx2160\tx2880\ tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\t x8640\tx9360\tx10079\plain\f0\fs24\b\cf3\par\pard\ s17\itap0\widctlpar\qc\tx720\tx1440\tx2160\tx2880\ tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\t x8640\tx9360\tx10079 SCHEDULE B - SECTION II\par\plain\f0\fs20\cf3 (Continued)\par\par }"


I already do enough weird (?) "trailing \par stripping" in other places and I think its because I never fully understood where it comes from and under what conditions it can be removed. Sometimes you need to leave it there (if it represents a list item or a table cell) but sometimes you need to strip it off.

How do you guys recommend dealing with these seemingly injected paragraph feeds? The way we work around them here just seems so loosey-goosey.

I did try saving it off as InternalFormat instead, but then when I restore it later the font face changes (although the extra line break is gone). RTF doesn't seem to do that.

Thanks!
Mark

[softpromark]

I am "ending" this thread because I've come to realize that the problem is almost entirely due to the FormattingPrinter property being assigned to something other than "Standard".

See new thread 321609 for details.