PipeTableParser strip opening and ending characters

[doggy8088]

Here is my code that can run under LINQPad:

void Main()
{
	var builder = new MarkdownPipelineBuilder();
	
	var pt = new PipeTableExtension();
	pt.Setup(builder);

	var pipeline = builder.Build();
	
	var markdownText = GetText();

	var document = Markdown.Parse(markdownText, pipeline);

	var blocks = document.ToList();
	
	foreach (var item in blocks)
	{
		string blockText = ExtractText(markdownText, item);
		blockText.Pre().Dump("Parsed Output");
	}
}

public static string ExtractText(string text, Block item)
{
	var pos = item.ToPositionText(); // $295, 0, 13301-13375

	// parse "$295, 0, 13301-13375" into lines, columns, range variables
	var line = pos.Split(",")[0].TrimStart('$').Trim();
	var column = pos.Split(",")[1].Trim();

	var range = pos.Split(",")[2].Trim();
	var start = range.Split("-")[0];
	var end = range.Split("-")[1];

	var blockText = text.Substring(int.Parse(start), int.Parse(end) - int.Parse(start) + 1);
	return blockText;
}

string GetText() => """

| Attributes               | Details                                           |
|:---                      |:---                                               |
| `<docs-card-container>`  | All cards must be nested inside a container       |
| `title`                  | Card title                                        |
| card body contents       | Anything between `<docs-card>` and `</docs-card>` |
| `link`                   | (Optional) Call to Action link text               |
| `href`                   | (Optional) Call to Action link href               |

""";

When running this code, the parsed output will remove the opening and ending characters. It seems a bug.

[xoofx]

It might be a bug in the position (not all elements, specially the one not CommonMark like table are necessarily tested for their text position)

That being said, this ExtractText function feels complicated to extract the line/column/span information. Not sure why you are not using the properties behind (Span, Line, Column) here

public static string ExtractText(string text, Block item)
{
	var pos = item.ToPositionText(); // $295, 0, 13301-13375

	// parse "$295, 0, 13301-13375" into lines, columns, range variables
	var line = pos.Split(",")[0].TrimStart('$').Trim();
	var column = pos.Split(",")[1].Trim();

	var range = pos.Split(",")[2].Trim();
	var start = range.Split("-")[0];
	var end = range.Split("-")[1];

	var blockText = text.Substring(int.Parse(start), int.Parse(end) - int.Parse(start) + 1);
	return blockText;
}

[doggy8088]

@xoofx Yes, you're right. This is so much easier.

public static string ExtractText(string text, Block item)
{
    return text.Substring(item.Span.Start, item.Span.Length);
}

But the Markdig.Extensions.Tables.Table is still missing the first and last characters when parsing syntax. Here is my workaround for this issue:

public static string ExtractText(string text, Markdig.Syntax.Block item)
{
	var start = item.Span.Start;
	var end = item.Span.End;
	var len = item.Span.Length;

	if (item is Markdig.Extensions.Tables.Table)
	{
		start--;
		end++;
		len += 2;
	}

	// Markdig.Extensions.Tables.Table is missing the first and last characters when parsing syntax
	return text.Substring(start, len);
}

[doggy8088]

As I know, GridTable looks like this:

+-------------+-------------+
| Header 1    | Header 2    |
| ----------- | ----------- |
| Row 1 Col 1 | Row 1 Col 2 |
| Row 1 Col 1 |             |
+-------------+-------------+

And the PipeTable should looks like this:

| Header 1    | Header 2    |
| ----------- | ----------- |
| Row 1 Col 1 | Row 1 Col 2 |
| Row 2 Col 1 | Row 2 Col 2 |

In the GridTableParser.cs, the OpeningCharacters is +.

But in PipeTableBlockParser.cs, the OpeningCharacters is -. Why -?

The PipeTable's codebase is still too complicated to me. I still can't find the bug.

[xoofx]

In the GridTableParser.cs, the OpeningCharacters is +.

Because this is coming from https://pandoc.org/MANUAL.html#extension-grid_tables

But in PipeTableBlockParser.cs, the OpeningCharacters is -. Why -?

Because this is coming from GitHub behavior and also https://pandoc.org/MANUAL.html#extension-grid_tables

[doggy8088]

Do you mean this format?

- | Header 1    | Header 2    |
  | ----------- | ----------- |
  | Row 1 Col 1 | Row 1 Col 2 |
  | Row 2 Col 1 | Row 2 Col 2 |

• Header 1	Header 2
  Row 1 Col 1	Row 1 Col 2
  Row 2 Col 1	Row 2 Col 2

[xoofx]

As explained in the comment of PipeTableBlockParser here it is to discard list (that can start with -).

a | b
- | -
0 | 1

which is not supported by GitHub but was supported by pandoc. See the comparison here.

The parser for pipe tables is more complicated because we can only detect it after we have processed a paragraph, so that's why it is an inline parser and not a block parser.

[doggy8088]

I never know that. I always think it's a block parser. Can you implement another block-based PipeTable parser? I never know there is a scenario for inline usage. At least I have never used it this way. 😅

[xoofx]

I never know that. I always think it's a block parser. Can you implement another block-based PipeTable parser? I never know there is a scenario for inline usage. At least I have never used it this way. 😅

It is not an inline usage. In order to parse a "block" pipe table, we can only use | because we could only detect if a paragraph is actually a table once we have parsed its content. That means that:

a | b
- | -
0 | 1

Is initially parsed as a paragraph because we don't know when parsing a that it is actually a table after (e.g you could have a backstick a ``|`` b that is actually escaping the table).

That's why the pipetable is so complicated because we are treating | as a delimiter (similar to * or _ or emphasis), and then from there, we are trying to rebuild a table.

A naive implementation could have said: I'm just gonna split the line by | but that's not the solution that was taken.

[doggy8088]

Can you take a look on why PipeTable missing 2 characters? I'd like to fix the bug but I can't find the entry point of the position info.