Publishing Syntax Extensions for use with Remark

[benrbray]

I finished writing a syntax extension that adds support for pandoc-style citations of the form [see @wadler1990; also @hughes1989, pp. 4]. Following the convention I see that other extensions use, this extension is divided into three parts:

• micromark-extension-cite tokenizer
• mdast-util-cite converts tokens to an MDAST tree
• remark-cite wraps the above two utilities into a plugin that can be used with remark

I'm now faced with the question of how to publish these packages. I see three options:

1. each util gets its own git repo and its own npm package
2. all of the utils live in the same git repo, but each gets its own npm package
3. all of the utils live in the same git repo and the same npm package

Currently, most extensions to remark choose the first option, but this has disadvantages:

• Whenever I change the micromark-extension-cite tokenizer, it is likely I will need to also make changes to both mdast-util-cite and remark-cite. If they all live in separate npm modules, there is a risk of them all getting out of sync. Their version numbers should all increase in lockstep, anyway.
• If all three live in separate git repos / npm modules, it will need to locally link them all together for testing, which adds a lot of friction to development, and might cause synchronization issues.

To me, Option 3 makes the most sense. However, I'm inclined to choose Option 2 as a compromise between convenience and convention. So I guess my question is:

• Why do most extensions currently use Option 1?
• Is there a reason Option 3 might not be desirable for consumers of my extension?
• Is there a reason Option 2 might not be desirable for consumers of my extension?

Thank you!

[wooorm]

Great question!

I’m going with 1 personally. This is because I value modularity a lot. If folks don’t like the remark layer, then they can use micromark and mdast. If folks don’t need ASTs either, they can go with micromark only. Or someone can replace they higher-level layers with new projects and still use the lower-level projects. Of course: the ASTs are very useful and sometimes required, so micromark-only is not the best for some ideas.

But this does lead to some complexity for users, especially newcomers, and complexity maintaining these projects.

On the other hand, you can more freely change things if they are separate: a micromark extension/mdast utility could change it’s user interface resulting in a semver major, but that change can be pulled into the remark plugin in a semver minor.

I think 3, or 2, are both fine for your use case. You can always split and externalize things later. I’d say it depends most on who you see using citations: just remark users?

[benrbray]

Thanks! I went with Option 2. Here's my remark-cite extension.

[wooorm]

Nice! Really cool to see folks making stuff with this, even though it's quite undocumented for now.

I'd love to hear more about what was complex. What you ran into. What worked well and what didn't!

[wooorm]

And also: what should be added to MM. Types through JSDoc and esm are planned. But you might have the unique insight on what's missing docs or api wise!

[benrbray]

Sure, I can share some thoughts, especially about type ergonomics. Where's the best place for that? Should I dump all my thoughts into a single discussion thread here? Draft PRs in the individual affected repositories?

[wooorm]

I’m most interested in the non-type things, as I’ll get to those soon anyway (unifiedjs/unified#121).
If there are already actionable things, they can go into PRs. But a more informal discussion thread/dump of your experience is also appreciated!