Feature Request: Provide a base serializer for YAML

[MrPippin66]

This has been requested multiple times, and I see no progress.

In the end, this boils down to PowerShell not supporting YAML conversion, like XML and JSON currently is.

The response is always "We don't support YAML, becuase the .Net runtime doesn't support YAML.

YAML is a base format in Linux based systems, etc, and a common format supported in a major language, like Python.

I don't understand why YAML isn't natively supported in .Net, and thus eliminate the reason why PowerShell doesn't natively support YAML.

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[MrPippin66]

Should be 'area-Serialization'

[teo-tsirpanis]

area-Serialization refers to the older Data Contract and XML serializers; assigning to area-Meta.

[ghost]

Tagging subscribers to this area: @dotnet/area-meta
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This has been requested multiple times, and I see no progress.

In the end, this boils down to PowerShell not supporting YAML conversion, like XML and JSON currently is.

The response is always "We don't support YAML, becuase the .Net runtime doesn't support YAML.

YAML is a base format in Linux based systems, etc, and a common format supported in a major language, like Python.

I don't understand why YAML isn't natively supported in .Net, and thus eliminate the reason why PowerShell doesn't natively support YAML.

Author:	MrPippin66
Assignees:	-
Labels:	area-Meta, untriaged
Milestone:	-

[davidfowl]

YAML is a base format in Linux based systems, etc, and a common format supported in a major language, like Python.

Is YAML suppoted natively by python? What other languages come with YAML support in the standard library?

[MrPippin66]

Apologies. It's not a standard part of Python.

PyYAML has become ubiquitous, but is not a part of the standard distribution.

[MrPippin66]

But the desire to have this function natively within PowerShell is still desirable due the native use of YAML within container and Ansible envioronments.

[stephentoub]

What prevents PowerShell from using one of the myriad of yaml libraries available via nuget? eg
https://www.nuget.org/packages/YamlDotNet

I see PowerShell relying on other such libraries, eg
https://github.com/PowerShell/PowerShell/blob/b58db8b8f70756301d63ca9c52d0cca69756ef73/src/Microsoft.PowerShell.Commands.Utility/Microsoft.PowerShell.Commands.Utility.csproj#L11

[MrPippin66]

My desire to have this supplied as a standard part of .Net (and thus resolving a reason PowerShell doesn't support it, natively) is primarily a support issue.

This isn't a niche solution, but a major infrastructure piece used by many mainstream products/solutions (beyond just Ansible and container solutions).

Y, PyYaml isn't s standard module in Python. But it is a solution Red Hat provides support for. (at least as a solution for other Red Hat products that also leverage this).

And it's not like Microsoft isn't internally using YAML for their own products (Azure Pipelines being one).

Lastly, yaml is a formal specification.

I would think all of the above would merit this being a technology provided within the core framework.

[davidfowl]

I don't think I've seen a YAML serializer built into any language's standard library. YAML certainly is ubiquitous in the cloud native space and https://www.nuget.org/packages/YamlDotNet is the defacto YAML implementation for .NET. I don't see a super strong argument for why it needs to go into the standard library (nothing in the standard library or anything that ships with .NET depends on it).

[amis92]

Just to point out one argument that seems to be a strong one from a perspective of user/customer, is having a good, stable, performant, supported, documented and hopefully standards-compliant implementation. Especially the support and documentation are major selling points for .NET I think. So, that's why I think it'd be great to have in BCL.

But another is, similarly as ASP.NET helped BCL embrace native JSON implementation, containerization and ubiquitous-ness of YAML in CI/CD pipelines is an argument I'd use. GitHub Actions and Azure Pipelines runners are both C#-based, so that's one major player at least potentially interested in YAML support :D

Edit: I've recently found https://github.com/hadashiA/VYaml to be a good, simple and perf-optimized yaml package, if anyone else stumbles upon this thread.

[StephenHodgson]

I don't think I've seen a YAML serializer built into any language's standard library. YAML certainly is ubiquitous in the cloud native space and https://www.nuget.org/packages/YamlDotNet is the defacto YAML implementation for .NET. I don't see a super strong argument for why it needs to go into the standard library (nothing in the standard library or anything that ships with .NET depends on it).

David, I love the stuff you do, but I have to respectfully disagree here. Just because there is a community supported library, doesn't mean it should not be a native runtime component to dotnet. In fact, I'd argue the opposite, since the community needed to come up with an alternative solution.

I agree with @amis92, since the Azure and Github runners are both c# based, it only makes sense that they should be able to natively parse the build workflow files.

I heard over and over again the same exact arguments for JSON, yet today we have System.Text.Json it only stands to reason we also need System.Text.Yaml to complete the loop.

[am11]

What other languages have YAML support directly in the standard library?

I only know of two: Ruby and Crystal (yep, the programming languages 😜).

When we were overhauling the YDN parser to add YAML 1.2 support back when source generators were still a new thing, we discussed potential corner cases with the YAML spec team yaml/yaml-spec#113 (comment). That conversation got me thinking: instead of manually handling every parsing nuance, why not use a source generator to process the intermediate representation (IR) from the spec tool and automatically emit the YAML parser?

Of course, we ultimately ended up adding the support manually, and I got sidetracked with other projects, but building it out as a source generator could still be a pretty neat project to tackle (someday 😅)

[crhaglun]

So if I understand this correctly, the main motivation to not provide a YAML serializer is that there's no dependency on that format in the standard runtime, unlike XML and JSON.

There seems to be an element of trust (and perhaps to some extent discoverability?) driving the requests for a first-party solution. Would it be appropriate with a framework extension providing this functionality, Microsoft.Extensions.Yaml rather than System.Text.Yaml?

[davidfowl]

Over time certain formats may become more and more ubiquitous and there may be a point in the future where yaml is a given. The cloud native world certainly has adopted it but it hasn't made it over the threshold of importance to spend the time building a new serializer from scratch to put into the core libraries. System.Text.Json is enough work as it is already and that's without the newly added support for JSON schema etc.

TL;DR at some point there may be enough motivation for this to tip over from nice to have to must have but it's not there yet. It's a mixture of science (data gathering), art (do we as maintainers think it's the right thing for the runtime) and capacity (do we have the resources to do a good job and maintain thing forever).

[iSazonov]

From MSFT PowerShell team

We understand and agree that YAML is used heavily in the industry and would also like to see it supported natively within PS7. Adding a new dependency on an external assembly will introduce dependency collisions. We've already experienced such issues in the past. We also recognize that the popularity of the YAML modules on PSGallery means that YAML support is needed, but also that many folks are ok using 3rd party modules for YAML. If YAML support shows up in .NET runtime, we would certainly approve of builtin YAML cmdlets in PS7.

The circle never ends. 😃

[StephenHodgson]

The circle never ends. 😃

Circle of life 😂 Just grateful to see some traction and movement. Especially with all the API specs and DevOps platforms all requiring YAML serialization these days.

[kilasuit]

TL;DR at some point there may be enough motivation for this to tip over from nice to have to must have but it's not there yet. It's a mixture of science (data gathering), art (do we as maintainers think it's the right thing for the runtime) and capacity (do we have the resources to do a good job and maintain thing forever).

TL;DR - are we likely to see this happen in 2025 at all? (expand for a fuller reply)

The longer version

This I can fully understand, and that will only come when Microsoft & others start doing more to get those in the community more better onboarded with helping in maintaining projects, and not doing so just on goodwill alone because they want to see things improve. Discussing these things is great but doing so is taxing on us, especially those of us doing so without proper financial support behind us to properly maintain and sustain ourselves & all our downstream dependencies.

This will only truly change when there is a stronger process at governmental levels as well as organisational and individual levels to help those that are passionate, build out software like dotnet and others and not only are willing to provide constructive criticism, but recieve it too.

In respect to YAML, I'd like to see this in dotnet by the end of the year so that downstream in PowerShell we could add this functionality in, which I don't feel is an unreasonable ask in the grand scheme of things & would allow us to move on from this conversation onto other more interesting ones.

If that means Microsoft has to go and buy licenses or support contracts or shell out the development on a multi-year contract or even look to aquire businesses to do so, then that that's what I feel is what needs to start happening to get some of these things moving. That then changes the dynamic of these conversations somewhat, and overall I feel would help the community for the better. As someone that understands business, it's a big thing to say Hey I have backing and support from the likes of Microsoft whether that be via investment from Microsoft as a whole or from just a smaller segment of the organisation, like in my case the PowerShell Team. Investment doesn't always need to be finanical, but it sure helps those that could do with it. It also would go far in saying how you support those both in your partner network and out of it too.

We have a lot of interwoven trusts in the community, and trusts of different types, and I know that the We can't do this right now, due to resource constraints frustrates us all much more than it should & is definitely more visible with open source discussions like this. I am glad we can have these discussions, as are so many of us in the community too.

Taking dotnet Open Source was the right thing to do to allow more open discussions like this, but only gets the full benefits when the community behind it is properly supported & can access the required tools & tooling they need to be able to do so. We certainly are getting there, even if there is more that we could do.

Some are in better positions to sustain themselves than others, and have better supporting infrastructure and systems behind them (especially when you think internationally) and can continue to engage because those systems exist. If the UK didn't have the support processes & systems that we have, even though they need improvement, I'm not sure if I'd be here right now & able to type this after some rough times in recent years, but who knows, I'm still here. Moving forward, I know there is more that we as a collective can do to progress & keeping true to Microsoft's mission statement, which resonates with many in and outside of Microsoft, myself included.

So a simple ask @davidfowl, could we see built-in yaml support, whether built from scratch and directly maintained by Microsoft or via one of the 3rd party packages already out there that you then ship as part of the runtime by the end of this year? Or is the bar for inclusion in dotnet so high, that it can only be a 1st party solution, because it certainly isn't for Windows.

We can always revist whether or not if we go down the 3rd party package then requires a rebuild as a Microsoft 1st party solution at a later date. I know theres a number of other elements like risk when it's a 3rd party solution that come into consideration, like the Edgio issue recently, so just an indication of yes/no/maybe right now would be great & would help us downstream in projects like PowerShell to either give a timeline for inclusion, or a recommendation where we take dependencies on ourselves, which is what many are looking for here. Those that aren't willing to wait for this to happen & are willing to take the risks that come with that, can already make use of the 3rd party solutions available to us, so it's not for what I can see mission critical as such, at least not right now, however others are already seeing it that way.

From a Configuration Management perspective, which PowerShell, DSC, winget and other products in the market already require a way to work with yaml & is certainly something we could have done with as far back as 2016 as that was when it started being more used in those areas & would be really really nice for loads of us to not to be going into 2026 without it in dotnet in one way or another.

[davidfowl]

So a simple ask @davidfowl, could we see built-in yaml support, whether built from scratch and directly maintained by Microsoft or via one of the 3rd party packages already out there that you then ship as part of the runtime by the end of this year? Or is the bar for inclusion in dotnet so high, that it can only be a 1st party solution, because it certainly isn't for Windows.

No, I doubt we will ever do this again (like we did with JSON.NET in the early days of .NET Core).

From a Configuration Management perspective, which PowerShell, DSC, winget and other products in the market already require a way to work with yaml & is certainly something we could have done with as far back as 2016 as that was when it started being more used in those areas & would be really really nice for loads of us to not to be going into 2026 without it in dotnet in one way or another.

I have no doubt that somethings might push us in this direction like I said, but it's definitely not happening in .NET 10. As you know building a serializer is a HUGE undertaking, even for a big company like Microsoft.

[kilasuit]

but it's definitely not happening in .NET 10.

Thanks for adding this additional bit of clarity @davidfowl - this certainly helps us with messaging to others when asked in downstream projects like PowerShell where we regularly get asked for support to be built in. I know I and others will look forward to it when it does end up making it into a future dotnet release.

As you know building a serializer is a HUGE undertaking, even for a big company like Microsoft.

Yeah, I & many others certainly do not doubt the scale of the task to bring this kind of functionality natively into DotNet. Though, perhaps we may have hoped that with the help provided via AI tooling that something as big as this may have been feasible for it to make the bar for inclusion in DotNet 10 and then could be available for downstream projects like PowerShell to make use of.

[StephenHodgson]

As you know building a serializer is a HUGE undertaking, even for a big company like Microsoft.

So instead we push the effort to the community to maintain such a critical part of so much of our DevOps and automated workflows? There is also precedent for Microsoft to just incorporate the community projects into their own first party while working with those authors on integration.

Though, perhaps we may have hoped that with the help provided via AI tooling

💯 This. With source generators and the like it really shouldn't be that big a lift.

There are a few oddities and edge cases in yaml but it is a superset of json.

[davidfowl]

AFAIK YamlDotNot is a pretty high quality yaml implementation maintained by the community.

[StephenHodgson]

AFAIK YamlDotNot is a pretty high quality yaml implementation maintained by the community.

It's exactly the point I'm trying to make. Why isn't it too hard for people to do this in their spare time, but you guys push back about wanting to do a first party support?

Which is it? It's hard to do? Or you just don't want to do it?

[MihaZupan]

Two things can be true at the same time.
A community maintained project being of high quality does not imply that the problem it's solving is trivial.

[davidfowl]

Our JSON implementation took about 4 versions (aka years) to get to a decent place, trying to support features that JSON.NET supported, taking security and async into consideration, AOT support etc. etc. I do think building the next one means we can go in with LOTS of experience, and yes, we can use AI to help write the code, but writing code is not shipping and maintaining professional production quality software in 20+ year old ecosystem.

[am11]

AFAIK YamlDotNet is a pretty high-quality YAML implementation maintained by the community.

Yup, YDN is currently garnering around 62K downloads per day and ranks #49 top package on nuget.org https://www.nuget.org/stats/packages. With platforms like GitHub Actions and Azure Pipelines utilizing YDN for configuration, it shows even first party platforms depend on open-source tools and libraries that are NIH where it makes sense.

There are a few oddities and edge cases in YAML, but it is a superset of JSON.

If parsing JSON is (famously dubbed as) navigating a minefield, parsing YAML can be twice as tricky (to put it lightly 😅). Thankfully, there are tools available to help streamline the process; sourcegen from specs to programming languages or validate implementations against the official YAML spec test suite. Using these tools, developers can quickly deliver implementations with high confidence (e.g., 100% spec-suite compliance). However, as @davidfowl pointed out, designing an intuitive and robust API experience on top of it, gathering real-world use cases from the community and maturing the implementation has its own set of challenges.