Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate regex patterns against non-backtracking engine #16687

Merged
merged 1 commit into from
Mar 24, 2025
Merged

Conversation

jeskew
Copy link
Member

@jeskew jeskew commented Mar 23, 2025

Resolves #16676
Resolves #16681

Microsoft Reviewers: Open in CodeFlow

@jeskew jeskew requested review from a team and Copilot March 23, 2025 19:45

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces functionality to validate regular expression patterns against .NET’s non-backtracking engine. The key changes include:

  • Adding helper methods in TypeHelper to validate regex patterns.
  • Enforcing regex pattern validation in StringType.
  • Updating various resource type factories to use the new validation helper.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/Bicep.Core.IntegrationTests/ScenarioTests.cs Adds a new test method to ensure provider-sourced regexes degrade gracefully.
src/Bicep.Core/TypeSystem/TypeHelper.cs Adds helper methods for validating regex patterns against the non-backtracking engine.
src/Bicep.Core/TypeSystem/Types/StringType.cs Enforces regex pattern validation in the StringType constructor.
src/Bicep.Core/TypeSystem/Providers/MicrosoftGraph/MicrosoftGraphResourceTypeFactory.cs Updates to apply the new regex validation helper.
src/Bicep.Core/TypeSystem/Providers/ThirdParty/ExtensibilityResourceTypeFactory.cs Updates to apply the new regex validation helper.
src/Bicep.Core/TypeSystem/Providers/Az/AzResourceTypeFactory.cs Updates to apply the new regex validation helper.
Comments suppressed due to low confidence (2)

src/Bicep.Core.IntegrationTests/ScenarioTests.cs:7085

  • Consider adding a case with an explicitly invalid regex pattern in this test. This would clearly verify that the graceful degradation behavior is working as intended.
public void Non_spec_compliant_provider_sourced_regexes_degrade_gracefully()

src/Bicep.Core/TypeSystem/Types/StringType.cs:13

  • [nitpick] The error message includes regex delimiters (slashes) which may be misleading if the pattern does not normally include them. Consider removing the delimiters or clarifying in the message that they are added for readability.
throw new ArgumentException($"The supplied regular expression pattern /{pattern}/ is not valid", error);
@@ -115,7 +115,7 @@ private TypeSymbol ToTypeSymbol(Azure.Bicep.Types.Concrete.TypeBase typeBase, bo
case Azure.Bicep.Types.Concrete.StringType @string:
return TypeFactory.CreateStringType(@string.MinLength,
@string.MaxLength,
@string.Pattern,
TypeHelper.AsOptionalValidFiniteRegexPattern(@string.Pattern),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We end up parsing each regex twice with this pattern, but I think that's fine since provider types are cached and including a check in StringType constructor means that downstream code can assume any pattern attached to a StringType instance is valid.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a second I thought this fixes #8409, maybe next time 😆

Copy link
Contributor

github-actions bot commented Mar 23, 2025

Test this change out locally with the following install scripts (Action run 14036496189)

VSCode
  • Mac/Linux
    bash <(curl -Ls https://aka.ms/bicep/nightly-vsix.sh) --run-id 14036496189
  • Windows
    iex "& { $(irm https://aka.ms/bicep/nightly-vsix.ps1) } -RunId 14036496189"
Azure CLI
  • Mac/Linux
    bash <(curl -Ls https://aka.ms/bicep/nightly-cli.sh) --run-id 14036496189
  • Windows
    iex "& { $(irm https://aka.ms/bicep/nightly-cli.ps1) } -RunId 14036496189"

Copy link
Contributor

Dotnet Test Results

    78 files   -     39      78 suites   - 39   33m 44s ⏱️ - 15m 34s
11 912 tests  -     12  11 912 ✅  -     12  0 💤 ±0  0 ❌ ±0 
27 582 runs   - 13 783  27 582 ✅  - 13 783  0 💤 ±0  0 ❌ ±0 

Results for commit f0dad59. ± Comparison against base commit 66b73c2.

This pull request removes 1780 and adds 610 tests. Note that renamed tests count towards both.

		nestedProp1: 1
		nestedProp2: 2
		prop1: true
		prop2: false
	1
	2
	\$'")
	prop1: true
	prop2: false
…
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000
�ӽ\u000e�0\u0010\u0007�}
���ʵ����Wh��\u001f�\u0018(������\u0001�`b�c/�^���to��5G[7LeD)���@\u0011��\u000fP"�~����\u0002 �g�dD�xS�Q���\u0007�mb���\u0005j\u0012Z��(G�\u0004mB�0Y��f��h��U�c��rCC9հ�C�Q����\u0015}�?�\u0002�B�\u0012Ze!�HC�\u0017�����X{�(��h
O9S��\u0000\u000c\u0000\u0000,"The path: index.json was not found in artifact contents")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000
�Խ
�0\u0010\u0007��>E�\u0003�|\rV� 8�H\u0015\u0004W	6`\u0005?�\u0015
��� .\u0016\u0017m\u0005�\u001b��\\u0012���K[M��ܥ`F\u0001�\u0019�4�\u0019���5\u0001H\u0004>p�Qk$�V\u001f��kQڋo���~�\u001cP[�\u0007�\u0008\u0004�:\u0006!��
Aa��m��DL϶�%�1s\u0015�\u0017�c]~h*���A���}���K ~\u0006���h\u0014�Fk���J&�<��h��G��x3[-�hH�\u0004 \u0008����s]�8\u0000\u000c\u0000\u0000,"'7' is an invalid end of a number. Expected a delimiter. Path: $.INVALID_JSON | LineNumber: 0 | BytePositionInLine: 20.")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000
�Խ\u000e�0\u0010\u0007��>E�\u0003�\u0016Z�������\u0000\u0015Έ\u0011$�	���-�q���ab�c�^����{�nѤX�,\u000c�\326n�R��w�\u0004"��\u0013\u000eJ\u0001!�\u001d}�\u001e��1�\u001de��~�\u001fQ�d9�\u0002�\u000fJK\u0001LG���_�Z2T\u0013���9�Y�b�.���������\u000c���tW�-�<���\u0001`�|\u0000Ax�Th�?K&�<�\u000f�\u001d
lKL\u001aLw�\u001f��6�d�5>���q\u001cǙ�\u000b\u00145k+\u0000\u000c\u0000\u0000,"Value cannot be null. (Parameter 'source')")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000\u0003��A\u000b�0\u0014\u0007��\u0014�\u0007����\u0016x\u0008:d�\u0005A�\u00189�@\u000b5\u0010��m��tQ\u000b��=xo����N�+�2]�0DR\u0004\u0010\u000c
\u0019\u0011���\u0016�\u001c`��\u0000�,"\u0000x���t�׍��(S��ADx��\u000b\u001dcN	g�\u0008\u0001�\u0008%\u000f��ܝ���n�9�y��\u0016^�kiˋ��o?������z|�?"\u0014�\u001d�M\u0015a6�\u0011c6��d��������$��z�M��g6�gr\u001c�q��\u0004��\u0000\u000c\u0000\u0000,"'7' is an invalid end of a number. Expected a delimiter. Path: $.INVALID_JSON | LineNumber: 0 | BytePositionInLine: 20.")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000\u0003��K
�0\u0010\u0006�=EN�ΤI'Yt��+\u0004-����\u0005��\u000b�E���
�[f2̄��d��m�\u000eE݈\u0014�I\u0004�\u001b\u0004�R��\u0003�Ȑ^�\u0001i�\u0019��워���a�%f� i�kOe�#)I�JC�Qij7����!����w�⫛\u0017��CC9հ�C�Q����\u0019��\u000f \u0019��|Ȥ�(�\u001f��|�L�y�7��(���\u0004\u0017ŏ�\u0000\u000c\u0000\u0000,"The path: index.json was not found in artifact contents")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000\u0003��K
�0\u0010\u0006�=E�\u0001b��U�{7�\u001b\u000f\u0010�\u0011+���BA���Bܴ��C0�230\u0013�\u001f���v\u000b6��&!�̚��QG	�{�aB#��(�Z*�\u0010nGߤǽnl�V�c�\u000f�\u0006�&�!fZp-#n\u000c1�1e��Ւ�\u001a�pi�s�\u0015)��Rߊ�=\u001fj_��ހ��M7�[�i(��\u0003���˿�2t��%����\u0007\u000e\u000e\u0005�%$
�;ȏP\u0005\u001b|��\u001a�K��y��M�\u0005�g�H\u0000\u000c\u0000\u0000,"Value cannot be null. (Parameter 'source')")
Bicep.Core.IntegrationTests.DecompilationTests ‑ Decompiler_handles_banned_function_replacement ("createArray(1, 2, 3)","array","[
  1
  2
  3
]")
Bicep.Core.IntegrationTests.DecompilationTests ‑ Decompiler_handles_banned_function_replacement ("createObject('key', 'value')","object","{
  key: 'value'
}")
Bicep.Core.IntegrationTests.DecompilationTests ‑ Decompiler_handles_strings_with_newlines ("
","\n")
Bicep.Core.IntegrationTests.DecompilationTests ‑ Decompiler_handles_strings_with_newlines ("
","\r\n")
…

@jeskew jeskew merged commit 1f661a2 into main Mar 24, 2025
47 checks passed
@jeskew jeskew deleted the jeskew/16676 branch March 24, 2025 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants