Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{App Service} Skip failed tests to unblock CI #30289

Merged
merged 1 commit into from
Nov 8, 2024

Conversation

yanzhudd
Copy link
Contributor

@yanzhudd yanzhudd commented Nov 8, 2024

Related command

Description

Skip the tests temporarily to unblock CI:
image

Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change
[Component Name 2] az command b: Add some customer-facing feature


This checklist is used to make sure that common guidelines for a pull request are followed.

Copy link

azure-client-tools-bot-prd bot commented Nov 8, 2024

️✔️AzureCLI-FullTest
️✔️acr
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️acs
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.12
️✔️3.9
️✔️ams
️✔️latest
️✔️3.12
️✔️3.9
️✔️apim
️✔️latest
️✔️3.12
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.12
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️aro
️✔️latest
️✔️3.12
️✔️3.9
️✔️backup
️✔️latest
️✔️3.12
️✔️3.9
️✔️batch
️✔️latest
️✔️3.12
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.12
️✔️3.9
️✔️billing
️✔️latest
️✔️3.12
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.12
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.12
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️compute_recommender
️✔️latest
️✔️3.12
️✔️3.9
️✔️computefleet
️✔️latest
️✔️3.12
️✔️3.9
️✔️config
️✔️latest
️✔️3.12
️✔️3.9
️✔️configure
️✔️latest
️✔️3.12
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.12
️✔️3.9
️✔️container
️✔️latest
️✔️3.12
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.12
️✔️3.9
️✔️core
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.12
️✔️3.9
️✔️databoxedge
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️dls
️✔️latest
️✔️3.12
️✔️3.9
️✔️dms
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.12
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.12
️✔️3.9
️✔️find
️✔️latest
️✔️3.12
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.12
️✔️3.9
️✔️identity
️✔️latest
️✔️3.12
️✔️3.9
️✔️iot
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️keyvault
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️lab
️✔️latest
️✔️3.12
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️maps
️✔️latest
️✔️3.12
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.12
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.12
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.12
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.12
️✔️3.9
️✔️network
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.12
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.12
️✔️3.9
️✔️profile
️✔️latest
️✔️3.12
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.12
️✔️3.9
️✔️redis
️✔️latest
️✔️3.12
️✔️3.9
️✔️relay
️✔️latest
️✔️3.12
️✔️3.9
️✔️resource
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️role
️✔️latest
️✔️3.12
️✔️3.9
️✔️search
️✔️latest
️✔️3.12
️✔️3.9
️✔️security
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.12
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.12
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.12
️✔️3.9
️✔️sql
️✔️latest
️✔️3.12
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.12
️✔️3.9
️✔️storage
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.12
️✔️3.9
️✔️telemetry
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️util
️✔️latest
️✔️3.12
️✔️3.9
️✔️vm
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9

Copy link

Hi @yanzhudd,
Since the current milestone time is less than 7 days, this pr will be reviewed in the next milestone.

Copy link

azure-client-tools-bot-prd bot commented Nov 8, 2024

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Collaborator

yonzhan commented Nov 8, 2024

Thank you for your contribution! We will review the pull request and get back to you soon.

@yanzhudd
Copy link
Contributor Author

yanzhudd commented Nov 8, 2024

+@khkh-ms for awareness
Our CI is blocked by the failed tests mentioned in this PR, so we skip them temporarily.

@jiasli
Copy link
Member

jiasli commented Nov 8, 2024

Error detail:

https://dev.azure.com/azclitools/public/_build/results?buildId=204565&view=logs&jobId=3791f883-8843-5e94-fc79-c8ca993c0a42&j=3791f883-8843-5e94-fc79-c8ca993c0a42&t=fc099b28-42bc-5603-6ee8-d61b88ef47c8

=========================== short test summary info ============================
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_functionapp_commands.py::FunctionAppWithPlanE2ETest::test_functionapp_on_linux_app_service_powershell
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_functionapp_commands.py::FunctionAppWithPlanE2ETest::test_functionapp_on_linux_app_service_powershell_with_runtime_version
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_functionapp_commands.py::FunctionAppWithLinuxConsumptionPlanTest::test_functionapp_consumption_linux_powershell
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_functionapp_commands.py::FunctionAppOnWindowsWithRuntime::test_functionapp_windows_runtime_powershell
FAILED src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_functionapp_commands.py::FunctionAppConfigTest::test_functionapp_powershell_version
2024-11-08T06:55:20.7100954Z __________ FunctionAppConfigTest.test_functionapp_powershell_version ___________
2024-11-08T06:55:20.7101263Z self = <azure.cli.testsdk.base.ExecutionResult object at 0x7f64b399af60>
2024-11-08T06:55:20.7101554Z cli_ctx = <azure.cli.core.mock.DummyCli object at 0x7f64b52bd6a0>
2024-11-08T06:55:20.7102203Z command = 'functionapp create -g clitest.rg000001 -n powershellfunctionapp000003 -c francecentral -s clitest000002 --os-type Windows --functions-version 4 --runtime powershell --runtime-version 7.2'
2024-11-08T06:55:20.7102568Z expect_failure = False
2024-11-08T06:55:20.7102668Z 
2024-11-08T06:55:20.7102897Z     def _in_process_execute(self, cli_ctx, command, expect_failure=False):
2024-11-08T06:55:20.7103138Z         from io import StringIO
2024-11-08T06:55:20.7103397Z         from vcr.errors import CannotOverwriteExistingCassetteException
2024-11-08T06:55:20.7103618Z     
2024-11-08T06:55:20.7103890Z         if command.startswith('az '):
2024-11-08T06:55:20.7104098Z             command = command[3:]
2024-11-08T06:55:20.7104280Z     
2024-11-08T06:55:20.7104453Z         stdout_buf = StringIO()
2024-11-08T06:55:20.7104663Z         logging_buf = StringIO()
2024-11-08T06:55:20.7104855Z         try:
2024-11-08T06:55:20.7105090Z             # issue: stderr cannot be redirect in this form, as a result some failure information
2024-11-08T06:55:20.7105349Z             # is lost when command fails.
2024-11-08T06:55:20.7105611Z >           self.exit_code = cli_ctx.invoke(shlex.split(command), out_file=stdout_buf) or 0
2024-11-08T06:55:20.7105751Z 
2024-11-08T06:55:20.7106058Z src/azure-cli-testsdk/azure/cli/testsdk/base.py:302: 
2024-11-08T06:55:20.7106322Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-11-08T06:55:20.7106676Z env/lib/python3.12/site-packages/knack/cli.py:245: in invoke
2024-11-08T06:55:20.7106937Z     exit_code = self.exception_handler(ex)
2024-11-08T06:55:20.7107312Z src/azure-cli-core/azure/cli/core/__init__.py:129: in exception_handler
2024-11-08T06:55:20.7107576Z     return handle_exception(ex)
2024-11-08T06:55:20.7107817Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-11-08T06:55:20.7107926Z 
2024-11-08T06:55:20.7108328Z ex = ValidationError('powershell has reached EOL on 2024-11-08 and is no longer supported. ')
2024-11-08T06:55:20.7108738Z args = (), kwargs = {}
2024-11-08T06:55:20.7108814Z 
2024-11-08T06:55:20.7109187Z     def _handle_main_exception(ex, *args, **kwargs):  # pylint: disable=unused-argument
2024-11-08T06:55:20.7109494Z         if isinstance(ex, CannotOverwriteExistingCassetteException):
2024-11-08T06:55:20.7109795Z             # This exception usually caused by a no match HTTP request. This is a product error
2024-11-08T06:55:20.7110070Z             # that is caused by change of SDK invocation.
2024-11-08T06:55:20.7110278Z             raise ex
2024-11-08T06:55:20.7110444Z     
2024-11-08T06:55:20.7110633Z >       raise CliExecutionError(ex)
2024-11-08T06:55:20.7110962Z E       azure.cli.testsdk.exceptions.CliExecutionError: The CLI throws exception ValidationError during execution and fails the command.
2024-11-08T06:55:20.7111142Z 
2024-11-08T06:55:20.7111487Z src/azure-cli-testsdk/azure/cli/testsdk/patches.py:35: CliExecutionError
2024-11-08T06:55:20.7111613Z 
2024-11-08T06:55:20.7111843Z During handling of the above exception, another exception occurred:
2024-11-08T06:55:20.7111975Z 
2024-11-08T06:55:20.7112278Z self = <azure.cli.command_modules.appservice.tests.latest.test_functionapp_commands.FunctionAppConfigTest testMethod=test_functionapp_powershell_version>
2024-11-08T06:55:20.7112900Z resource_group = 'clitest.rg000001', storage_account = 'clitest000002'
2024-11-08T06:55:20.7113024Z 
2024-11-08T06:55:20.7113261Z     @ResourceGroupPreparer(location=WINDOWS_ASP_LOCATION_FUNCTIONAPP)
2024-11-08T06:55:20.7113509Z     @StorageAccountPreparer()
2024-11-08T06:55:20.7113787Z     def test_functionapp_powershell_version(self, resource_group, storage_account):
2024-11-08T06:55:20.7114077Z         functionapp_name = self.create_random_name(
2024-11-08T06:55:20.7114394Z             'powershellfunctionapp', 40)
2024-11-08T06:55:20.7114870Z >       self.cmd('functionapp create -g {} -n {} -c {} -s {} --os-type Windows --functions-version 4 --runtime powershell --runtime-version 7.2'
2024-11-08T06:55:20.7115253Z                  .format(resource_group, functionapp_name, WINDOWS_ASP_LOCATION_FUNCTIONAPP, storage_account)).assert_with_checks([
2024-11-08T06:55:20.7115660Z                      JMESPathCheck('state', 'Running'),
2024-11-08T06:55:20.7116005Z                      JMESPathCheck('name', functionapp_name),
2024-11-08T06:55:20.7116350Z                      JMESPathCheck('kind', 'functionapp'),
2024-11-08T06:55:20.7116745Z                      JMESPathCheck('hostNames[0]', functionapp_name + '.azurewebsites.net')])
2024-11-08T06:55:20.7116872Z 
2024-11-08T06:55:20.7117274Z src/azure-cli/azure/cli/command_modules/appservice/tests/latest/test_functionapp_commands.py:3006: 
2024-11-08T06:55:20.7117582Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-11-08T06:55:20.7117943Z src/azure-cli-testsdk/azure/cli/testsdk/base.py:176: in cmd
2024-11-08T06:55:20.7118251Z     return execute(self.cli_ctx, command, expect_failure=expect_failure).assert_with_checks(checks)
2024-11-08T06:55:20.7118667Z src/azure-cli-testsdk/azure/cli/testsdk/base.py:251: in __init__
2024-11-08T06:55:20.7118951Z     self._in_process_execute(cli_ctx, command, expect_failure=expect_failure)
2024-11-08T06:55:20.7119363Z src/azure-cli-testsdk/azure/cli/testsdk/base.py:314: in _in_process_execute
2024-11-08T06:55:20.7119620Z     raise ex.exception
2024-11-08T06:55:20.7119953Z env/lib/python3.12/site-packages/knack/cli.py:233: in invoke
2024-11-08T06:55:20.7120217Z     cmd_result = self.invocation.execute(args)
2024-11-08T06:55:20.7120598Z src/azure-cli-core/azure/cli/core/commands/__init__.py:666: in execute
2024-11-08T06:55:20.7120835Z     raise ex
2024-11-08T06:55:20.7121192Z src/azure-cli-core/azure/cli/core/commands/__init__.py:733: in _run_jobs_serially
2024-11-08T06:55:20.7121486Z     results.append(self._run_job(expanded_arg, cmd_copy))
2024-11-08T06:55:20.7121882Z src/azure-cli-core/azure/cli/core/commands/__init__.py:725: in _run_job
2024-11-08T06:55:20.7122140Z     return cmd_copy.exception_handler(ex)
2024-11-08T06:55:20.7122675Z src/azure-cli/azure/cli/command_modules/appservice/commands.py:46: in _ex_handler
2024-11-08T06:55:20.7122928Z     raise ex
2024-11-08T06:55:20.7123267Z src/azure-cli-core/azure/cli/core/commands/__init__.py:703: in _run_job
2024-11-08T06:55:20.7123532Z     result = cmd_copy(params)
2024-11-08T06:55:20.7123892Z src/azure-cli-core/azure/cli/core/commands/__init__.py:336: in __call__
2024-11-08T06:55:20.7124161Z     return self.handler(*args, **kwargs)
2024-11-08T06:55:20.7124553Z src/azure-cli-core/azure/cli/core/commands/command_operation.py:121: in handler
2024-11-08T06:55:20.7124826Z     return op(**command_args)
2024-11-08T06:55:20.7125216Z src/azure-cli/azure/cli/command_modules/appservice/custom.py:4996: in create_functionapp
2024-11-08T06:55:20.7125546Z     matched_runtime = runtime_helper.resolve("dotnet" if not runtime else runtime,
2024-11-08T06:55:20.7125974Z src/azure-cli/azure/cli/command_modules/appservice/custom.py:4421: in resolve
2024-11-08T06:55:20.7126237Z     self.validate_end_of_life_date(
2024-11-08T06:55:20.7126483Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-11-08T06:55:20.7126587Z 
2024-11-08T06:55:20.7126865Z self = <azure.cli.command_modules.appservice.custom._FunctionAppStackRuntimeHelper object at 0x7f64b3d91ac0>
2024-11-08T06:55:20.7127369Z runtime = 'powershell', version = '7.2'
2024-11-08T06:55:20.7127467Z 
2024-11-08T06:55:20.7127682Z     def validate_end_of_life_date(self, runtime, version):
2024-11-08T06:55:20.7127947Z         from dateutil.relativedelta import relativedelta
2024-11-08T06:55:20.7128212Z         today = datetime.datetime.now(datetime.timezone.utc)
2024-11-08T06:55:20.7128462Z         six_months = today + relativedelta(months=+6)
2024-11-08T06:55:20.7128737Z         runtimes_eol = [r for r in self.end_of_life_dates if runtime == r.name]
2024-11-08T06:55:20.7129042Z         matched_runtime_eol = next((r for r in runtimes_eol if r.version == version), None)
2024-11-08T06:55:20.7129304Z         if matched_runtime_eol:
2024-11-08T06:55:20.7129525Z             eol = matched_runtime_eol.eol
2024-11-08T06:55:20.7129775Z             runtime_deprecation_link = matched_runtime_eol.deprecation_link or ''
2024-11-08T06:55:20.7130003Z     
2024-11-08T06:55:20.7130185Z             if eol < today:
2024-11-08T06:55:20.7130583Z >               raise ValidationError('{} has reached EOL on {} and is no longer supported. {}'
2024-11-08T06:55:20.7130883Z                                       .format(runtime, eol.date(), runtime_deprecation_link))
2024-11-08T06:55:20.7131380Z E               azure.cli.core.azclierror.ValidationError: powershell has reached EOL on 2024-11-08 and is no longer supported.
2024-11-08T06:55:20.7131550Z 
2024-11-08T06:55:20.7131908Z src/azure-cli/azure/cli/command_modules/appservice/custom.py:4382: ValidationError

The error is raised from

if eol < today:
raise ValidationError('{} has reached EOL on {} and is no longer supported. {}'
.format(runtime, eol.date(), runtime_deprecation_link))

and the date is hard-coded in the YAML:

7.2","value":"7.2","stackSettings":{"windowsRuntimeSettings":{"runtimeVersion":"7.2","isDefault":true,"isPreview":false,"isHidden":false,"remoteDebuggingSupported":false,"appInsightsSettings":{"isSupported":true},"gitHubActionSettings":{"isSupported":true},"appSettingsDictionary":{"FUNCTIONS_WORKER_RUNTIME":"powershell"},"siteConfigPropertiesDictionary":{"use32BitWorkerProcess":true,"powerShellVersion":"7.2","netFrameworkVersion":"v6.0"},"supportedFunctionsExtensionVersions":["~4"],"supportedFunctionsExtensionVersionsInfo":[{"version":"~4","isDeprecated":false,"isDefault":true}],"endOfLifeDate":"2024-11-08T00:00:00Z"},"linuxRuntimeSettings":{"runtimeVersion":"PowerShell|7.2","isDefault":true,"isPreview":false,"isHidden":false,"remoteDebuggingSupported":false,"appInsightsSettings":{"isSupported":true},"gitHubActionSettings":{"isSupported":true},"appSettingsDictionary":{"FUNCTIONS_WORKER_RUNTIME":"powershell"},"siteConfigPropertiesDictionary":{"use32BitWorkerProcess":false,"linuxFxVersion":"PowerShell|7.2"},"supportedFunctionsExtensionVersions":["~4"],"supportedFunctionsExtensionVersionsInfo":[{"version":"~4","isDeprecated":false,"isDefault":true}],"endOfLifeDate":"2024-11-08T00:00:00Z"}}},{"displayText":"PowerShell

"endOfLifeDate":"2024-11-08T00:00:00Z"

So this check basically sets a time bomb and blows up the CI at exactly today.

@yanzhudd yanzhudd merged commit db505d1 into Azure:dev Nov 8, 2024
55 checks passed
@yanzhudd yanzhudd deleted the skip_appservice_tests branch November 8, 2024 08:31
@yonzhan yonzhan removed this from the December 2024 (2024-12-10) milestone Nov 28, 2024
@yonzhan yonzhan added this to the January 2025 (2025-01-14) milestone Nov 28, 2024
jiasli pushed a commit to jiasli/azure-cli that referenced this pull request Dec 18, 2024
jiasli added a commit that referenced this pull request Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants