Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates "include-publish-npm-package-deployment" pipeline deployment job to be 1ES compliant #23913

Open
wants to merge 60 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
087a1e5
moves checkout step to input section of template context
seanimam Feb 21, 2025
e8a9689
remove commented section
seanimam Feb 21, 2025
b123c16
Initial upgrade to 1es release deployment job & moves checkout step t…
seanimam Feb 21, 2025
f78198f
Moves all artifact downloads to templateContext
seanimam Feb 21, 2025
66dc389
adjustments to pipeline artifact download locations
seanimam Feb 21, 2025
0b95297
removes whitesapce
seanimam Feb 21, 2025
86d94e5
switching destinationPath to targetPath syntax for inputs
seanimam Feb 21, 2025
1a7a54e
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
seanimam Feb 21, 2025
154db47
Updates copy step file selection and changes prerequisite job name
seanimam Feb 21, 2025
eef259e
adds comments and small cleanup
seanimam Feb 24, 2025
3a84481
Adjusts pipeline to install existing build tools pack
seanimam Feb 25, 2025
6fe99d3
attempt to fix artifact download issue with 'pipeline' shorthand usage
seanimam Feb 25, 2025
b529653
attempt 2 to fix artifact download issue with 'pipeline' shorthand usage
seanimam Feb 25, 2025
47fa883
attempt 3 - change pipeline variable to definition
seanimam Feb 25, 2025
f6cd7c3
syntax fix
seanimam Feb 25, 2025
b5f1792
Merge branch 'main' of https://github.com/microsoft/FluidFramework in…
seanimam Feb 25, 2025
01b01a3
commenting out test steps to speed up pipeline change tests
seanimam Feb 25, 2025
f140cb2
comment out test syntax fix
seanimam Feb 25, 2025
71df921
uncomment taskTest params
seanimam Feb 25, 2025
24b4e39
removing test steps to speed up pipeline testing and chaning branch r…
seanimam Feb 25, 2025
33b81a4
remove dependsOn
seanimam Feb 25, 2025
a0b54e4
swapping input download to alternate format
seanimam Feb 25, 2025
61fbf85
1 more change to input artifact syntax
seanimam Feb 25, 2025
74da248
injecting artifact download step early to confirm it works
seanimam Feb 26, 2025
501b9ea
Adds build build tools as resource to build client packages
seanimam Feb 26, 2025
f5f02f3
attempt to list auto downloaded files from pipeline resource
seanimam Feb 26, 2025
351ff8b
Just try to find consumed resources in workspace
seanimam Feb 26, 2025
b69fe19
adds test deployment job to see if consumed pipeline variables are av…
seanimam Feb 27, 2025
9b10631
makes deployment job name one word
seanimam Feb 27, 2025
c11ec4d
attempt to install build tools from tarball
seanimam Feb 27, 2025
90a5314
fixes syntax issue
seanimam Feb 27, 2025
f56f05f
updated step to list contents of buildTools-resoruce folder
seanimam Feb 27, 2025
2aea7ad
Adding last change to build package pipeline
seanimam Feb 27, 2025
b8758fe
environemnt update for test deployment job
seanimam Feb 27, 2025
09b524d
revert to env name
seanimam Feb 27, 2025
050e4f4
minor adjustments to folder paths
seanimam Feb 27, 2025
4c7aeec
adjust tarball install to install all tarballs at the same time
seanimam Feb 27, 2025
bbae646
Moves new logic to publish npm package deployment and removes changes…
seanimam Feb 28, 2025
9eb0ee8
removes pipeline resource added to build-npm-client-package
seanimam Feb 28, 2025
a58fd29
moves around comment location
seanimam Feb 28, 2025
970d7a6
Adds build tools pipeline as a resource to build npm client package a…
seanimam Feb 28, 2025
3a5ded5
attempt to resolve Unexpected value 'resources' error
seanimam Feb 28, 2025
89d712e
Removes resource section from include-publish-npm-package-deployment
seanimam Feb 28, 2025
ab5a2ce
Moves resource section to include publish npm exclusively
seanimam Mar 10, 2025
4bcd488
Revert "Moves resource section to include publish npm exclusively"
seanimam Mar 10, 2025
bce2190
comments out build stage to figure out if template stage deployment j…
seanimam Mar 10, 2025
25bd471
moves publish stage to top of build npm client package for testing
seanimam Mar 10, 2025
db4cdec
more adjustmnets to get testing to go faster
seanimam Mar 10, 2025
8fa5a92
comments out build stage
seanimam Mar 10, 2025
0f89faa
comments out another stage for testing
seanimam Mar 10, 2025
1ddfbbf
more stage comment outs
seanimam Mar 10, 2025
0062563
comment out pack input to publish job
seanimam Mar 10, 2025
6776d10
Revert "comment out pack input to publish job"
seanimam Mar 10, 2025
09985c4
Revert "more stage comment outs"
seanimam Mar 10, 2025
c37ec94
Revert "comments out another stage for testing"
seanimam Mar 10, 2025
fb88dad
Revert "comments out build stage"
seanimam Mar 10, 2025
ff32355
Attempt to use a test tempalted deployment job without extensive buil…
seanimam Mar 10, 2025
d956161
removes parameters from test deployment template job
seanimam Mar 10, 2025
1a20fcc
Removes all paramters from test deployment template except pool
seanimam Mar 10, 2025
21451ca
adds environemnt paramter to test deployment template
seanimam Mar 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions tools/pipelines/build-client.yml
Original file line number Diff line number Diff line change
Expand Up @@ -207,12 +207,12 @@ extends:
# to the "pack" stage since it can use the already-packed packages in that case. As it is the pipeline packs some
# packages twice.
- { name: "ci:test:jest", jobName: "JestTest" }
- { name: "ci:test:realsvc:tinylicious", jobName: "RealsvcTinyliciousTest" }
- { name: "ci:test:stress:tinylicious", jobName: "StressTinyliciousTest" }
- { name: "check:are-the-types-wrong", jobName: "AreTheTypesWrong" }
# - { name: "ci:test:realsvc:tinylicious", jobName: "RealsvcTinyliciousTest" }
# - { name: "ci:test:stress:tinylicious", jobName: "StressTinyliciousTest" }
# - { name: "check:are-the-types-wrong", jobName: "AreTheTypesWrong" }
coverageTests:
- { name: "ci:test:mocha", jobName: "MochaTest" }
- { name: "ci:test:realsvc:local", jobName: "RealsvcLocalTest" }
# - { name: "ci:test:realsvc:local", jobName: "RealsvcLocalTest" }
testResultDirs:
- nyc/examples
- nyc/experimental
Expand Down
9 changes: 9 additions & 0 deletions tools/pipelines/templates/build-npm-client-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,12 @@ resources:
type: git
name: 1ESPipelineTemplates/M365GPT
ref: refs/tags/release
pipelines:
# Listing this pipeline as a resource makes its artifacts automatically downloaded within a deployment job.
- pipeline: buildTools-resource
project: internal
source: Build - build-tools
branch: main

extends:
# The pipeline extends the 1ES pipeline template which will inject different SDL and compliance tasks.
Expand Down Expand Up @@ -187,6 +193,9 @@ extends:
# Install all dependencies, not just the root ones
dependencyInstallCommand: pnpm install --frozen-lockfile

# Test deployment job
- template: /tools/pipelines/templates/test-template-deployment-job.yml@self

# Install / Build / Test Stage
- stage: build
displayName: Build Stage
Expand Down
6 changes: 6 additions & 0 deletions tools/pipelines/templates/build-npm-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,12 @@ resources:
type: git
name: 1ESPipelineTemplates/M365GPT
ref: refs/tags/release
pipelines:
# Listing this pipeline as a resource makes its artifacts automatically downloaded within a deployment job.
- pipeline: buildTools-resource
project: internal
source: Build - build-tools
branch: main

extends:
# The pipeline extends the 1ES pipeline template which will inject different SDL and compliance tasks.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,23 @@ parameters:

jobs:
- deployment: publish_${{ replace(parameters.environment, '-', '_') }}
# Depend on the build job
# dependsOn: checkoutAndUploadThisRepoAsArtifact
displayName: Publish ${{ parameters.environment }}
pool: ${{ parameters.pool }}
environment: ${{ parameters.environment }}
# This templateContext section is necessary for 1ES compliance for deployment jobs.
# see https://eng.ms/docs/cloud-ai-platform/devdiv/one-engineering-system-1es/1es-docs/1es-pipeline-templates/features/releasepipelines/releaseworkflows/releasejob?tabs=combined-pipeline for more info.
templateContext:
type: releaseJob
isProduction: true
# Inputs here are automatically downloaded at the beginning of the job.
inputs:
- input: pipelineArtifact
artifactName: pack
buildType: current
targetPath: $(Pipeline.Workspace)/pack

workspace:
clean: all
variables:
Expand All @@ -55,24 +69,45 @@ jobs:
runOnce:
deploy:
steps:
- checkout: self
clean: true
persistCredentials: true # Necessary for creation of git tags to work
- download: current
artifact: pack
- template: /tools/pipelines/templates/include-use-node-version.yml@self
- template: /tools/pipelines/templates/include-install-build-tools.yml@self
parameters:
buildDirectory: ${{ parameters.buildDirectory }}
buildToolsVersionToInstall: ${{ parameters.buildToolsVersionToInstall }}
pnpmStorePath: ${{ parameters.pnpmStorePath }}
- template: /tools/pipelines/templates/include-publish-npm-package-steps.yml@self
parameters:
artifactPath: tarballs
feedUrl: ${{ parameters.feedUrl }}
customEndPoint: ${{ parameters.customEndPoint }}
feedKind: ${{ parameters.feedKind }}
publishFlags: ${{ parameters.publishFlags }}
- template: /tools/pipelines/templates/include-git-tag-steps.yml@self
parameters:
tagName: ${{ parameters.tagName }}

- task: Bash@3
name: listFilesInWorkspace
displayName: List files in worksspace
inputs:
targetType: 'inline'
workingDirectory: '$(Pipeline.Workspace)'
script: |
echo "Listing files in directory: $(pwd)"
ls -la

- task: Bash@3
name: InstallBuildToolsFromTarball
displayName: Install Fluid Build Tools from Artifact Tarball
inputs:
targetType: 'inline'
workingDirectory: '$(Pipeline.Workspace)/buildTools-resource/pack/tarballs'
script: |
echo "Listing files in directory: $(pwd)"
ls -la
echo "attempting tarball install"
npm i -g ./*.tgz

# - template: /tools/pipelines/templates/include-publish-npm-package-steps.yml@self
# parameters:
# artifactPath: tarballs
# feedUrl: ${{ parameters.feedUrl }}
# customEndPoint: ${{ parameters.customEndPoint }}
# feedKind: ${{ parameters.feedKind }}
# publishFlags: ${{ parameters.publishFlags }}

# - job: gitTagForBuild
# displayName: Push Git Tag for Build
# dependsOn: publish_${{ replace(parameters.environment, '-', '_') }}
# steps:
# - checkout: self
# clean: true
# persistCredentials: true # Necessary for creation of git tags to work
# - template: /tools/pipelines/templates/include-git-tag-steps.yml@self
# parameters:
# tagName: ${{ parameters.tagName }}
64 changes: 64 additions & 0 deletions tools/pipelines/templates/test-template-deployment-job.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Copyright (c) Microsoft Corporation and contributors. All rights reserved.
# Licensed under the MIT License.

# include-publish-npm-package-deployment

parameters:
- name: pool
type: object
default: Small-eastus2
- name: environment
type: string
default: test-package-build-feed

jobs:
- deployment: publish_${{ replace(parameters.environment, '-', '_') }}
# Depend on the build job
# dependsOn: checkoutAndUploadThisRepoAsArtifact
displayName: Publish ${{ parameters.environment }}
pool: ${{ parameters.pool }}
environment: ${{ parameters.environment }}
# This templateContext section is necessary for 1ES compliance for deployment jobs.
# see https://eng.ms/docs/cloud-ai-platform/devdiv/one-engineering-system-1es/1es-docs/1es-pipeline-templates/features/releasepipelines/releaseworkflows/releasejob?tabs=combined-pipeline for more info.
templateContext:
type: releaseJob
isProduction: true
# Inputs here are automatically downloaded at the beginning of the job.
# inputs:
# - input: pipelineArtifact
# artifactName: pack
# buildType: current
# targetPath: $(Pipeline.Workspace)/pack

workspace:
clean: all
variables:
version: $[ stageDependencies.build.build.outputs['SetVersion.version']]
isLatest: $[ stageDependencies.build.build.outputs['SetVersion.isLatest']]
strategy:
runOnce:
deploy:
steps:
- template: /tools/pipelines/templates/include-use-node-version.yml@self

- task: Bash@3
name: listFilesInWorkspace
displayName: List files in worksspace
inputs:
targetType: 'inline'
workingDirectory: '$(Pipeline.Workspace)'
script: |
echo "Listing files in directory: $(pwd)"
ls -la

- task: Bash@3
name: InstallBuildToolsFromTarball
displayName: Install Fluid Build Tools from Artifact Tarball
inputs:
targetType: 'inline'
workingDirectory: '$(Pipeline.Workspace)/buildTools-resource/pack/tarballs'
script: |
echo "Listing files in directory: $(pwd)"
ls -la
echo "attempting tarball install"
npm i -g ./*.tgz