Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add New Sample: Power Automate Desktop push a file into a Foundry Palantir Dataset #29

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
SET Foundry_API_BASEURL TO $'''https://<Enter your base URL for your foundry instance including>'''
SET AuthTok TO $'''<Replace with your Authentication Token>'''
SET DataSetID TO $'''<Replace with your Dataset ID>'''
Display.SelectFileDialog.SelectFile Title: $'''Select file to upload''' IsTopMost: True CheckIfFileExists: True SelectedFile=> SelectedFile ButtonPressed=> ButtonPressed
# Now We need to Open the Transaction and set it to SNAPSHOT mode
# This command is just here for debugging. It can be disabled or removed, but it allows you to do a GET of the latest transaction (in case you want to append rather than SNAPSHOT). However, that isn't supported in the script
Web.InvokeWebService.InvokeWebService Url: $'''%Foundry_API_BASEURL%/foundry-catalog/api/catalog/datasets/%DataSetID%/transactions/master''' Method: Web.Method.Get Accept: $'''application/json''' ContentType: $'''application/json''' CustomHeaders: $'''authorization: Bearer %AuthTok%''' ConnectionTimeout: 30 FollowRedirection: True ClearCookies: False FailOnErrorStatus: False EncodeRequestBody: False UserAgent: $'''Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.21) Gecko/20100312 Firefox/3.6''' Encoding: Web.Encoding.AutoDetect AcceptUntrustedCertificates: False ResponseHeaders=> WebServiceResponseHeaders_GetTransaction Response=> WebServiceResponse_GetTransaction StatusCode=> StatusCode_GetTransaction
# Create a new Transaction and then pull that into a JSON object so we can get the transaction ID (rid)
Web.InvokeWebService.InvokeWebService Url: $'''%Foundry_API_BASEURL%/foundry-catalog/api/catalog/datasets/%DataSetID%/transactions''' Method: Web.Method.Post Accept: $'''application/json''' ContentType: $'''application/json''' CustomHeaders: $'''authorization: Bearer %AuthTok%''' RequestBody: $'''{\"branchId\": \"master\"}''' ConnectionTimeout: 30 FollowRedirection: True ClearCookies: False FailOnErrorStatus: False EncodeRequestBody: False UserAgent: $'''Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.21) Gecko/20100312 Firefox/3.6''' Encoding: Web.Encoding.AutoDetect AcceptUntrustedCertificates: False ResponseHeaders=> WebServiceResponseHeaders_GetTransaction Response=> WebServiceResponse_GetTransaction StatusCode=> StatusCode_GetTransaction
Variables.ConvertJsonToCustomObject Json: WebServiceResponse_GetTransaction CustomObject=> Json_Transaction
# Set the transaction to SNAPSHOT mode so we can upload a brand new file.
Web.InvokeWebService.InvokeWebService Url: $'''%Foundry_API_BASEURL%/foundry-catalog/api/catalog/datasets/%DataSetID%/transactions/%Json_Transaction.rid%''' Method: Web.Method.Post Accept: $'''application/json''' ContentType: $'''application/json''' CustomHeaders: $'''authorization: Bearer %AuthTok%''' RequestBody: $'''\"SNAPSHOT\"''' ConnectionTimeout: 60 FollowRedirection: True ClearCookies: False FailOnErrorStatus: False EncodeRequestBody: False UserAgent: $'''Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.21) Gecko/20100312 Firefox/3.6''' Encoding: Web.Encoding.AutoDetect AcceptUntrustedCertificates: False ResponseHeaders=> WebServiceResponseHeaders_OpenTransaction Response=> WebServiceResponse_OpenTransaction StatusCode=> StatusCode_OpenTransaction
# Now we are going to upload the file, commit the transaction, and apply the schema.
Scripting.RunDOSCommand.RunDOSCommand DOSCommandOrApplication: $'''curl --request POST --url %Foundry_API_BASEURL%/foundry-data-proxy/api/dataproxy/datasets/%DataSetID%/transactions/%Json_Transaction.rid% --header \"Authorization: Bearer %AuthTok%\" -F file=\"@%SelectedFile.FullName%\"''' WorkingDirectory: SelectedFile.Directory StandardOutput=> CommandOutput StandardError=> CommandErrorOutput ExitCode=> CommandExitCode
# Commit the transaction. If it comes back anything by 204, it is an error, however, 204 also will be returned if the file fails to upload. This is problematic and bad implementation of the API. However, if we get a hard fail, the next IF statement will see it and then we abort ther transaction we created. Note: If the API responds 204, but there isn't really a file, it'll get committed, but a file won't be found in the transaction.
Web.InvokeWebService.InvokeWebService Url: $'''%Foundry_API_BASEURL%/foundry-catalog/api/catalog/datasets/%DataSetID%/transactions/%Json_Transaction.rid%/commit''' Method: Web.Method.Post Accept: $'''application/json''' ContentType: $'''application/json''' CustomHeaders: $'''authorization: Bearer %AuthTok%
''' RequestBody: $'''{}''' ConnectionTimeout: 60 FollowRedirection: True ClearCookies: False FailOnErrorStatus: False EncodeRequestBody: False UserAgent: $'''Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.21) Gecko/20100312 Firefox/3.6''' Encoding: Web.Encoding.AutoDetect AcceptUntrustedCertificates: False ResponseHeaders=> WebServiceResponseHeaders_commit Response=> WebServiceResponse_commit StatusCode=> StatusCode_commit
IF StatusCode_commit <> 204 THEN
Web.InvokeWebService.InvokeWebService Url: $'''%Foundry_API_BASEURL%/foundry-catalog/api/catalog/datasets/%DataSetID%/transactions/%Json_Transaction.rid%/abort''' Method: Web.Method.Post Accept: $'''application/json''' ContentType: $'''application/json''' CustomHeaders: $'''authorization: Bearer %AuthTok%
''' ConnectionTimeout: 60 FollowRedirection: True ClearCookies: False FailOnErrorStatus: False EncodeRequestBody: True UserAgent: $'''Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.21) Gecko/20100312 Firefox/3.6''' Encoding: Web.Encoding.AutoDetect AcceptUntrustedCertificates: False ResponseHeaders=> WebServiceResponseHeaders_abort Response=> WebServiceResponse_abort StatusCode=> StatusCode_abort
END
# Tell Envision to Try to read the file and set the schema automatically.
Web.InvokeWebService.InvokeWebService Url: $'''%Foundry_API_BASEURL%/foundry-schema-inference/api/datasets/%DataSetID%/branches/master/schema''' Method: Web.Method.Post Accept: $'''application/json''' ContentType: $'''application/json''' CustomHeaders: $'''authorization: Bearer %AuthTok%''' RequestBody: $'''{}''' ConnectionTimeout: 30 FollowRedirection: True ClearCookies: False FailOnErrorStatus: False EncodeRequestBody: False UserAgent: $'''Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.21) Gecko/20100312 Firefox/3.6''' Encoding: Web.Encoding.AutoDetect AcceptUntrustedCertificates: False ResponseHeaders=> WebServiceResponseHeaders_SetSchema Response=> WebServiceResponse_SetScheme StatusCode=> StatusCode_SetSchema
28 changes: 28 additions & 0 deletions samples/PAD_Push_file_to_Palantir_Foundry/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Use Power Automate Desktop to push a file into an existing dataset in Palantir Foundry

## Summary

This sample project provides a template recipe that can be pasted into Power Automated Desktop and customized for pushing files into an existing dataset in your instance of Palantir Foundry.

## Applies to

* [Microsoft Power Automate Desktop](https://docs.microsoft.com/en-us/power-automate/desktop-flows/introduction)
* [Palantir Foundry](https://www.palantir.com/docs/foundry/)

## License Requirements

* RPA license not required.

## Author

| Solution | Author(s) |
| --- | --- |
| PAD_Push_file_to_Palantir_Foundry | [Scott Thompson](https://github.com/lordneeko) [LinkedIn](https://www.linkedin.com/in/kscottthompson/)

## How to Use

1. Open a new (empty) flow within Power Automate Desktop
2. Copy the text from this git repo into your clipboard
3. Select the Editor window and CTRL+V to paste the recipe into the window
4. Set the variables: Foundry_API_BASEURL, AuthTok, and DataSetID