Call Databricks API from PowerShell on Azure
Introduction
The objective here is to share some samples and tips on how to call Databricks API from PowerShell.
We will call the SCIM API that lets us manage Azure Active Directory Service Principals and Groups within Databricks.
- The API documentation for Databticks Service Principals is available here,
- the one for Databricks Groups is available here.
Prerequisite
Make sure you have already the following resources :
- An Azure Databricks resource.
- An Azure AD service principal with the Owner privilege on the Databricks resource (we will use it to assign Databricks privileges — I will name it: dbx-adm-spn1).
- An Azure AD service principal with no specific role assignment (we will assign it some Databricks privileges — I will name it : dbx-datascientist-spn1).
Authentication
We will need to gather the two following tokens:
- A token related to the Azure AD Enterprise application called AzureDatabricks.
- A token related to the Azure management portal.
We will set the authentication among this article but I’m not going to explain its mechanism in detail here, the authentication setup is actually very well described in the following article.
The related commands to get authenticated →
Get a Databricks Service Principal
Let’s check if the Service Principal that has been granted the Owner role is referenced in Databricks.
To do that we will need the two mentioned token that has been generated with our “Owner” Service Principal. Actually when an Identity is the owner of a Databricks resource it becomes automatically admin of Databricks.
The related commands →
This API’s documentation is available here.
Get a Databricks group
Databricks groups have specific ids within the workspace itself, as illustrated in the following screenshot our service principal belongs to the “Admins” groups that have the following id: “2130741292350679”.
When you want to add a user or a service principal into a Databricks group through API you need to know it’s group id.
Let’s get the Databricks groups of our workspace.
The related commands →
You can note in the “Header” that we now only need to use the token related to the Azure AD Enterprise application called AzureDatabricks, no need to reconnect to the Azure management portal.
Note one group’s id, we will use it in the next chapter.
This API’s documentation is available here.
Add a Databricks Service Principal
Let’s now add our Service Principal “dbx-datascientist-spn1” into Databricks and make it a member of a Databricks group called “datascientist”.
The related commands →
You can also re-use the API that Get Databricks Service Principals to check that the change has been done.
I hope you enjoyed this article and its tips for calling Databricks API from PowerShell.
See you in the Cloud,
Jamesdld
Follow us on Twitter 🐦 and Facebook 👥 and Instagram 📷 and join our Facebook and Linkedin Groups 💬.
To join our community Slack team chat 🗣️ read our weekly Faun topics 🗞️, and connect with the community 📣 click here⬇
