AZURE DATABRICKS - How To Modify existing Secret Scope?

-



I have been working on Azure Databricks (DBx) for 3/4 weeks and there are loads of things that I have been fortunate enough to learn. I always wanted to explore or find out more about Azure DBx and just grateful enough to be given a chance to use it in a real project. 

Let's get to the technicalities then. 

The other day I stumbled upon this error while running a DBx notebook 

User username@domain.com does not have READ permission on scope xxx_yyy to perform this action.

So the problem statement was to find out the details of scope xxx_yyy. 

I assume if you are reading this article you will have an idea of what secret scopes are in context to Azure Databricks. 

You can read more on it here Secret Scopes - Azure DBx

Now I will list out the steps to create and manage secret scopes. Creating can be done through UI on azure portal but for managing the scope the process is little different. Please go through the following steps

CREATE

1. For creating scopes using key vault open the following web page
<yourdatabricksurl>#secrets/createScope

2. Refer to the step detailed in this article

MANAGE/MODIFY

For managing scopes, we'll have to use CLIs. It cant be done using portal

1. Make sure python is installed and the path is copied to be used in subsequent step. 
The path will look like this -      
C:\Users\Arnold Commando\AppData\Local\Programs\Python\Python39

 

2. Open command prompt and make sure you are under this path
cd C:\Users\Arnold Commando\AppData\Local\Programs\Python\Python39

3. Navigate to https://bootstrap.pypa.io/get-pip.py and copy the code in notepad file. Give it a name get-pip.py and make sure it is saved under  
C:\Users\Arnold Commando\AppData\Local\Programs\Python\Python39

4. run this command
python get-pip.py

5. For steps 3 and 4, you can refer to this article

6. Install data bricks CLI by using below command
pip install databricks-cli

7. Downgrade the cli version by using below command
python -m pip install --upgrade pip setuptools wheel databricks-cli==0.11.0

8. run the following command
databricks configure --token
Make sure you are inside scripts folder
cd C:\Users\Arnold Commando\AppData\Local\Programs\Python\Python39\Scripts

9. The Interface will ask for "Databricks Host (should begin with https://):"
[What I entered]
https://eastus.azuredatabricks.net

10. Interface will now ask for 
"Token:"
Generate token from databricks workspace and paste here.

11. Run the following command to see if the authentication was succesfully 
databricks fs ls

12. It should return, something like
ml
mnt
tmp

13. Run the following command to check the scope details
databricks secrets list-acls --xxx_yyy <secret scope name>

14. Modify the permissions by using below command
databricks secrets put-acl --scope <secret scope name>  --principal <username@domain.com> --permission READ


So here it is. Hope this was helpful.  

Comments

Post a Comment

Popular posts from this blog

SQL QUERY NIGHTMARE

-
Image
Last week, we faced a basic and yet the only major problem that sql developers/admin had to deal with and yes your guess is right, It was with SQL QUERY RUNNING SLOW.  I will be explaining the problem below, however I want everyone reading this to help me find answers because I still haven't found one! One more amazing aspect of the issue was that it worked fairly quickly in Development but was running like a tortoise in Production even though the number of records were practically the same. I'll start by throwing some light on to the execution plan, so that we narrow down to the code which was the culprit here. The code is enclosed into a stored proc. On contacting my dba to enquire about the issue, I got the following replay:- I must hand it to our dba to provide such a detailed response.  So my job was now to look at the code and modify it. What I did was to shrink the calender table into a temp table consisting of few rows that would speed up the operation. Be...
Read more

Visual Studio Git Error | "terminal prompts disabled"

-
Image
I was trying to sync our team's database project and was getting the below error git: 'credential-manager-core' is not a git command could not read Username for 'https://github.com': terminal prompts disabled One thing to call out here is that I had not synced the changes in the database project for over a month and there were no issues before.  My guess is that somehow the OWA authentication got wiped out from my system and this might be the reason for this issue.  In terms of the Solution I tried searching online on  terminal prompts disabled and got a plethora of solutions mostly suggesting some git commands. I tried a few of the suggestions and didn't get any luck. This morning I started searching for  git: 'credential-manager-core' is not a git command   and came across the following issue reported on Microsoft forums  Warning/error message when running git fetch within Visual Studio 2019 It basically suggests downloading/updating the latest versi...
Read more

Issues Integrating Azure Data Factory with GITHUB | IN spite of admin rights on repository

-
Image
Although I have been aware of GitHub for quite a while but got a chance to use it as a repository in the project for storing artifacts only over the past few months.  We use GitHub as the sole cloud repo for storing SQL, Databricks, and Azure Data Factory artifacts.  Got an opportunity to set up GitHub with ADF right from scratch and all the excitement to learn more about GitHub took a back seat when I stumbled upon the following issue Problem Statement - You are an admin on a repo in GitHub and even then get a permission error while linking the ADF to GitHub Solution - After scratching my hairs and going on a searching spree on google for some days, I was able to figure out what needed to be done, however was not aware of how to implement it. Finally thanks to Microsoft for getting on a call and helping me resolve this issue. What needed to be done - Need to authorize AzureDataFactory service so that Github can communicate with it.  How it should be done -  Basicall...
Read more