How to Upload to Onedrive Using a Python Script

Jason

Posted on May 14, 2020

Automating files upload to Microsoft OneDrive. Unexpected challenges and a success story!

In the current data era we live in, a huge number of reports are being generated every minute. Uploading and distributing these reports is an inconvenience to say the to the lowest degree, and a security threat if sensitive/confidential data is included in these reports.

I am currently in accuse of approximately 200 analytics reports generated daily, fastened to emails and sent to the according audition at different departments. While this is a time consuming task, information technology was not a security business organization simply because these emails and reports were accessed on visitor approved devices in a closed and highly secured network.

At present with the pandemic and the stay in home orders. The chances of downloading these reports on unsecured machines became an urgent effect.

So I decided to take reward of Microsoft OneDrive as a central storage solution. Automating uploading these reports to a departmental folders and e-mail just a link to the file location to my audition, reducing the chances of someone downloading the files on their machine when Microsoft OneDrive encourages web previews.

This sounds like an easy task, correct? Specially if yous take worked with Python and APIs earlier. Include the requests library, configure your Client Id and Client underground, request an AccessToken/RefreshToken, and start rolling.

Well it turned out that working with Microsoft Graph is not that easy especially if you want to schedule your application to run in the groundwork with no human being interference.

While Microsoft has a practiced documentation, it was even so vague on many critical subjects:

• Authorization and what endpoint to use
• Authenticating using you Microsoft Account
• An running in the background (A Daemon app) requires high privileged admin access I don't have
• How to use delegate permission instead and still run your app in the background
• How to prepare up the header for resumable large files upload

After doing what every developer does from reading documentation to looking upwards a solution or thought on stack overflow, I wasn't able to find a solution using python especially for the resumable large files upload.

So I decided to publish my solution to help any beau developer who is currently in the aforementioned shoes I was in couple weeks ago!

For the consummate lawmaking please visit my GitHub

At that place are ii major parts to this tutorial:

1. Create, ready and configure the API on Azure Portal
2. Write the python script

Part 1 - Create, set up and configure the API on Azure Portal

Stride one: Register your awarding
Go to https://portal.azure.com/#home
Azure Active Directory -> App Registration -> New Registration

• Name your API
• Accounts in this organizational directory just (Unmarried Tenant)
• Redirect URI is non needed as our app running and authenticating in the background
• Click register

Once your new API is created, click on the API and Save the following ii data for the code afterwards:

• Application (client) ID
• Directory (tenant) ID

And then take hold of the OAuth ii.0 potency endpoint (v2)

We will use information technology during the potency script to become the URL link to permissions consent

Step 2: Configure the API permissions

API permissions → Add a permission → Microsoft APIs → Microsoft Graph → Delegated permissions → Select permissions Permissions needed "Sites.ReadWrite.All" and "Files.ReadWrite.All"

Stride 3: Expose the API

After adding the permissions in Step three, nosotros accept to expose the API and those permissions to the scope.

Then we need to add together the client ID and select the authorized scopes nosotros just added.

Expose an API → Add together a client awarding → Enter Client ID → select the Authorized scopes → click add awarding

Step 4: Edit the manifest (Very important to allow Implicit grant)

This is a very important step in the API set. Go to Manifest and fix the Oauth2IdToken and ImplicitFlow to true

Now that our API is all set and configured we can start writing some Python code!!!

Part 2- Write the python script

Our code is two different python scripts:

1- generateOneDriveAPIConsentURL-public.py

Script to generate the consent URL.This script is basically run once afterwards setting the permissions in the API setup to give the user'due south consent to these permissions.
This is the best solution to run the chief app in the background without using the app permissions (which pose high security take chances equally it is global loftier privileged and requires admin approval)
Sticking to delegate permission limit the permissions to the current user privileges and in near all cases, it doesn't crave admin approving.

import requests import json from requests_oauthlib import OAuth2Session from oauthlib.oauth2 import MobileApplicationClient  client_id = "xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx" scopes = ['Sites.ReadWrite.All','Files.ReadWrite.All'] auth_url = 'https://login.microsoftonline.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/oauth2/v2.0/authorize'  #OAuth2Session is an extension to requests.Session #used to create an authorization url using the requests.Session interface #MobileApplicationClient is used to get the Implicit Grant  oauth = OAuth2Session(client=MobileApplicationClient(client_id=client_id), scope=scopes) authorization_url, state = oauth.authorization_url(auth_url) consent_link = oauth.get(authorization_url) print(consent_link.url)

This script basically connect the Microsoft authorization V2. endpoint, send the Client ID and the scope of permission we are asking for then a URL will be generated and sent back in the concluding

c:/Users/jsnmtr/Code/onedrive/generateOneDriveAPIConsentURL-public.py https://login.microsoftonline.com/xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx/oauth2/v2.0/authorize?response_type=token&client_id=xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx&scope=Sites.ReadWrite.All+Files.ReadWrite.All&land=xxxxxxxxxxxxxxxxxxxxxxxx        

Open the link in your web browser and click to accept to accept the permissions requested

2- AutomatedOneDriveAPIUploadFiles-public.py

This is the master script, information technology creates a public client application using the MSAL library, asking a token on behalf of the user, gain admission to Microsoft Graph and apply the OneDrive API to upload files.

Basic flow:
-Importing libraries

import os import requests import json import msal        

-Configuration

CLIENT_ID = 'xxxxxxxx-xxxxxxx-xxxxxx-xxxxxxx-xxxxxxxxxx' TENANT_ID = 'xxxxxxxx-xxxxxxx-xxxxxx-xxxxxxx-xxxxxxxxxx' AUTHORITY_URL = 'https://login.microsoftonline.com/{}'.format(TENANT_ID) RESOURCE_URL = 'https://graph.microsoft.com/' API_VERSION = 'v1.0' USERNAME = 'xxxxxxxxx@xxxxxx.thirty' #Office365 user's business relationship username Password = 'xxxxxxxxxxxxxxx' SCOPES = ['Sites.ReadWrite.All','Files.ReadWrite.All'] # Add together other scopes/permissions as needed.        

-Create a public client application using the Microsoft Hallmark Library (MSAL)

#Creating a public client app, Aquire a access token for the user and gear up the header for API calls cognos_to_onedrive = msal.PublicClientApplication(CLIENT_ID, authority=AUTHORITY_URL)        

-Learn a token from Microsoft identity platform endpoint to access Microsoft Graph API

token = cognos_to_onedrive.acquire_token_by_username_password(USERNAME,Password,SCOPES)        

-Set up the request header with admission token

headers = {'Authorization': 'Bearer {}'.format(token['access_token'])}        

-Read all the file in source directory
and so nosotros loop into the directory, become the file path, file size and read the data in the file

            #Looping through the files inside the source directory for root, dirs, files in os.walk(cognos_reports_source):     for file_name in files:         file_path = bone.path.join(root,file_name)         file_size = os.stat(file_path).st_size         file_data = open(file_path, 'rb')        

-If the file is less than 4mb:

if file_size < 4100000:        

-perform a simple upload

#Perform elementary upload to the OneDrive API             r = requests.put(onedrive_destination+"/"+file_name+":/content", data=file_data, headers=headers)        

-If the file is larger than 4mb:
Create an upload session

upload_session = requests.post(onedrive_destination+"/"+file_name+":/createUploadSession", headers=headers).json()        

-Carve up the file into byte chunks

total_file_size = os.path.getsize(file_path) chunk_size = 327680 chunk_number = total_file_size//chunk_size chunk_leftover = total_file_size - chunk_size * chunk_number chunk_data = f.read(chunk_size) start_index = i*chunk_size end_index = start_index + chunk_size        

-Gear up the header to match the starting index and end alphabetize of the byte chunk

headers = {'Content-Length':'{}'.format(chunk_size),'Content-Range':'bytes {}-{}/{}'.format(start_index, end_index-ane, total_file_size)}        

-Upload chunks

chunk_data_upload = requests.put(upload_session['uploadUrl'], data=chunk_data, headers=headers)        

For the complete code please visit my GitHub

To me, anytime you achieve a goal, is a success story and by writing this code I achieved my goal to automate my reports upload process to OneDrive! SUCCESS!

---

Let me know your thoughts!

Find me on LinkedIn | Twitter | Dev.to | StackOverflow

whalensuren1981.blogspot.com

Source: https://dev.to/jsnmtr/automating-files-upload-to-microsoft-onedrive-unexpected-challenges-and-a-success-story-2ini