Build your first Microsoft Graph Data Connect application

Microsoft Graph Data Connect (Data Connect) augments transactional APIs with an intelligent way to access rich data at scale. Data Connect is ideal for big data applications and machine learning as it allows you to develop applications for analytics, intelligence, and business process optimization by extending Microsoft 365 data into Microsoft Azure. Through this offering, you'll be able to take advantage of the vast suite of compute and storage in Azure while staying compliant with industry standards and keeping your data secure.

Data Connect uses Microsoft Fabric, Azure Synapse, or Azure Data Factory to copy Microsoft 365 data to your application's storage at configurable intervals. It also provides a set of tools to streamline the delivery of this data to Azure, letting you access the most applicable development and hosting tools available.

Data Connect also grants a more granular control and consent model: you can manage data, see who is accessing it, and request specific properties of an entity. This enhances the Microsoft Graph model, which grants or denies applications access to entire entities.

Additionally, you can use Data Connect to enable machine learning scenarios for your organization. In these scenarios, you can create applications that provide valuable information to your stakeholders, train machine learning models, and even perform forecasting based on large amounts of acquired data.

Get started

In this tutorial, you'll create your first single-tenant Data Connect application. The following general flow explains the Data Connect onboarding process.

1. Admin enables Data Connect: The first step in onboarding is for your global administrator to enable Data Connect.

2. Developer creates a new Microsoft Entra application: The developer needs to first create a new Microsoft Entra application.

3. Developer registers the application with Data Connect: Once the Microsoft Entra application is created, the developer needs to register the application with Data Connect using the new registration portal for Microsoft Graph Data Connect applications. In this step, the developer specifies what data they require for their application.

4. Admin approves the application: After the developer has registered their application with Data Connect, the global administrator can use the new portal for Data Connect application consent to review the registered application and approve it.

5. Developer runs their pipelines: After the administrator has consented to the application, the developer might run their pipelines without any stalling for runtime consent. The pipeline creation and run via Azure Data Factory or Azure Synapse remains the same.

Prerequisites

To complete this tutorial, you need the following subscriptions or licenses.

• For this tutorial, we strongly recommend that you use a Microsoft 365 developer tenant.
• Your Microsoft 365 and Azure tenants must be in the same Microsoft Entra tenancy.
• The Azure subscription must be in the same tenant as the Microsoft 365 tenant.
• One user in your Microsoft 365 tenant has the Global Administrator role enabled. Going forward, this tutorial refers to this user as the "admin". Only an "admin" user can approve the test application.
• A different user in your Microsoft 365 tenant with Application Administrator or Application Developer role. Going forward, this tutorial refers to this user as the "developer". The developer user does the majority of this tutorial.

1. Microsoft 365 tenancy

   • If you don't have a Microsoft 365 tenant, you might qualify for one through the Microsoft 365 Developer Program; for details, see the FAQ. Alternatively, you can sign up for a 1-month free trial or purchase a Microsoft 365 plan.

   Note

   You can't approve your own test application using the same account. Make sure that you have another member (or account) in your tenant that acts as an admin.

2. Microsoft Azure subscription

   • If you don't have a Microsoft Azure subscription, visit the following link to get one (for free): Create your Azure Free Account.
   • Your Azure subscription must be in the same tenant as your Microsoft 365 tenant and both must be in the same Microsoft Entra tenancy.
   • If your Azure subscription isn't in the same tenant as your Microsoft 365 tenant, you can associate your subscription with Microsoft Entra ID in your Microsoft 365 tenant by following the steps listed in Associate or add an Azure subscription to your Microsoft Entra tenant.

   Note

   The screenshots and examples used in this tutorial are from a Microsoft 365 developer tenant with a sample email from test users. You can use your own Microsoft 365 tenant to perform the same steps. No data is written to Microsoft 365. A copy of email data is extracted from all users in a Microsoft 365 tenant and copied to an Azure Blob Storage account. You maintain control over who has access to the data within the Azure Blob Storage.

Set up your Microsoft 365 tenant and enable Microsoft Graph Data Connect

Before you use Microsoft Graph Data Connect (Data Connect) for the first time, please work with your Microsoft 365 tenant admin to enable the Data Connect service for your tenant.

For this tutorial, please ensure that you have an account with admin privileges available. You must complete this step to set up your first pipeline. We recommend having at least two users in your Microsoft 365 tenant with the global administrator role enabled.

• Global Administrator built-in role.
• Elevate access to gain the Global Administrator role.

Enable Microsoft Graph Data Connect in your Microsoft 365 tenant

Set up your Microsoft 365 tenant to enable the usage of Microsoft Graph Data Connect.

1. Open a browser, go to your Microsoft 365 admin portal, and sign in with your admin user.

2. Select Settings > Org settings. You might have to choose Show all before you're able to view Settings.

3. In the Services tab (which should be the default selection), select Microsoft Graph Data Connect.

4. Select the checkbox to Turn Microsoft Graph Data Connect on or off for your entire organization and choose Save.

   

Congratulations, you just enabled Microsoft Graph Data Connect for your organization!

Set up your Microsoft Entra application

In this exercise, you learn how to create a Microsoft Entra application. This serves as the security principal to run the data extraction process with Microsoft Graph Data Connect.

1. Sign in to the Microsoft Entra admin center with at least the Application Administrator orApplication Developer role.

2. Expand the Identity menu > select Applications > App registrations > New registration.

3. Use the following values to create a new Microsoft Entra application and select Register.

   • Name: Microsoft Graph Data Connect Data Transfer (provide the name of your choice).
   • Supported account types: Accounts in this organizational directory only.
   • Redirect URI: Leave the default values.

   

4. Locate the Application (client) ID and copy it as you need it later in this tutorial. Going forward, this is referred to as the service principal ID.

5. Locate the Directory (tenant) ID and copy it as you need it later in this tutorial. Going forward, this is referred to as the tenant ID.

   

6. On the left navigation pane, select Certificates & secrets under Manage.

7. Select the New client secret button. Set Description to any name, set Expires to any value in the dropdown, and choose Add.

   

   • After the client secret is created, make sure you save the Value somewhere safe, as it will no longer be available later; otherwise, you'll need to create a new one.
   • Going forward, the client secret is referred as the service principal key.

8. On the left navigation pane for the application, select Owners.

9. Ensure that your account (if you're using a developer tenant use the Global admin account) meets the following requirements to be an application owner:

   • Your user account corresponds to a valid user in the tenant who isn't a service principal name.
   • Your account must have an Exchange Online license assigned (must have a mailbox).
   • Alongside the Exchange Online license, your account must also have a working E5 subscription/license.

10. Verify that your account is listed as an application owner. If that isn't the case, add it to the list.

    

Set up your Azure Storage resource

In this step you will create an Azure Storage account where Microsoft Graph Data Connect will store the data extracted from Microsoft 365 for further processing.

1. Open a browser and go to your Azure portal.

2. Sign in using an account with an Application developer role to your Azure portal.

3. On the sidebar navigation, select Create a resource.

4. Find the Storage account resource type and use the following values to configure it. For Performance, Redundancy, and the Advanced tab, the values for these fields are an example.

   • Subscription: select your Azure subscription
   • Resource group: mgdc-app-resource (or select an existing resource group)
   • Storage account name: mgdcdemoap (or you can name and select your own storage account)
   • Region: pick an Azure region in the same region as your Microsoft 365 region
   • Performance: Standard
   • Redundancy: Geo-redundant storage
   • Advanced tab:
     • Access tier: Hot

   

5. After configuring the values for the Basics and Advanced tabs, leave the rest of the settings as default.

   Note

   Ensure that within the Networking tab, the Enable public access from all networks is enabled. You can also select the second option Enable public access from selected virtual networks and IP addresses; however, additional steps outside of this tutorial need to be performed, which are highlighted in the IP addresses section.

   

6. Review that the settings match those shown in the previous steps and select Create to finalize.

7. After the Azure Storage account has been created, grant the Microsoft Entra application previously created the proper access to it.

   1. Select the Azure Storage account.
   2. On the sidebar menu, select Access control (IAM).
   3. Select the Add button in the Add a role assignment block.
   4. Grant the Storage Blob Data Contributor role to the application you previously created.
      1. Select Storage Blob Data Contributor as the Role, and click Next.
      2. Assign access to User, group or service principal.
      3. Click + Select members, and in the right pane that displays, search for the application (Microsoft Graph data connect Data Transfer) that you previously created, and click Select.
   5. Select Review + assign.

   

8. Create a new container in the mgdcdemoap Azure Storage account.

   1. Select the mgdcdemoapp Azure Storage account (or your account name from step 4).

   2. On the sidebar menu, select Containers under the Data storage service section.

   3. Select the +Container button at the top of the page and use the following values and then select Create.

      • Name: m365mails
      • Public access level: Private (no anonymous access)