Version 2 of Azure Data Factory is the product that keeps getting better and better. With a whole range of features available currently which, arguably, places the product at a comparable feature parity to SQL Server Integration Services (SSIS), it is worth a look when you have a demanding data integration requirement. And, as we have seen on the blog previously, it is remarkably easy to incorporate any Data Factory development work as part of your existing DevOps processes. If you’re looking for an easy to set up, inexpensive means of fulfilling an Extract, Transform and Load (ETL) type scenario, I think you’d be hard pressed to find a similar service available today to meet your needs. Additionally, for those who are still wedded to any existing SSIS workloads, Data Factory has full support for hosting and running .dtsx packages within the cloud, with minimal alteration required.

Somewhat surprisingly, amongst the list of over 80 + Linked Service connectors available for Data Factory, there are three specifically available for Dynamics 365 Customer Engagement (AKA Dynamics CRM):

• Dynamics 365
• Dynamics CRM
• Common Data Service for Apps

When reviewing the entire list of connectors from the Microsoft Docs website and attempting to navigate to the relevant page for the above Linked Services, we are redirected to the same page for each connector. This fact would seem to indicate that there is no practical difference between each one, a suspicion that can be confirmed when reviewing the configurable properties for the Dynamics CRM and Dynamics 365 connectors within Data Factory:

Rather bizarrely as well, the Common Data Service for Apps connector has the same Deployment Type option, allowing you to indicate an on-premise environment. Perhaps I missed the memo on CDS V2 being released for on-premise too 🙂

So, at this stage, it does beg the question - does it matter which connector you use?

Typically, I would advise choosing the connector that suits your particular environment, which would, therefore, dictate the following:

• If you are running an on-premise version of Dynamics CRM 2016 or earlier, then go for the Dynamics CRM.
• If you are running an 8.2 or 9 version of Dynamics 365, either on-premise or online, then the Dynamics 365 connector is your best bet.
• For online-only instances running the latest version (9.1 at the time of writing), then Common Data Service for Apps makes the most logical sense as, unless you are already aware, this is basically what the underlying CRM database is now moving forward.

However, in this particular scenario, I’m not too sure if it matters at all. Here’s why: the next stage, once your appropriate Linked Service is ready, is to define the corresponding dataset that you wish to access. For databases, this will typically be the underlying table/view that you wish to expose, and the Dynamics/Common Data Service for Apps connectors are no different in this regard. So, for example, to access the Account entity, you would need to create a corresponding dataset for this. For all three cases, this will load a dataset with the referenceName value of Dynamics365LinkedService, as indicated in the below screenshot and (shortened) code example:

{
    "name": "D365_Account",
    "properties": {
        "linkedServiceName": {
            "referenceName": "Dynamics365LinkedService",
            "type": "LinkedServiceReference"
        },
        "folder": {
            "name": "D365"
        },
        "type": "DynamicsEntity",
        "structure": [
            {
                "name": "address2_addresstypecode",
                "type": "Int32"
            },
            {
                "name": "merged",
                "type": "Boolean"
            },
            {
                "name": "statecode",
                "type": "Int32"
            },
            {
                "name": "emailaddress1",
                "type": "String"
            }
			
			...
        ],
        "typeProperties": {
            "entityName": "account"
        }
    },
    "type": "Microsoft.DataFactory/factories/datasets"
}

When creating our Linked Service using any of the connectors listed above, our initial suspicions from the Microsoft Docs are confirmed - they are listed with a type value of Dynamics:

{
    "name": "LinkedService",
    "type": "Microsoft.DataFactory/factories/linkedservices",
    "properties": {
        "type": "Dynamics",
        "typeProperties": {
            "deploymentType": "Online",
            "serviceUri": "https://mycrminstance.crm11.dynamics.com",
            "authenticationType": "Office365",
            "username": "[email protected]",
            "encryptedCredential": "mynotactuallyencryptedcredential"
        }
    }
}

So the answer is pretty obvious at this stage:

Where does this leave us? Given the fact that it is the same connector under the hood and, due to the whole Common Data Service for Apps element, I would argue it makes sense to stick to using this connector moving forward, regardless of what version of Dynamics 365/CRM you are using. Although I haven’t been in a position to test this (answers on a postcard if you have!), I would assume that this very same connector will work just fine and dandy for on-premise, internet-facing deployments of Dynamics CRM, regardless of version. I would also predict that the other listed connectors for Dynamics CRM and Dynamics 365 will be removed eventually; if not for simplicity purposes, then certainly due to any end-of-lifecycle support for these applications.

In a way, any initial confusion that these multiple connectors present when first being evaluated does ultimately detract away from an important point - that Azure Data Factory can be straightforwardly implemented alongside Dynamics 365/CRM with minimum effort involved. This capability opens up a whole host of different use cases, enabling CRM developers to embrace a fully cloud-hosted data integration tool that can meet the requirements other products on the market today, such as Scribe Online and Kingswaysoft, have traditionally been relied upon to fulfil.