The importance of segregated deployment environments for any IT application cannot be understated. Even if this only comprises of a single test/User Acceptance Testing (UAT) environment, there are a plethora of benefits involved, which should easily outweigh any administrative or time effort involved:

  • They provide a safe “sandbox” for any functionality or developmental changes to be carried in sequence and verify the intended outcome.
  • They enable administrators to carry out the above within regular business hours, without risk of disruption to live data or processes.
  • For systems that experience frequent updates/upgrades, these types of environments allow for major releases to be tested in advance of a production environment upgrade.

When it comes Dynamics 365 Customer Engagement (D365CE), Microsoft more recently have realised the importance that dedicated test environments have for their online customers. That’s why, with the changes announced during the transition away from Dynamics CRM, a free Sandbox instance was granted with every subscription. The cost involved in provisioning these can start to add up significantly over time, so this change was and remains most welcome; mainly when it means that any excuse to carrying out the bullet point highlighted above in bold immediately gets thrown off the table.

Anyone who is presently working in the D365CE space will be intently aware of the impending forced upgrade to version 9 of the application that must take place within the next few months. Although this will doubtless cause problems for some organisations, it is understandable why Microsoft is enforcing this so stringently and – if managed correctly – allows for a more seamless update experience in the future, getting new features into the hands of customers much faster. This change can only be a good thing, as I have argued previously on the blog. As a consequence, many businesses may currently have a version 9 Sandbox instance within their Office 365 tenant which they are using to carry out the types of tests I have referenced already, which typically involves testing custom developed or ISV Managed/Unmanaged Solutions. One immediate issue you may find if you are working with solutions containing the Marketing List (list) entity is that your Solution suddenly stops importing successfully, with the following error messages:

The error message on the solution import is a little vague…

…and the brain starts hurting when we look at the underlying error text!

The problem relates to changes to the Marketing List entity in version 9 of the application, specifically the Entity property IsVisibleInMobileClient. This assertion is confirmed by comparing the Value and CanBeChanged properties of these settings in both an 8.2 and 9.0 instance, using the Metadata Browser tool included in the SDK:

Here’s the setting in Version 8.2…

…and how it appears now in Version 9.0

Microsoft has published a support article which lists the steps involved to get this resolved for every new solution file that you find yourself having to ship between 8.2 and 9.0 environments. Be careful when following the suggested workaround steps, as modifications to the Solution file should always be considered a last resort means of fixing issues and can end up causing you no end of hassle if applied incorrectly. There is also no way of fixing this issue at source as well, all thanks to the CanBeChanged property being set to false (this may be a possible route of resolution if you are having this same problem with another Entity that has this value set to true, such as a custom entity). Although Microsoft promises a fix for this issue, I wouldn’t necessarily expect that 8.2 environments will be specially patched to resolve this particular problem. Instead, I would take the fix to mean the forced upgrade of all 8.2 environments to version 9.0/9.1 within the next few months. Indeed, this would seem to be the most business-sensible decision rather than prioritising a specific fix for an issue that is only going to affect a small number of deployments.

Working with Dynamics CRM/Dynamics 365 Customer Engagement (CRM/D365CE) solution imports can often feel a lot like persuing a new diet or exercise regime; we start out with the best of intentions of how we want things to proceed, but then something comes up to kick the wheel off the wagon and we end up back at square one 🙂 Anything involving a change to an IT system can generally be laborious to implement, due to the dependencies involved, and things can invariably go wrong at any stage in the process. The important thing is to always keep a cool head, take things slowly and try not to overcomplicate things from the outset, as often the simplest or most obvious explanation for an issue is where all due attention should be focused towards.

In the case of CRM/D365CE, we have the ability to access full log information relating to a solution import – regardless of whether it has failed or succeeded. This log can prove to be incredibly useful in troubleshooting solution import failures. Available as an XML download, it can be opened within Excel to produce a very readable two tab spreadsheet containing the following information:

  • The Solution tab provides high-level information regarding the solution package, its publisher, the status of the import and any applicable error messages.
  • The Components tab lists every single attempted action that the solution attempted to execute against the target instance, providing a timestamp and any applicable error codes for each one.

The above document should always be your first port of call when a solution import fails, and it will almost certainly allow you to identify the root cause of the failure – as it did for me very recently.

An unmanaged solution import failed with the top-level error message Fields that are not valid were specified for the entity. Upon closer investigation within the import log, I was able to identify the affected component – a custom attribute on the Quote entity – and the specific error message generated – Attribute…has SourceType 0, but 1 was specified:

The reason why the error was being generated is that a field with the same logical name was present within the environment, something which – for clearly understandable reasons – is not allowed. In this particular scenario, we were doing some tidy up of an existing solution and replacing a calculated field with a new field, with a different data type, using the same attribute name. The correct step that should have been taken before the solution import was to delete the “old” field in the target environment, but this was accidentally not included in the release notes. After completing this and re-attempting the solution import, it completed successfully.

The likelihood of this error ever occurring in the first place should be remote, assuming that you are customising your system the right way (i.e. using Solution Publisher prefixes for all custom attributes/entities). In this occasion, the appropriate note as part of the release documentation for the solution would have prevented the issue from occurring in the first place. So, as long as you have implemented a sufficiently robust change management procedure, that includes full instructions that are required to be completed both before and after a solution import, you can avoid a similar situation when it comes to replacing entity attributes within your CRM/D365CE solution.

For those who are well versed in rolling out solution updates within Dynamics CRM/365 for Enterprise (CRM/D365E), the process will always have a certain familiarity to it, with a few surprises rolled in now and again. Often, the update will proceed as anticipated; sometimes, you may encounter bizarre issues. I can remember a particularly strange incident I had last year where a solution import would get to about 90-95% completion…and then the green progress bar would suddenly start rolling back to nothing. The import progress window would then hang with no further guidance or error message. To try and determine the root cause, we had to interrogate the importjob entity within the system, which ended up showing the import job progress stuck at 0.15846281% : / In the end, we had to escalate the issue to Microsoft for further investigation, but rest assured that if you have not yet witnessed your own curious solution import experience, it’s definitely in the post 🙂

Thankfully, if you are new to the whole “rolling out solution update” thing, you can be assured that the process is relatively straightforward, and mostly without issue. If you have been handed a set of solution import instructions for the first time, though, you may be wondering why something similar to the following step is included:

Go into the Data Management -> Duplicate Detection Rules page and click Publish on all Duplicate Detection Rules that have a Status Reason of Unpublished

Unfortunately, after importing a solution update, CRM/D365E will automatically unpublish all of your Duplicate Detection Rules automatically. You are therefore required to explicitly publish them again, lest you start to encounter a sudden increase in duplicate records and database storage within your system. The reason why this happens is both understandable and frustrating in equal measure. As outlined in the following MSDN article on the subject:

A duplicate rule condition specifies the name of a base attribute and the name of a matching attribute. For example, specify an account as a base entity and a contact as a matching entity to compare last names and addresses

As part of the above, explicit matchcodes are created for every record that the Duplicate Detection Rule is targeting, based on the current metadata of your CRM/D365E entities and attributes. Because your solution update can potentially alter significant aspects of this metadata, the system automatically unpublishes all Duplicate Detection Rules as a precaution.

The above is perhaps trivial in nature, as the actual process of re-publishing all Duplicate Detection Rules is somewhat negligible in effort terms. Where difficulties can arise is if someone innocently overlooks this part of the process or if your system has many different Duplicate Detection Rules, in a mixture of Unpublished/Published state. You would have to specifically make a note of which rules were Published before beginning your solution import so that you can ensure that the correct rules are published after the fact. I would have thought that after so many versions of the product, that something would be added to address this – for example, perhaps a checkbox at the start of the Solution Import Wizard that lets you specify whether all currently published rules should be reactivated after the import completes successfully.

If you find that the above is an annoyance that you can do without no longer, like with many things on the platform, there is a solution that can be deployed in code. The SDK exposes the PublishDuplicateRuleRequest class, which does exactly what it says on the tin – meaning that you can write a plugin that applies this functionality accordingly. The tricky bit comes in determining which Message (i.e. action) on the platform that you wish to run this against. CRM/D365E does not expose a SolutionUpdate or SolutionImport message that we can piggy-back onto, so we have to look at the PublishAll message instead – the action that is triggered when you press Publish All Customizations in the system. This is because this is generally the action you will always need to take when importing an (unmanaged) solution. As a result, we can write a plugin class that is triggered on the Post-Operation event of this entity to automatically publish all Unpublished Duplicate Detection Rules in the system!

The snippet below is adapted from the sample code provided by Microsoft, but has been tweaked as follows:

  • A QueryExpression is used as opposed to QueryByAttribute, since we need to query on two separate attributes and their values – statecode and statuscode. You also cannot return an easily accessible count on all results returned with QueryByAttribute. We will see why is useful in a few moments.
  • The code explicitly checks for if there are any Unpublished rules first before attempting to proceed further – no point in running code unnecessarily!
  • Instead of activating each rule one-by-one using an Execute request, all of the requests are collected together as part of an ExecuteMultipleRequest, given that we now know the performance benefits that this can have.
  • Tracing has been implemented in liberal amounts, to provide remote debugging from within CRM/D365E.

Here’s the code – just copy into an empty class file on your plugin project, modify the namespace to reflect the name of your project and you will be good to go!

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

using Microsoft.Xrm.Sdk;
using Microsoft.Xrm.Sdk.Query;
using Microsoft.Xrm.Sdk.Messages;
using Microsoft.Crm.Sdk.Messages;

namespace MyPlugin.Plugins
    public class PostPublishAll_PublishDuplicateDetectionRules : IPlugin 
        public void Execute(IServiceProvider serviceProvider)
            //Obtain the execution context from the service provider.

            IPluginExecutionContext context = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));

            //Get a reference to the Organization service.

            IOrganizationServiceFactory factory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
            IOrganizationService service = factory.CreateOrganizationService(context.UserId);

            //Extract the tracing service for use in debugging sandboxed plug-ins

            ITracingService tracing = (ITracingService)serviceProvider.GetService(typeof(ITracingService));

            tracing.Trace("Tracing implemented successfully!");

            if (context.MessageName == "PublishAll")

                PublishRules(service, tracing);

        private void PublishRules(IOrganizationService service, ITracingService tracing)

            EntityCollection rules = GetDuplicateDetectionRules(service);

            tracing.Trace("Obtained " + rules.TotalRecordCount.ToString() + " duplicate detection rules.");

            if (rules.TotalRecordCount >= 1)

                // Create an ExecuteMultipleRequest object.
                ExecuteMultipleRequest request = new ExecuteMultipleRequest()
                    // Assign settings that define execution behavior: don't continue on error, don't return responses. 
                    Settings = new ExecuteMultipleSettings()
                        ContinueOnError = false,
                        ReturnResponses = false
                    // Create an empty organization request collection.
                    Requests = new OrganizationRequestCollection()

                //Create a collection of PublishDuplicateRuleRequests, and execute them in one batch

                foreach(Entity entity in rules.Entities)


                    PublishDuplicateRuleRequest publishReq = new PublishDuplicateRuleRequest { DuplicateRuleId = entity.Id };




                tracing.Trace("Plugin execution cancelled, as there are no duplicate detection rules to publish.");

        private EntityCollection GetDuplicateDetectionRules(IOrganizationService service)

            QueryExpression qe = new QueryExpression("duplicaterule");

            qe.ColumnSet = new ColumnSet("duplicateruleid");

            ConditionExpression condition1 = new ConditionExpression();
            condition1.AttributeName = "statecode";
            condition1.Operator = ConditionOperator.Equal;

            ConditionExpression condition2 = new ConditionExpression();
            condition2.AttributeName = "statuscode";
            condition2.Operator = ConditionOperator.Equal;

            FilterExpression filter = new FilterExpression();
            filter.FilterOperator = LogicalOperator.And;


            //Have to add this, otherwise the record count won't be returned correctly

            qe.PageInfo.ReturnTotalRecordCount = true;

            return service.RetrieveMultiple(qe);


The only caveat with the above is that it is arguably only useful for if you are regularly importing Unmanaged, as opposed to Managed solutions, as the Publish All Customizations option is not displayed on the import wizard for unmanaged solutions. Nevertheless, by rolling out the above into your environment, you no longer need to scrabble around for the mental note you have to make when performing a solution update 🙂

Change control and management are important considerations for any IT system – and CRM is no different in this respect. Business processes are often subject to change at the drop of a hat, and organisations should ensure that they have robust, effective and, most of all, efficient processes in place for change management. Often these changes may preclude the removal of a particular aspect of a system – in CRM’s case, this could be a Form, a View or even an Entity field. You may often just decide to take the “easy” way out and not remove these components at all, choosing instead to hide or obscure them. Whilst this is fine in the immediate to short term, you are storing up problems long-term if you do not have a robust process in place to ensure these unnecessary or legacy components are eventually removed. The problem is, though, depending on your customisation deployment method – either as part of managed or unmanaged solution – your options in this regard could be hampered. It is, therefore, important to be aware of what the potential limitations are to both types of solutions so that you can structure your customisations in the most appropriate way. In this week’s blog post, we will take a closer look at some of the for and against arguments when it comes to removing solution components from your production CRM system, the behaviour of solution files and how they can assist and even impede a change management process.

So why should you ensure that unnecessary/legacy components are removed from your Production system? And is there a case to not remove them at all?

  • Components can potentially take up unnecessary space within your solution, leading to delays in performing solution updates. If you also have entity fields in your CRM that are no longer in use but are still storing data, then this could have adverse effects on your database storage levels – an important and essential consideration for CRM Online deployments in particular.
  • Clarity and simplicity are hallmarks of a well managed and maintained system. Being in a situation where you have components in your system, that could easily be interpreted as being still in use or active, could lead to hours, if not days, of confusion and wasted time.

Clearly, there are practical, if not somewhat idealistic, arguments in favour of the above. So what are the arguments against?

  • Removing a component almost straightaway could present problems if, for example, it turns out that the reasons for its removal were mistaken. You would potentially create more work for yourself in having to re-create a particular customisation when keeping it could have saved considerable effort and time.
  • The above can be compounded further if it turns out that crucial business information was stored in, for example, a field that is deleted. Keeping the field intact can ensure that these potential situations are avoided.

Let’s see now things look in practice when we attempt to emulate a change process within CRM

For testing purposes, we have created a custom entity – Test Solution Entity – which contains 2 custom Forms, Views and Fields:



This entity has been moved into an unmanaged solution, which will be exported as unmanaged and managed and then deployed into a separate CRM environment. We will then observe what happens when we push out an update to the solution, that has had certain components removed – in this case, the following components:

  • Test Form 1
  • Test View 1
  • Test Date Field


Updating an unmanaged solution will do nothing to existing components that have been deleted – even if you specifically delete them from the solution in your development system. Therefore, as a best practice, any component that you choose to specifically delete from your unmanaged solution will need to be noted down and included as part of your release notes for your solution update. Once the update has been completed, you will then need to go into your production system and proceed with removing these components. Regardless of the type of customisation, you should encounter no problems deleting them – in most cases, all required dependencies for the component will have been removed as part of your solution update and the components themselves will be in an unmanaged state, meaning you are unrestricted in what you can do with them.


You may assume that a Managed solution update would differ from an unmanaged in behaviour. In fact, for this example at least, when we import our updated managed solution, then the components we deleted are persisted in the solution. What’s worse, because these components are in a managed state, the steps involved in removing them may be complicated significantly. Fortunately, for Forms and Views, there is a Managed Property that can be configured to allow us to delete a Form/View if it is in a managed state:

5 6

These settings always default to True, so you do not need to specifically remember to set them.

There is some bad news, however – there is no such setting available for entity fields:


In addition, the following customisation components can not be set to Can be Deleted whilst in a managed state:

  • Entity Relationships
  • Business Rules
  • Global Option Sets
  • Web Resources
  • Processes (Workflows, Dialogs etc.)
  • Reports
  • Connection Roles
  • Templates (Article, Email etc.)
  • Security Roles

This can present a problem over time if we assume that over the lifecycle of your solution, you need to remove redundant fields – these will be maintained and will only be removed if you choose to completely uninstall and re-install your solution. Depending on the nature of your solution, this could cause the following problems:

  • Uninstalling the solution will delete all entity records from the system, as all components in the solution will be deleted.
  • There could be significant time and effort involved in the re-installation process – most likely this will need to be done out of hours, given that everything within your solution will not be available for the duration of the re-install.
  • If you have other unmanaged/managed solutions that have dependencies on components within your managed solution, then this could cause issues with these customisations; and could even mean that you have to re-install these solutions as well.

Conclusions or Wot I Think

The above examples have looked at situations purely where you are making bespoke customisations to CRM. Sometimes deleting components may not be possible at all, particularly if you are using “out of the box” (OOB) components included with CRM. In these cases, you have no choice but to obfuscate these parts of the CRM systems as part of the customisations you make to the system – either through a well-defined security structure or by simply not exposing such elements of the system via the sitemap, for example. Putting OOB components aside, if we are to look at purely bespoke customisation elements of a CRM deployment, I would argue that striking a balance between the two extremes – leaving components that are no longer required in your system compared with deleting them at the first available opportunity – is often the best way to proceed. For example, having an internal process in place that ensures removal requests are signed off by a senior member of the business or by having a “grace period”, where customisations are flagged for deletion but not actioned until a set amount of time has expired. Tied up as part of this, you can very straightforwardly perform backups of your CRM data via the Export to Excel feature. Given how easy and accessible this feature is, there really is no excuse not to perform backups of fields that you intend to delete; then, if the worse happens and you need to restore customisations at a later date (which, on a separate note, should be straightforward so long as you keeping regular backups of your CRM solution files!), you can very quickly restore the data within these fields.

No matter which approach you take, I would argue that it is ultimately preferable to ensure that your CRM solutions are kept in a tidy, current and clear state at all times. You are doing a huge disservice to your current and future colleagues within your business by not following this mantra. In the process as well, you take a rather cavalier approach to what I would hope would be(!) one of your businesses most important assets.

When I first started looking more closely at some of the new features within Dynamics CRM 2016, my initial thoughts was this was that this was a release for the fans (i.e. CRM Administrators, Customisers & Developers). Putting aside some of the big headline grabbing features such as the Interactive Service Hub and Word Templates, there looks to be a lot of subtle changes underneath the hood which those who work with CRM on a daily basis will be jumping for joy at.

One of the fundamental concepts that needs to be grasped as part of any CRM development work is Solutions. This includes understanding the differences between managed/unmanaged solution files, how to export/import solutions between different environments and how managed/unmanaged customisations are handled by CRM. There are many debates and discussions online regarding the subject, but the general rule of thumb is to use unmanaged solutions for the majority of your customisations, and only use managed solutions for when you are an ISV developing a bespoke solution to performs a specific function and/or links in with a separate application.

One of my personal bugbears regarding Solutions is the time and effort involved as part of doing an update. If, for example, you have just one Solution file for your entire businesses CRM customisations, then over time pushing out a solution update into your Production environment can take up to 30 minutes or more to complete – even if your update only contains, for example, a field name change! This can make pushing out hot-fixes or urgent updates more time-consuming and lead to delays in addressing problems within a live production system.

Whilst doing some digging around within CRM 2016, I was very excited to see the following 2 new buttons on the Solutions page:




Could it be that we now have a better and more efficient way of pushing out small parts of our CRM customisations and urgent hot-fixes? Before we take a look at how these buttons work in practice, it is useful to first explain what each one does in more detail:

Clone a Patch

Clone a Patch creates a patch of your main solution, which can contain a selection of CRM customisations that you wish to export from the system. A solution can have multiple patch solutions. One thing to point out is that you must manually add the components you wish to include as part of your patch solution into the Patch solution file. When, for example, you add in the Entity which you wish to include in the patch, you can specify individual entity forms, views, charts etc. that you want to include, something which I really like:


Whilst there is at least one patch of a solution present in your CRM system, you will not be able to edit the main solution file. CRM presents you with a yellow banner message to inform you of this when you enter the CRM Solution in question:



In order for the solution to become available for customisation again, you will need to either a) delete all patch solution files that derive from the main solution or b) use the Clone Solution button, which will be explained in more detail next.

Clone Solution

Clone Solution enables you to quickly combine together a solution and all of its descendant patches back into one solution file. Say for example, you have 1 Main Solution file and 3 Patch Solutions from the Main Solution. Clicking the Clone Solution button will combine all of these together again as part of 1 solution file. As a result, this will then allow you to customise the main solution file again without any issues.

So how does this work in practice then? Let’s find out:

To start of with, here is a custom entity within a new solution file. Nothing has been customised for this entity at this stage:


This is then exported out of the system as a solution file and imported into our target environment. Next, let’s say we want to add a newly created field to our entity as well as making some other changes as well within the main solution. These are in an incomplete stage, so therefore we just want to export our new field only as a patch solution for now.

On the Solutions page, we select our Main solution file and press the Clone a Patch button. We are greeted with a screen similar to the below where we can specify a few optional details, before then confirming:

CRM2016SolutionChanges_3 CRM2016SolutionChanges_4

Next, we then have to go into our Patch Solution and add in the elements that we wish to push out as part of a patch. In this case, we want to add a new field to our Test Entity. We therefore go into a Patch Solution file and go to Add Existing… in order to add in our Test Entity. You will be greeted with the Entity Assets window above and, in this example, we want to include all assets so we just press Finish to add this in.

Now we want to add in our new field:



As above, this is then imported into our target CRM environment, as you would do normally for any other solution file. It even looks the same!

CRM2016SolutionChanges_13 CRM2016SolutionChanges_14


Now, lets say we want to pull together all of our other changes as part of a traditional solution update. We also want to rename our Testing Field as follows:

We select our Solution file and click on the Clone Solution button. At this stage, we can modify the display name of the Solution and also specify a new version:


When we click Save, the patch solution file will vanish from our solution file, replaced by our one solution file which can now be edited and exported accordingly.

And that’s it! It takes a while to get your head around, particularly if you have been used to working with Solutions in previous versions of CRM, and I think it will take a while before this starts to become widely used (to be honest, I have not yet used it as part of some of the recent solution updates I have had to do 😳). Still, it is good to know that for those who are just coming into CRM Administration/Development, you can get to grips with this feature straight away without having to concern yourself too greatly with the “old” method. Just don’t be surprised if you hear from CRM veterans something like “Back in my day, we just had Solution updates – and we were lucky to have them!”

Let me know in the comments below if you have started to use Solution Patches yourself within your CRM 2016 environment.