The importance of segregated deployment environments for any IT application cannot be understated. Even if this only comprises of a single test/User Acceptance Testing (UAT) environment, there are a plethora of benefits involved, which should easily outweigh any administrative or time effort involved:

  • They provide a safe “sandbox” for any functionality or developmental changes to be carried in sequence and verify the intended outcome.
  • They enable administrators to carry out the above within regular business hours, without risk of disruption to live data or processes.
  • For systems that experience frequent updates/upgrades, these types of environments allow for major releases to be tested in advance of a production environment upgrade.

When it comes Dynamics 365 Customer Engagement (D365CE), Microsoft more recently have realised the importance that dedicated test environments have for their online customers. That’s why, with the changes announced during the transition away from Dynamics CRM, a free Sandbox instance was granted with every subscription. The cost involved in provisioning these can start to add up significantly over time, so this change was and remains most welcome; mainly when it means that any excuse to carrying out the bullet point highlighted above in bold immediately gets thrown off the table.

Anyone who is presently working in the D365CE space will be intently aware of the impending forced upgrade to version 9 of the application that must take place within the next few months. Although this will doubtless cause problems for some organisations, it is understandable why Microsoft is enforcing this so stringently and – if managed correctly – allows for a more seamless update experience in the future, getting new features into the hands of customers much faster. This change can only be a good thing, as I have argued previously on the blog. As a consequence, many businesses may currently have a version 9 Sandbox instance within their Office 365 tenant which they are using to carry out the types of tests I have referenced already, which typically involves testing custom developed or ISV Managed/Unmanaged Solutions. One immediate issue you may find if you are working with solutions containing the Marketing List (list) entity is that your Solution suddenly stops importing successfully, with the following error messages:

The error message on the solution import is a little vague…

…and the brain starts hurting when we look at the underlying error text!

The problem relates to changes to the Marketing List entity in version 9 of the application, specifically the Entity property IsVisibleInMobileClient. This assertion is confirmed by comparing the Value and CanBeChanged properties of these settings in both an 8.2 and 9.0 instance, using the Metadata Browser tool included in the SDK:

Here’s the setting in Version 8.2…

…and how it appears now in Version 9.0

Microsoft has published a support article which lists the steps involved to get this resolved for every new solution file that you find yourself having to ship between 8.2 and 9.0 environments. Be careful when following the suggested workaround steps, as modifications to the Solution file should always be considered a last resort means of fixing issues and can end up causing you no end of hassle if applied incorrectly. There is also no way of fixing this issue at source as well, all thanks to the CanBeChanged property being set to false (this may be a possible route of resolution if you are having this same problem with another Entity that has this value set to true, such as a custom entity). Although Microsoft promises a fix for this issue, I wouldn’t necessarily expect that 8.2 environments will be specially patched to resolve this particular problem. Instead, I would take the fix to mean the forced upgrade of all 8.2 environments to version 9.0/9.1 within the next few months. Indeed, this would seem to be the most business-sensible decision rather than prioritising a specific fix for an issue that is only going to affect a small number of deployments.

Earlier this month, a colleague escalated an issue to me involving Dynamics CRM/365 Customer Engagement (CRM/D365CE), specifically relating to email tracking. This feature is by far one of the most useful and unwieldy within the application, if not configured correctly. In days of yore, the setup steps involved could be tedious to implement, mainly if you were operating within the confines of a hybrid environment (for example, CRM 2015 on-premises and Exchange Server Online). Or, you could have been one of the handful of unfortunate individuals on the planet today that had to rely on the abomination that is the Email Router. We can be thankful today that Server-Side Synchronization is the sole method for pulling in emails from any manner of SMTP or POP3 mail servers; although note that only Exchange mailboxes support Appointment, Contact & Task synchronisation. Lucky though we are to be living in more enlightened times, careful attention and management of Server-Side Synchronization deployments is still an ongoing requirement. This is primarily to ensure all mailboxes operate as intended and – most critically – to ensure that only the most relevant emails are tagged back into the application, and not instead a deluge of unrelated correspondence.

Going back to the issue mentioned at the start of this post – a user in question was having a problem with certain emails not synchronising automatically back into the application, even though the emails in question had a corresponding Contact record within CRM/D365CE. We were also able to observe that other emails sent from the user to the Contact record(s) in question were being tagged back without issue. When first diagnosing problems like this, you can forgive yourself for not straight away making a beeline to the user’s Mailbox record within the application to verify that:

  • The Mailbox is enabled for Server-Side Synchronization for Incoming/Outgoing Email.
  • No processing errors are occurring that could be preventing emails from being successfully handled by the application.

These options can be accessed from the System Settings area of the application, on the Email tab, and define the default settings for all newly created users.

Likewise, these details can are accessible from the Mailbox record for the user concerned.

Although not likely (more often than not) to be the cause of any mail flow issues, it is worthwhile not to potentially overcomplicate a technical issue at the first juncture by overlooking anything obvious. 🙂

As we can see in this example, there are no problems with the over-arching Server-Side Synchronization configuration, nor are there any problems with the individual mailbox. It is at this point that we must refer to the following screen that all users in the application have access to via the gear icon at the top of the screen – the User Options screen:

The Track option allows users to specify how CRM/D365CE handles automatic email tracking, based on four options:

  • All Email Messages: Does exactly what it says on the tin, and is not recommended to leave on as default, for the reasons I alluded to earlier.
  • Email messages in response to Dynamics 365 Email: Only emails sent from within Dynamics 365 (or tracked accordingly via Outlook) will be stored in the application, alongside any replies that are received.
  • Email messages from Dynamics 365 Leads, Contacts and Accounts: Only emails which match back to the record types listed, based on email address, will be stored within the application.
  • Email messages from Dynamics 365 records that are email enabled: The same as the previous option, but expanded out to include all record types that are configured with the Sending email… option on the Entity configuration page.

For the user who was having email tracking issues, the default setting specified was Email messages in response to Dynamics 365 Email. So, to resolve the issue, it is necessary for the user to update their settings to either the 3rd or 4th option.

Any situation that involves detailed, technical configuration by end-users are generally ones that I like to avoid – for a few simple, business-relevant reasons:

  • IT/Technical teams should be the ones making configuration changes to applications, not end users who have not had training or experience on the steps they are being asked to follow.
  • End-users are busy, and it is always essential that we are conscious of their time and in making any interaction short and positive as opposed to long and arduous.
  • If the above instructions are relayed over the telephone, as opposed to in-person, then the propensity for mistakes to occur rises significantly.

However, from what we have seen so far, it will be necessary to access the application as the user to make the change – either by taking control of their session or by (perish the thought) relaying user credentials to enable someone in IT support to make the configuration change. Don’t EVER do this option by the way! Fortunately, there is a better way of updating user profile settings, using a tool whose importance has been highlighted in no uncertain terms previously on the blogthe XrmToolbox. One of the handiest out of the box tools that this provides is the User Settings Utility which…well…see for yourself:

As a consequence, application administrators can “magically” modify any of the settings contained within the User Options page, including – as we can see below – the Track email messages setting:

With a few clicks, the appropriate changes can be applied not just to a single user, but to everyone within the application – avoiding any potential end-user confusion and making our jobs easier. This simple fact is another reason why you should immediately launch the XrmToolBox whenever you find yourself with a CRM/D365CE issue that stumps you and why the community tools available for the application are top-notch.

The Voice of the Customer (VoC) add-on solution for Dynamics 365 Customer Engagement (D365CE) presents a really nice way of incorporating survey capabilities within your existing Dynamics application estate, without any additional cost or significant administrative overhead. I’ve talked about the tool previously, within the context of specific application errors, and I can attest to its capabilities – both as a standalone solution and as one that can be leveraged alongside other D365CE functionality to generate additional value.

One feature that is particularly useful is the ability to include diverse Survey Response controls. This can cover the range of anticipated user inputs that most web developers would be used to – text inputs, ratings, date pickers etc. – along with more marketing specific choices such as Net Promoter Score and even a Smilies rating control. The final one of these really does have to be seen to wholly appreciate:

I hope you agree that this is definitely one of those features that becomes so fun that it soaks up WAY more time than necessary 🙂

One of the final options that VoC provides you is the ability to upload files to a Survey Response, which is stored within the application and made retrievable at any time by locating the appropriate Survey Response record. You can customise the guidance text presented to the user for this control, such as in the example below:

Uploaded files are then saved onto an Azure Blob Storage location (which you don’t have direct access to), with the access URL stored within D365CE. The inclusion of this feature does provide the capability to accommodate several potential business scenarios, such as:

  • Allowing a service desk to create an automated survey that allows error logs or screenshots to be uploaded for further diagnosis.
  • The gathering of useful photographic information as part of a pre-qualification process for a product installation.
  • Enabling customers to upload a photo that provides additional context relating to their experience – either positive or negative.

Putting all of this aside, however, and there are a few things that you should bear in mind when first evaluating this feature for your particular requirements. What follows is my list of major things to be aware of, along with some tips to sidestep any issues.

Privacy concerns…

To better understand why this is relevant, it helps to be aware of exactly how files can be stored on Azure. Azure file storage works on the principle of “blobs” (i.e. files), which can only be created within a corresponding Storage Container. These can be configured using a couple of different options, depending on how you would like to access your data, which is elaborated upon in this really helpful article:

You can configure a container with the following permissions:

  • No public read access: The container and its blobs can be accessed only by the storage account owner. This is the default for all new containers.

  • Public read access for blobs only: Blobs within the container can be read by anonymous request, but container data is not available. Anonymous clients cannot enumerate the blobs within the container.

  • Full public read access: All container and blob data can be read by anonymous request. Clients can enumerate blobs within the container by anonymous request, but cannot enumerate containers within the storage account.

To presumably mitigate the need for complex deployments of the VoC solution, all uploaded Survey Response files are saved in Full public read access storage containers, meaning that anyone with the URL can access these files. And, as mentioned already, administrators have no direct access to the Azure Storage Account to modify these permissions, potentially compounding this access problem. Now, before you panic too much, the VoC solution deliberately structures the uploaded file in the following format:

https://<VoC Region Identifier>.blob.core.windows.net/<Survey Response GUID>-files/<File GUID>-<Example File>

This degree of complexity added during this goes a long way towards satisfying any privacy concerns – it would be literally impossible for a human being or computer to guess what a particular uploaded file path is, even if they did have the Survey Response record GUID – but this still does not address the fact that the URL can be freely accessed and shared by anyone with sufficient permissions over the Survey Response entity in D365CE. You should, therefore, take appropriate care when scoping your security privileges within D365CE and look towards carrying out a Privacy Impact Assessment (PIA) over the type of data you are collecting via the upload file control.

…even after you delete a Survey Response.

As mentioned above, the Blob Storage URL is tagged to the Survey Response record within D365CE. So what happens when you delete this record? The answer, courtesy of Microsoft via a support request:

Deleting Survey Response should delete the file uploaded as part of the Survey Response

Based on my testing, however, this does not look to be the case. My understanding of the VoC solution is that it needs to regularly synchronise with components located on Azure, which can lead to a delay in certain actions completing (publish a Survey, create Survey Response record etc.). However, a file from a Survey Response test record that I deleted still remains accessible via its URL up to 8 hours after completing this action. This, evidently, raises a concern over what level of control you have over potentially critical and sensitive data types that may be included in uploaded files. I would urge you to carry out your own analysis as part of a PIA to sufficiently gauge what impact, if any, this may have on your data collection (and, more critically, storage) activities.

Restrictions

For the most part, file upload controls are not a heavily constrained feature, but it is worthwhile to keep the following restrictions in mind:

  • Executable file types are not permitted for upload (.exe, .ps1, .bat etc.)
  • Larger file types may not upload successfully, generating 404 server errors within the control. There is not a documented size limitation, but my testing would indicate that files as big as 60MB will not upload correctly.
  • Only one file upload control is allowed per survey.

The last of these limitations is perhaps the most significant constraint. If you do have a requirement for separate files to be uploaded, then the best option is to provide instructions on the survey, advising users to compress their files into a single .zip archive before upload.

Conclusions or Wot I Think

Despite what this post may be leaning towards, I very much believe the VoC solution and, in particular, the ability to upload Survey Response files, is all in a perfect, working condition. Going a step further on this, when viewed from a technical standpoint, I would even say that its method of execution is wholly justified. With the advent of the General Data Protection Regulations (GDPR) earlier this year, current attention is all around ensuring that appropriate access controls over data have been properly implemented, that ensures the privacy of individuals is fully upheld. Here is where the solution begins to fall over to a degree and evidence of the journey that VoC has made in becoming part of the Dynamics 365 “family” becomes most apparent. As can be expected, any product which is derived from an external acquisition will always present challenges when being “smushed” with a new application system. I have been informed that there is an update coming to the VoC solution in August this year, with a range of new features that may address some of the data privacy concerns highlighted earlier. For example, the option will be provided for administrators to delete any uploaded file within a Survey Response on-demand. Changes like this will surely go a long way towards providing the appropriate opportunities for VoC to be fully utilised by businesses looking to integrate an effective, GDPR-proof, customer survey tool.

The Voice of the Customer (VoC) solution, available as part of Dynamics 365 Customer Engagement (D365CE), works most effectively when you are tightly integrating your survey’s around other features or datasets stored within the application. That’s not to say that it must only ever be utilised in tandem as opposed to isolation. If you have the requirement to quickly send out a survey to a small list of individuals (regardless of whether they are D365CE Contact records), VoC presents a natural choice if you are already paying for D365CE Online, as it is included as part of your subscription cost. As services such as SurveyMonkey tend to charge money to let you develop more complex, bespoke branded surveys, VoC, by comparison, offers all of this to you at no additional cost. Just don’t be buying licenses to use only this specific piece of functionality. 🙂 Ultimately, the upside of all this is that VoC represents a solid solution in and of itself, but working with the solution in this manner is just the icing on top of the cake. When you start to take into account the myriad of different integration touchpoints that VoC can instantly support, thanks to its resident status within D365CE, this is where things start to get really exciting. With this in mind, you can look to implement solutions that:

  • Send out surveys automatically via e-mail.
  • Route survey responses to specific users, based on answers to certain questions or the Net Promoter Score (NPS) of the response.
  • Tailor a specific email to a customer that is sent out after a survey is completed.
  • Include survey data as part of an Azure Data Export Profile and leverage the full power of T-SQL querying to get the most out of your survey data.

It is in the first of these scenarios – sending out surveys via a WorkFlow – that you may find yourself encountering the error referenced in the title of this post. The error can occur when you look to take advantage of the survey snippet feature as part of an Email Template – in laymen’s terms, the ability to send out a survey automatically and tag the response back to the record that it is triggered from. To implement this, you would look towards creating a WorkFlow that looks similar to the below:

With then either the desired email message typed out or a template specified that contains the survey snippet code, available from a published survey’s form:

All of this should work fine and dandy up until the user in question attempts to trigger the workflow; at which point, we see the appropriate error message returned when viewing the workflow execution in the System Jobs area of the application:

Those who are familiar with errors like these in D365CE will instantly realise that this is a security role permission issue. In the above example, the user in question had not been granted the Survey User role, which is included as part of the solution and gives you the basic “set menu” of permissions required to work with VoC. A short while following on from rectifying this, we tried executing the workflow again and the error still occurred, much to our frustration. Our next step was to start combing through the list of custom entity privileges on the Security Role tab to see if there was a permission called Azure Deployment or similar. Much to our delight, we came across the following permission which looked like a good candidate for further scrutiny:

When viewing this security role within the Survey User role, the Read permission for this organization-owned custom entity was not set. By rectifying this and attempting to run the workflow again, we saw that it executed successfully 🙂

It seems a little odd that the standard user security role for the VoC module is missing this privilege for an arguably key piece of functionality. In our situation, we simply updated the Survey User security role to include this permission and cascaded this change accordingly across the various dev/test/prod environments for this deployment. You may also choose to add this privilege to a custom security role instead, thereby ensuring that it is properly transported during any solution updates. Regardless, with this issue resolved, a really nice piece of VoC functionality can be utilised to streamline the process of distributing surveys out to D365CE customer records.

After going through a few separate development cycles involving Dynamics 365 Customer Engagement (D365CE), you begin to get a good grasp of the type of tasks that need to be followed through each time. Most of these are what you may expect – such as importing an unmanaged/managed solution into a production environment – but others can differ depending on the type of deployment. What ultimately emerges as part of this is the understanding that there are certain configuration settings and records that are not included as part of a Solution file and which must be migrated across to different environments in an alternate manner.

The application has many record types that fit under this category, such as Product or Product Price List. When it comes to migrating these record types into a Production environment, those out there who are strictly familiar with working inside the application only may choose to utilise the Advanced Find facility in the following manner:

  • Generate a query to return all of the records that require migration, ensuring all required fields are returned.
  • Export out the records into an Excel Spreadsheet
  • Import the above spreadsheet into your target environment via the Data Import wizard.

And there would be nothing wrong with doing things this way, particularly if your skillset sits more within a functional, as opposed to technical, standpoint. Where you may come unstuck with this approach is if you have a requirement to migrate Subject record types across environments. Whilst a sensible (albeit time-consuming) approach to this requirement could be to simply create them from scratch in your target environment, you may fall foul of this method if you are utilising Workflows or Business Rules that reference Subject values. When this occurs, the application looks for the underlying Globally Unique Identifier (GUID) of the Subject record, as opposed to the Display Name. If a record with this exact GUID value does not exist within your target environment, then your processes will error and fail to activate. Taking this into account, should you then choose to follow the sequence of tasks above involving Advanced Find, your immediate stumbling block will become apparent, as highlighted below:

As you can see, there is no option to select the Subject entity for querying, compounding any attempts to get them exported out of the application. Fortunately, there is a way to get overcome this via the Configuration Migration tool. This has traditionally been bundled together as part of the applications Solution Developer Kit (SDK). The latest version of the SDK for 8.2 of the application can be downloaded from Microsoft directly, but newer versions – to your delight or chagrin – are only available via NuGet. For those who are unfamiliar with using this, you can download version 9.0.2.3 of the Configuration Migration tool alone using the link below:

Microsoft.CrmSdk.XrmTooling.ConfigurationMigration.Wpf.9.0.2.3

With everything downloaded and ready to go, the steps involved in migrating Subject records between different D365CE environments are as follows:

  1. The first step before any export can take place is to define a Schema – basically, a description of the record types and fields you wish to export. Once defined, schemas can be re-used for future export/import jobs, so it is definitely worth spending some time defining all of the record types that will require migration between environments. Select Create schema on the CRM Configuration Migration screen and press Continue.

  1. Login to D365CE using the credentials and details for your specific environment.

  1. After logging in and reading your environment metadata, you then have the option of selecting the Solution and Entities to export. A useful aspect to all of this is that you have the ability to define which entity fields you want to utilise with the schema and you can accommodate multiple Entities within the profile. For this example, we only want to export out the Subject entity, so select the Default Solution, the entity in question and hit the Add Entity > button. Your window should resemble the below if done correctly:

  1. With the schema fully defined, you can now save the configuration onto your local PC. After successfully exporting the profile, you will be asked whether you wish to export the data from the instance you are connected to. Hit Yes to proceed.

  1. At this point, all you need to do is define the Save to data file location, which is where a .zip file containing all exported record data will be saved. Once decided, press the Export Data button. This can take some time depending on the number of records being processed. The window should update to resemble the below once the export has successfully completed. Select the Exit button when you are finished to return to the home screen.

  1. You have two options at this stage – either you can either exit the application entirely or, if you have your target import environment ready, select the Import data and Continue buttons, signing in as required.

  1. All that remains is to select the .zip file created in step 5), press the Import Data button, sit back and confirm that all record data imports successfully.

It’s worth noting that this import process works similarly to how the in-application Import Wizard operates with regards to record conflicts; namely, if a record with the same GUID value exists in the target instance, then the above import will overwrite the record data accordingly. This can be helpful, as it means that changes to records such as the Subject entity can be completed safely within a development context and promoted accordingly to new environments.

The Configuration Migration tool is incredibly handy to have available but is perhaps not one that it is shouted from the rooftops that often. It’s usefulness not just extends to the Subject entity, but also when working with the other entity types discussed at the start of this post. Granted, if you do not find yourself working much with Processes that reference these so-called “configuration” records, then introducing the above step as part of any release management process could prove to be an unnecessary administrative burden. Regardless, there is at least some merit to factor in the above tool as part of an initial release of a D365CE solution to ensure that all development-side configuration is quickly and easily moved across to your production environment.