Ranking Categories within Power BI Datasets (DAX)

by ⋅ Leave a Comment

I’ve been doing some work with Power BI Online datasets this week. It’s the first time I’ve taken a look at them in any great detail, as I have traditionally preferred to create and deploy any data sources needed for my reports via the Desktop client. Datasets address the needs for users who do not necessarily have the necessary Power Query/DAX language and require a mechanism to quickly hook up to a pre-prepared dataset, thereby allowing them to start consuming any data rapidly. They are defined within an individual workspace and, once deployed, can then be shared out to other users to connect to. Datasets can help towards ensuring that any extensive piece of work developing a data model can be benefitted by as many users as possible and can reduce the need for people having to create data sources themselves if organised correctly. To find out more about how to create a dataset, you can take a look through my recent series all around Microsoft Exam 70-778, where I cover this topic in further detail.

Datasets can remove some headaches for more substantial Power BI deployments, but you should be aware of they can’t do. The most noteworthy limitations include:

  • You can only connect a Power BI report to a single Power BI dataset at any time. You are unable to bring in additional data sources via Power Query and, likewise, if you have already defined several data sources within your report, you cannot then bring a Power BI dataset into your model.
  • It is not possible to make any modifications to a Power BI dataset from within Power BI Desktop, either via Power Query manipulation or by adding DAX custom columns; you can, however, define new Measures using DAX.

Where this can start to be problematic is when you are attempting to surface data generated by the Applications Insights Continuous Export facility via a Stream Analytics job. The great thing about Stream Analytics is that you can define multiple input/output locations for data that it processes, one of which includes a Power BI dataset. A convenient feature and one that can potentially sidestep the need to export any information out to a SQL-based destination for further processing. However, one area where the feature handicaps itself is in the fact that you cannot output multiple tables within the same Power BI dataset. If you attempt to do this, you get the following error message:

When you take into account the limitations mentioned above concerning a strict 1:1 mapping between dataset/report, problems can arise when it comes to defining your queries on the Stream Analytics side of things. You either have to export out all of the base information in a raw format or instead develop highly customised queries on the Stream Analytics side. The latter solution may successfully meet a specific business requirement, but risks putting you in a position where your BI solution contains many different reports, some of which may only include a meagre amount of visualisations. I’m not sure about you, but my preference would be more towards the first option, as opposed to building out an overtly complex BI solution; but the limitations that Power BI datasets introduce for this particular scenario does present some challenge in sticking to this mantra.

For example, assume we have the following Stream Analytics query that outputs Request information from Application Insights into a Power BI dataset:

SELECT 
    requestflat.ArrayValue.id AS RequestID,
    requestflat.ArrayValue.name AS RequestType,
    requestflat.ArrayValue.responseCode AS ResponseCode,
    requestflat.ArrayValue.success AS RequestSuccessful,
    requestflat.ArrayValue.url AS URL,
    requestflat.ArrayValue.urlData.base AS BaseURL,
    requestflat.ArrayValue.urlData.host AS URLHost,
    requestflat.ArrayValue.durationMetric.value AS Duration,
    r.internal.data.id AS ID,
    r.internal.data.documentVersion AS DocumentVersion,
    r.context.data.eventTime AS EventTime,
    r.context.data.isSynthetic AS IsSynthetic,
    r.context.data.syntheticSource AS SyntheticSource,
    r.context.data.samplingRate AS SamplingRate,
    r.context.device.type AS DeviceType,
    r.context.device.roleName AS SlotName,
    r.context.device.roleinstance AS RoleInstance,
    r.context.session.isFirst AS IsFirstSession,
    r.context.operation.id AS OperationID,
    r.context.operation.parentID AS OperationParentID,
    r.context.operation.name AS OperationName,
    GetRecordPropertyValue(GetArrayElement(r.[context].[custom].[dimensions], 0), 'Platform') AS Platform,
    GetRecordPropertyValue(GetArrayElement(r.[context].[custom].[dimensions], 1), 'Browser') AS Browser,
    GetRecordPropertyValue(GetArrayElement(r.[context].[custom].[dimensions], 3), 'UserAgent') AS UserAgent,
    GetRecordPropertyValue(GetArrayElement(r.[context].[custom].[dimensions], 4), 'Browser_Version') AS BrowserVersion,
    GetRecordPropertyValue(GetArrayElement(r.[context].[custom].[dimensions], 5), 'Referrer') AS ReferralURL,
    r.EventProcessedUtcTime AS RequestEventProcessedUtcTime,
    r.context.[user].anonId AS AnonymousID,
    r.context.location.continent AS ClientLocation,
    r.context.location.country AS ClientCountry
INTO [PowerBIRequestsOutput]
FROM [Requests] AS r
CROSS APPLY GetElements(r.[request]) AS requestflat

Having the data in this format provides us with the most flexibility when tailoring things on the Power BI side of things. But what if you wanted to perform some ranking on the data, based on a distinct category value – for example, ranking each of the Browser values in popularity order, based on each visitor to the website? While it is certainly possible to do this via a Stream Analytics query, you would end up having to group the data, thereby reducing the wider usage that the dataset could accommodate. Fortunately, thanks to the fact that we can create DAX Measures, it is possible to overcome this to generate a ranking per category and then display the most popular browser as part of a Card visualisation. We first need to create the following Measure within Power BI Desktop:

Browser Ranking = RANKX(
                        ALLSELECTED(PowerBIRequestsOutput[browser]), 
                        CALCULATE(
                                DISTINCTCOUNT(
                                    PowerBIRequestsOutput[anonymousid]
                                    )
                                ),,,Dense
                            )

We can confirm this works as expected by dropping a quick table visualisation in and verifying that each Browser is being ranked correctly, based on the count of the anonymousid field:

So far, so good 🙂 And this is potentially a visualisation that has a good usage case solely on its own; but, with an additional DAX Measure, we can return the highest ranked value above, IE, via the following Measure:

Most Used Browser = TOPN(1, FILTER(VALUES(PowerBIRequestsOutput[browser]), [Browser Ranking] = 1))

Now, the thing to mention here is that the above does not explicitly handle ties in any sophisticated way; therefore, there is no guarantee over which value will display in the event of a draw for the top ranking. Nevertheless, it is encouraging to know that DAX provides us with the types of capabilities we need when we find ourselves unable to do the kind of manipulation we would want to, either via a SQL query or Power Query manipulation.

Backup Azure SQL Database within Azure DevOps Release Pipeline

by ⋅ Leave a Comment

Software deployments and updates are always a painful event for me. This feeling hasn’t subsided over time, even though pretty much every development project I am involved with these days utilises Azure DevOps Release Pipelines to automate this whole process for our team. The key thing to always stress around automation is that it does not mean that all of your software deployments suddenly become entirely successful, just because you have removed the human error aspect from the equation. In most cases, all you have done is reduce the number of times that you will have to stick your nose in to figure out what’s gone wrong 🙂

Database upgrades, which are done typically via a Data-tier Application Package (DACPAC) deployment, can be the most nerve-racking of all. Careful consideration needs putting towards the types of settings you define as part of your publish profile XML, as otherwise, you may find either a) specific database changes are blocked entirely, due to dependency issues or because intended data loss will occur or b) the types of changes you make could result in unintended data loss. This last one is a particularly salient concern and one which can be understood most fully by implementing staging or pre-production environments for your business systems. Despite some of the thought that requires factoring in before you can look to take advantage of DACPAC deployments, they do represent the natural option of choice in managing your database upgrades more formally, mainly when there is need to manage deployments into Azure SQL databases. This state of play is mostly thanks to the dedicated task that handles this all for us within Azure DevOps:

What this feature doesn’t make available to us are any appropriate steps we may need to take to generate a snapshot of the database before the deployment begins, a phase which represents both an equally desirable and necessary business requirement for software deployments. Now, I should point out that Azure SQL includes many built-in options around recovery and point in time restore options. These options are pretty extensive and enable you, depending on the database size tier you have opted for, to restore your database to any single time point over a 30-day point. The question that therefore arises from this is fairly obvious – why go to the extra bother (and cost) to create a separate database backup? Consider the following:

  • The recovery time for a point-in-time restore can vary greatly, depending on several factors relating to your database size, current pricing tier and any transactions that may be running on the database itself. In situations where a short release window constraints you and your release must satisfy a strict success/fail condition, having to go through the restore process after a database upgrade could lead to your application from being down longer then is mandated within your organisation. Having a previous version of the database already available there means you can very quickly update your application connection strings to ensure the system returns to operational use if required.
  • Having a replica copy of the database available directly after an upgrade can be incredibly useful if you need to reference data within the old version of the database post-upgrade. For example, a column may have been removed from one table and added to another, with the need to copy across all of this data accordingly. Although a point-in-time restore can be done to expose this information out, having a backup of the old version of the database available straight after the upgrade can help in expediting this work.
  • Although Microsoft promise and provide an SLA with point-in-time restore, sometimes its always best to err on the side of caution. 🙂 By taking a snapshot of the database yourself, you have full control over its hosting location and the peace of mind in knowing that the database is instantly accessible in case of an issue further down the line.

If any of the above conditions apply, then you can look to generate a copy of your database before any DACPAC deployment takes place via the use of an Azure PowerShell script task. The example script below shows how to achieve this requirement, which is designed to mirror a specific business process; namely, that a readily accessible database backup will generate before any upgrade is taken place and to create a copy of this within the same Azure SQL Server instance, but with the current date value appended onto it. When a new deployment triggers in future, the script will delete the previously backed up database:

#Define parameters for the Azure SQL Server name, resource group and target database

$servername = 'mysqlservername'
$rg = 'myresourcegroup'
$db = 'mydb'

#Get any previous backed up databases and remove these from the SQL Server instance

$sqldbs = Get-AzureRmSqlDatabase -ResourceGroupName $rg -ServerName $servername | select DatabaseName | Where-Object {$_.DatabaseName -like $db + '_Backup*'}

if (($sqldbs |  Measure-Object).Count)
{
	Remove-AzureRmSqlDatabase -ResourceGroup $rg -ServerName $servername -DatabaseName $sqldbs[0].DatabaseName
}

#Get the current date and convert it into a string, with format DD_MM_YYYY

$date = Get-Date
$date = $date.ToShortDateString()
$date = $date -replace "/", "_"

#Create the name of the new database

$copydbname = $db + '_Backup_' + $date

#Actually create the copy of the database

New-AzureRmSqlDatabaseCopy -CopyDatabaseName $copydbname -DatabaseName $db -ResourceGroupName $rg -ServerName $servername -CopyServerName $servername

Simply add this on as a pipeline task before any database deployment task, connect up to your Azure subscription and away you go!

Backups are an unchanging aspect of any piece of significant IT delivery work and one which cloud service providers, such as Microsoft, have proactively tried to implement as part of their Platform-as-a-Service (PaaS) product lines. Azure SQL is not any different in this regard and, you could argue that the point-in-time restore options listed above provide sufficient assurance in the event of a software deployment failure or a disaster-recovery scenario, therefore meaning that no extra steps are necessary to protect yourself. Consider your particular needs carefully when looking to implement a solution like the one described in this post as, although it does afford you the ability to recover quickly from any failed software deployment, it does introduce additional complexity into your deployment procedures, overall technical architecture and – perhaps most importantly – cost.

“Cannot move resources because some site(s) are hosted by other resource group(s)…” Error Message (Azure Web App)

by ⋅ Leave a Comment

After a few months of working with Microsoft Azure, you start to become familiar with the concepts behind the Resource Manager model and also how it is possible to move resources around, depending on business requirements. Whereas in the past, you would typically need to open a support request to Microsoft to complete this action, the vast majority of these operations can now be given to Azure administrators to finish instead. As such, we have the capability to:

  • Move a resource into a different resource group
  • Move a resource/resource group into a separate subscription, thereby altering its billing arrangements.
  • Migrate an entire subscription to a new Azure Active Directory tenant, which alters both its ownership and billing arrangements in one fell swoop.

There are a few things regarding a resource/resource group that remain unchangeable post-creation. For example, its name and location (UK South, North Europe etc.) are static properties that cannot be altered for the entirety of a resources lifespan, and the only way to modify this is to delete and re-create it from scratch – action steps that have severe implications for a production workload. There are also a few scenarios where it will not be possible to move a resource to another subscription. The list of resources this affects is dwindling as the months go by, but there are still many popular services that are affected by this – such as Azure Data Factory, Logic Apps and Service Fabric. Partners who have a desire to move their customers away from Microsoft direct to Cloud Solutions Provider (CSP) billing need to take note in particular of these limitations when scoping any migration exercise, as it could be that an entire project becomes derailed if an un-supported resource is in the mix.

Sometimes, even if you think you will be OK when migrating a resource, you will occasionally hit an issue during the validation stage of a move. I came across a good example of this when attempting to move an App Service Plan and its corresponding App Service to a new subscription location, with the following JSON error message being generated (modified so that it is readable):

{
	"code": "ResourceMoveProviderValidationFailed",
	"message": "Resource move validation failed. Please see details. Diagnostic information: timestamp '20190221T124038Z', subscription id '4e846969-3196-476f-a088-6e393bb5ce98', tracking id '6cf8ddf7-8d15-48bc-8b34-f7729bd44f0a', request correlation id '68c44a05-eeba-4d96-887c-56a228ab69a3'.",
	"details": [
		{
			"target": "Microsoft.Web/serverFarms",
			"message": "{\"Code\":\"BadRequest\",\"Message\":\"Cannot move resources because some site(s) are hosted by other resource group(s) but located in resource group 'myoriginalrg'. The list of sites and corresponding hosting resource groups: 'myappserviceplan:myoriginalrg,mywebapp1:myoriginalrg'. This may be a result of prior move operations. Move the site(s) back to respective hosting resource groups and try again."\ ... }
		}
	]
}

In this particular example, the App Service Plan and App Service had been the subject of a previous move operation from their original resource group (in this example, myoriginalrg) into the current resource group. The location of the original and current resource group when this move took place was in the same subscription, so the transfer completed without issue. Now, because I was looking to move the resources to a new subscription, the error message above appeared. What’s worse, the myoriginalrg resource group had been deleted a long time ago, meaning there it wasn’t immediately clear whether the option suggested in the error message was even possible. Fortunately, this was not the case, and the following workaround steps can be used to get your App Service Plan/App Services moved to your desired location.

  1. Create a new resource group in the same subscription where the App Service Plan/App Service exists, with the same name listed in the error message (in this case, myoriginalrg).
  2. Move the resources into the myoriginalrg resource group. Verify that this completes successfully.
  3. Re-attempt the resource move operation to your preferred new subscription location. The procedure should complete successfully.

My main worry when I first saw this error message is that the resources in question were tied permanently to a non-existent resource group and that the listed workaround steps were not going to work. Fortunately, my glass half full outlook proved me to be categorically wrong, and the workaround solved the issue entirely. I can’t really understand or explain why an App Service Plan / App Service creates such a binding to a resource group (and, to a lesser extent, a subscription), which therefore causes an issue like this to appear; fortunately, as we have seen, there is a way of getting things working as intended without having to involve Microsoft support 🙂

“Access to the registry key…” Error in PowerShell Script Task (Azure DevOps)

by ⋅ Leave a Comment

As epitomised by the recent rebranding exercise Microsoft conducted with the product, I very much see Visual Studio Team Services/Team Foundation Server Azure DevOps becoming an increasingly dominant tool for development teams the world over. I came into the product after discovering its benefits from Ben Walker at a CRMUG meeting a few years back and, since then, the product has proved to be a real boon in helping me to:

  • Manage code across various .NET and Dynamics CRM/Dynamics 365 Customer Engagement projects.
  • Formalise the code review process, via the usage of pull request approvals.
  • Log and track backlogs across various project work.
  • Automate the project build process and deployment into development environments, allowing us to identify broken builds quickly.
  • Fully automate release cycles, reducing the risk of human error and potentially laborious late night working.

I really would urge any company carrying out some form of bespoke development, mainly with the Microsoft technology stack, to give Azure DevOps a look. Contrary to some of the previous biases that would creep in as part of Microsoft’s products, Azure DevOps is very much built with openness at its core, allowing you to straightforwardly leverage sections or the whole breadth of its functionality to suit your particular purpose.

When you start working with Azure DevOps more in-depth, it becomes obvious that having a general awareness of PowerShell will hold you in good stead. Although targeted towards more Microsoft-focused deployments, PowerShell becomes your go-to tool when working with Azure in particular. Recently, I had a PowerShell Script task that previously executed with no issue whatsoever on a Windows 10 self-hosted agent. The first thing that the script did was to define the Execution Policy, using the following command:

Set-ExecutionPolicy Unrestricted

(For the uninitiated, the above is an atypical requirement for all PowerShell scripts to ensure that you do not hit any permission issues during script execution.)

As stated already, this was all working fine and dandy, until one day, when my deployments suddenly started failing with the following error message:

Executing the script manually on the machine in question, in an elevated PowerShell window, confirmed that it was not an issue on Azure DevOps side:

Well, this is one of those occasions where the actual error message tells you everything you need to know to get things working again. Whether something had changed on the machine or not as part of an update, it was now necessary to modify the PowerShell command to include the -Scope option outlined above. Therefore, the script should resemble the following:

Set-ExecutionPolicy Unrestricted -Scope CurrentUser

We can then confirm that the script no longer errors when being executed on the machine in question:

And, most importantly, the release pipeline in question now completes without any error 🙂

Perhaps its just me, but often in these types of situations, you can overlook the completely obvious and find yourself going down the rabbit hole, chasing a non-existant or grandiose solution to a particular IT problem. As this example clearly demonstrates, sometimes just reading the error message you are presented with properly can reduce a lot of wasted effort and allow you to resolve a problem faster than expected.

Exam 70-778 Revision Notes: Summary Post

by ⋅ 3 Comments

For the past 13 weeks on the blog, I have delivered a series of posts concerning Microsoft Exam 70-778, specifically focused towards providing a set of detailed revision notes that cover the broad array of Power BI features assessed as part of the exam. To round things off, today’s blog will bridge together everything I have discussed thus far in the series; with the hope being that this post can be a single reference point for those who have not been following the series to date.

Microsoft Exam 70-778 Overview

The exam, with its full title Analyzing and Visualizing Data with Microsoft Power BI, is targeted towards Business Intelligence (BI) and data professionals who are looking to validate their skills in working with Power BI. The exam is a necessary component, alongside Exam 70-779: Analyzing and Visualizing Data with Microsoft Excel, in attaining the Microsoft Certified Solutions Associate (MCSA) certification in BI Reporting. Successful candidates can then (optionally) pass an additional “elective” exam to gain the Microsoft Certified Solutions Expert (MCSE) certification in Data Management and Analytics.

Skills Measured in the Exam

The skills measured are outlined below, alongside links to the relevant posts from the series and the list of essential points to remember:

Consuming and Transforming Data By Using Power BI Desktop

Connect to data sources.
Skills Measured

May include: Connect to databases, files, folders; import from Excel; connect to SQL Azure, Big Data, SQL Server Analysis Services (SSAS)

Revision Notes

Exam 70-778 Revision Notes: Importing from Data Sources

Key Takeaways
  • Power BI supports a broad range of database systems, flat file, folder, application and custom data sources. While it is impossible to memorise each data source, you should at least broadly familiarise yourself with the different types at our disposal.
  • A crucial decision for many data sources relates to the choice of either Importing a data source in its entirety or in taking advantage of DirectQuery functionality instead (if available). Both routes have their own defined set of benefits and disadvantages. DirectQuery is worth consideration if there is a need to keep data regularly refreshed and you have no requirement to work with multiple data sources as part of your solution.
  • Live Connection is a specific data connectivity option available for SQL Server Analysis Services. It behaves similarly to DirectQuery.
  • It is possible to import an existing Excel BI solution into Power BI with minimal effort, alongside the ability to import standard worksheet data in the same manner as other flat file types.
Perform transformations
Skills Measured

May include: Design and implement basic and advanced transformations; apply business rules; change data format to support visualization

Revision Notes

Exam 70-778 Revision Notes: Performing Data Transformations

Key Takeaways
  • The Power Query M formula language is used to perform transformations to data once loaded into Power BI. Although it is possible to do this via code, Power BI allows us to define all of our required data changes from within the interface, without the need to write a single line of code.
  • Each data source connected to represents itself as a Query within Power BI. There are many options at your disposal when working with Queries, such as renaming, merging, duplication and the ability to disable or reference as part of other Queries.
  • There are wide-range of column transformations that can be applied, which are too numerous to mention. The Transform tab provides the best means of seeing what is available, with options ranging from formatting through to grouping and pivoting/unpivoting.
  • New columns are addable via the Add Column tab. You can choose to base new columns on calculations, conditional logic, other column values or as a defined list of ascending numbers, which may be useful for indexing purposes.
  • It is possible to merge or append queries together to suit your specific requirements. Merging involves the horizontal combination of Queries, whereas appending represents a vertical combination.
  • Parameters can be used to help optimise any complex filtering requirements.
  • Where possible, Power Query will attempt to use the most optimal query for your data source, based on the transformation steps you define. This action is known as Query Folding and, in most cases, SQL-derived data sources will support this option by default.
Cleanse data
Skills Measured

May include: Manage incomplete data; meet data quality requirements

Revision Notes

Exam 70-778 Revision Notes: Cleansing Data

Key Takeaways
  • Data can be filtered directly within Power Query, using Excel-like functionality to assist you in only returning the most relevant data in your queries. The data type of each field plays a particularly important part of this, as only specific filter options will be at your disposal if, for example, you are working with numeric data.
  • From a data quality perspective, you typically will need to handle column values that contain one of two possible value types:
    • Errors: This will usually occur as a result of a calculated column field not working correctly. The best solution will always be to address any issues with your calculated column, such as by using a conditional statement to return a default value.
    • Blanks/NULLs: A common symptom when working with SQL derived data sources, your real problems with blank values start to appear when you attempt to implement DAX custom columns/Measures outside of the Power Query Editor. It is, therefore, recommended that these are dealt with via a Replace action, depending on your fields data types. For example, a number field with blank/NULL values should be replaced with 0.
  • The Remove Rows option(s) can act as a quick way of getting rid of any Error or Blank/NULL rows and can also be utilised further to remove duplicates or a range of rows. In most cases, you will have similar options available to you with Keep Rows instead.
  • There are a variety of formatting options available to us when working with text/string data types. These range from fixing capitalisation issues in data, through to removing whitespace/non-printable character sets and even the ability to prepend/append a new value.

Modeling and Visualizing Data

Create and optimize data models.
Skills Measured

May include: Manage relationships; optimize models for reporting; manually type in data; use Power Query

Revision Notes

Exam 70-778 Revision Notes: Create and Optimise Data Models

Key Takeaways
  • Relationships form the cornerstone of ensuring the long-term viability and scalability of a large data model. Assuming you are working with well-built out, existing data sources, Power BI will automatically detect and create Relationships for you. In situations where more granular control is required, these Relationships can be specified manually if needed. It is worth keeping in mind the following important features of Relationships:
    • They support one-to-one (1:N), one-to-many (1:N) and many-to-one (N:1) cardinality, with many-to-many (N:N) currently in preview.
    • Filter directions can be specified either one way or bi-directionally.
    • Only one relationship can be active on a table at any given time.
  • It is possible to sort columns using more highly tailored custom logic via the Sort By Column feature. The most common requirement for this generally involves the sorting of Month Names in date order but can be extended to cover other scenarios if required. To implement, you should ensure that your data has a numbered column to indicate the preferred sort order.
  • Moving outside of the Power Query Editor presents us with more flexibility when it comes to formatting data to suit particular styling or locale requirements. While the majority of this functionality provides date/time and currency formatting options, for the most part, it is also possible to categorise data based on Location, the type of URL it is or on whether or not it represents a Barcode value; these options can assist Power BI when rendering certain types of visualizations.
  • There may be ad-hoc requirements to add manually defined data into Power BI – for example, a list of values that need linking to a Slicer control. The Enter Data button is the “no-code” route to achieving this and supports the ability to copy & paste data from external sources. For more advanced scenarios, you also have at your disposal a range of M code functionality to create Lists, Records and Tables, which can be extended further as required.
Create calculated columns, calculated tables, and measures
Skills Measured

May include: Create DAX formulas for calculated columns, calculated tables, and measures; Use What If parameters

Revision Notes

Exam 70-778 Revision Notes: Using DAX for Calculated Columns

Key Takeaways
  • DAX is the primary formula language when working with datasets outside of Power Query. It includes, to date, more than 200 different types of functions that can assist in all sorts of data modelling.
  • An important concept to grasp within DAX is context and, specifically, row context (formulas that calculate a result for each row in a dataset) and filter context (formulas that automatically apply any filtering carried out at report level).
  • The sheer amount of DAX functions available makes it impossible to master and remember all of them, particularly when it comes to the exam. Your learning should, therefore, focus on learning the general syntax of DAX and the general types of functions available (aggregation, date/time etc.)
  • There are three principal means of utilising DAX with Power BI:
    • As Measures: These typically present a scalar value of some description, often an aggregation or a result of a complex formula. Using them in association with a Card visualization type is recommended, but this is not a strict requirement.
    • As Calculated Columns: Similar to the options available within Power Query, Calculated Columns provide a dynamic and versatile means of adding new columns onto your datasets. Compared with the options available within Power Query and the complexity of the M language, DAX Calculated Columns might represent a more straightforward means of adding custom columns onto your datasets.
    • As Calculated Tables: A powerful feature, mainly when used in conjunction with Calculated Columns, you have the ability here to create entirely new datasets within the model. These will typically derive from any existing datasets you have brought in from Power Query, but you also have functionality here to create Date tables, sequence numbers and manually defined datasets as well.
  • What-if Parameters provide of means of testing DAX formulas, as well as allowing report users to perform predictive adjustments that can affect multiple visualizations on a report.
Measure performance by using KPIs, gauges and cards.
Skills Measured

May include: calculate the actual; calculate the target; calculate actual to target; configure values for gauges; use the format settings to manually set values

Revision Notes

Exam 70-778 Revision Notes: Utilising KPIs with Gauge Visualisations

Key Takeaways
  • There are two principle visualization types available within Power BI to help track actual-to-target progress – KPIs and Gauges.
  • KPIs provide a more visually unique means of a binary success/fail determination when tracking towards a target. It is also possible to use KPI’s to track variance over time via the Trend axis. The Indicator will typically be the result of some form of aggregation or Measure.
  • Gauges provide a less visually distinctive, but non-binary, mechanism of viewing progress towards a target. Gauges support more potential field well values when compared with KPIs, nearly all of which are optional in some way. You can also manually define some of these values, for situations where your data model does not contain the required information.
  • All visualizations within Power BI are modifiable from a display or formatting perspective. The same basic options will generally be supported – such as changing a font type or background colour – with more specific configuration properties available per unique visualization type. For example, a KPI visualization can be customised to hide the background Trend Axis entirely. All of these options are designed to give developers greater control over the look and feel of their reports and to mirror them as closely as possible to any potential branding requirement.
  • When building out a solution designed to monitor progress within Power BI, the steps involved will typically be more in-depth than merely creating a new visualization. In most cases, there will be a requirement to bring together a lot of the other skills that have been discussed previously within this series – such as creating DAX formulas, modifying data within Power Query or bringing together different data sources into a single model. It is essential, therefore, not to underestimate the amount of time and effort involved in creating a practical solution that takes advantage of KPIs or Gauges.
Create hierarchies
Skills Measured

May include: Create date hierarchies; create hierarchies based on business needs; add columns to tables to support desired hierarchy

Revision Notes

Exam 70-778 Revision Notes: Creating Hierarchies

Key Takeaways
  • Hierarchies within Power BI provide a means of logically categorising data into an order of preference or precedence, providing greater flexibility to Power BI report users when they interact with visualizations.
  • Date Hierarchies are created and managed automatically by Power BI for each Date or Date/Time field defined within your model. These automatically create fields that contain the Year, Quarter, Month & Day values from the respective date fields. These fields can then be utilised as part of a Table visualization or within a DAX formula. Date Hierarchies can also be completely disabled if required.
  • Custom (or User-Defined) Hierarchies need to be created manually and provide additional customisation options when compared to Date Hierarchies, such as the number of fields they contain, the order and its name. A Custom Hierarchy will typically make use of one of several Parent/Child DAX functions, such as PATH or PATHITEM.
  • Including a hierarchy as part of a chart visualization, such as a Pie chart or Donut chart, opens up other drill-down capabilities around your data. Indicated by the additional arrow icons included at the top of the visualization, they provide the means for users to interrogate data points that interest them the most straightforwardly.
Create and format interactive visualizations.
Skills Measured

May include: Select a visualization type; configure page layout and formatting; configure interactions between visual; configure duplicate pages; handle categories that have no data; configure default summarization and data category of columns; position, align, and sort visuals; enable and integrate R visuals; format measures; Use bookmarks and themes for reports

Revision Notes

Exam 70-778 Revision Notes: Create and Format Interactive Visualizations

Key Takeaways
  • Power BI delivers, out of the box, a range of different visualizations that cater towards most (if not all) reporting requirements. Should you find yourself in need of additional visualizations, then Microsoft AppSource is your go-to destination for finding visualizations developed by others. If you have experience working with either Node.js or R, then these can be used to build bespoke visualizations also.
  • When first developing a report, you should be able to match a requirement for a specific visualization type, to ensure that you are delivering a solution that is both meaningful and useful. From an exam perspective, this becomes a more critical consideration, and you should be prepared to suggest the most optimal visualization to use when given a specific scenario.
  • After adding visualization’s to your report, you have additional options available to customise them further. For example, you can specify a preferred sorting order for your data, override any summarizations used and move/align your visual on the report page.
  • By default, visualizations in Power BI are designed to change automatically, based on how users interact with the report. All of these options are controllable via the Edit interactions button, allowing you to specify your preferred cross-filtering and cross-highlighting conditions.
  • There is a range of report page customisation options available to developers. It is possible to resize a page to any possible height/width, allowing you to optimise your report for specific devices. Also, you can modify the colour of a page (or its wallpaper) or add an image instead. Pages can also be renamed, reordered or duplicated.
  • Measures can be formatted in the same way as calculated columns, meaning you can specify a data type or, for numerics, modify the number of decimal places.
  • Bookmarks allow developers to set up “checkpoints” within a report, based on how a report page has been filtered. These can then be used to automatically navigate the user through a report, applying these filtering steps automatically. This feature can help transform your report into an interactive story.
  • Visualizations will automatically inherit their various colour properties from the currently selected report theme. Although these can be modified granularly, the fastest and most consistent way of making these changes en masse is to change the Theme. Power BI includes some Themes out of the box, but you also have the option of building your own using a custom JSON file; this can then be imported into different reports, providing a portable means of enforcing a particular branding requirement.
Manage custom reporting solutions
Skills Measured
  • May include: Configure and access Microsoft Power BI Embedded; enable developers to create and edit reports through custom applications; enable developers to embed reports in applications; use the Power BI API to push data into a Power BI dataset; enable developers to create custom visuals
Revision Notes

Exam 70-778 Revision Notes: Managing Custom Reporting Solutions

Key Takeaways
  • Power BI Embedded is an Azure hosted offering that allows you add Power BI Report content into bespoke applications. This deployment option can be incredibly useful if you wish to make available your Power BI solution to users outside of your organisation or if you have an existing, bespoke application system that can benefit from utilising Power BI content. An Azure subscription is required to begin working with Power BI Embedded and you are billed based on node size, not individual user licenses. All Power BI content requires publishing to the online service before its contents become available for Power BI Embedded to access. Report developers will, therefore, need granting a Power BI Professional license to carry out these activities.
  • The Power BI API grants access to developers to perform automation or administrative actions programmatically against the Power BI Online service. Utilising a REST API, developers can determine the optimal programming language of choice to interact with the API, allowing them to streamline the deployment of Reports or Dashboards to the Power BI service or leverage additional functionality when utilising Power BI Embedded. The API can also cater to specific data load requirements, although more complex needs in this area would require addressing via alternate means (SSIS, Azure Data Factory etc.)
  • Developers can add their own bespoke visualizations to a Power BI Report by either developing them using Node.js or using the R language. The first of these options facilitate a more streamlined deployment mechanism and allows developers to add their visualizations to AppSource, whereas the second option may be more useful for complex visualization types with an analytical or statistical function.

Configure Dashboards, Reports and Apps in the Power BI Service

Access on-premises data
Skills Measured

May include: Connect to a data source by using a data gateway;publish reports to the Power BI service from Power BI Desktop; edit Power BI Service reports by using Power BI desktop

Revision Notes

Exam 70-778 Revision Notes: Report Publishing, On-Premise Gateway & Creating Dashboards

Key Takeaways
  • The Power BI On-Premise Gateway provides a streamlined route to working with non-cloud data sources within Power BI, Microsoft Flow and PowerApps. As a lightweight and easy-to-configure client application, it supports a wide variety of data sources, making them accessible as if they were in the cloud. Once set up, corresponding Data Sources are then made available for configuration and for leveraging as part of any Power BI Dataset.
  • Reports can be published into Power BI Online, meaning that they become accessible online and to a broader group of users, without requiring access to Power BI Desktop. Reports need deploying into a Workspace, which can be created manually or derived from an Office 365 Group. Each Report contains a corresponding Dataset, where all queries defined within Power BI Desktop exist.
  • Reports that already exist on Power BI Online can be updated by just publishing a new version of the Report from Power BI Desktop. It is also possible to modify Reports from directly within the browser and by downloading a copy of the .pbix Report file as well, which can then be altered and re-published.
Configure a dashboard
Skills Measured

May include: Add text and images; filter dashboards; dashboard settings; customize the URL and title; enable natural language queries

Revision Notes

Exam 70-778 Revision Notes: Report Publishing, On-Premise Gateway & Creating Dashboards

Key Takeaways
  • Dashboards provide a means of grouping together various content as tiles, designed for at-a-glance analysis and optimal end-user experience.
  • The list of content that can be pinned to a Dashboard includes:
    • Visualizations
    • Web content
    • Images
    • Text boxes
    • Videos
    • Custom streaming data
  • Pinned content can be re-arranged on Dashboard via drag and drop functionality. It is also possible to resize tiles to any given height/width.
  • Within the settings of a Dashboard, it is possible to enable/disable features such as natural language queries (Q&A’s) and Notes.
  • Some features of a Dashboard are only available if you have a Power BI Professional subscription, such as sharing and email subscriptions.

 

Publish and embed reports
Skills Measured

May include: Publish to web; publish to Microsoft SharePoint; publish reports to a Power BI Report Server

Revision Notes

Exam 70-778 Revision Notes: Publish and Embed Reports

Key Takeaways
  • The Publish to web option allows for non-licensed, external users to view a Power BI Report in its entirety. A URL and IFrame embed code can be generated for this at any time within the portal and then dropped into virtually any website. Although you will lose some functionality when deploying a Report out in this manner, you can expect that users will be able to perform most types of interactions with visualizations, Report pages and other components, as if they were accessing the Report through Power BI Online. In some cases, you may be unable to use the Publish to web option if your Report uses certain kinds of features, such as R Visuals or row-level security. You must also take into account any privacy or data protection concerns, as Reports deployed out in this manner will be publically accessible; where this is an issue, the Embed option is available as a secure alternative.
  • There are three steps involved if you wish to add a Report to SharePoint. First, you must generate the unique SharePoint embed URL within Power BI. Secondly, you then need to add on the dedicated control for this feature on your target SharePoint page and configure the relevant display options. Finally, you then need to ensure that all SharePoint users have been granted access to the Report, either at a Workspace level (recommended option) or by having the Report shared with them. By implication, in this scenario, all SharePoint users would have to have at least a Power BI Professional license to take full advantage of this functionality.
  • Publishing a Report to Power BI Report Server is mostly the same as if you were to do the same with the online version of the product. Instead of selecting a Workspace to add the Report to you, specify the name of the Report Server folder where the Report will reside. From a development standpoint, the dedicated Power BI Desktop for Power BI Report Server must be used and may differ in functionality from the “normal” version of the tool. There is also no option to edit a report from within Power BI Report Server like you can through the online version.
Configure security for dashboards, reports and apps.
Skills Measured

May include: Create a security group by using the Admin Portal; configure access to dashboards and app workspaces; configure the export and sharing setting of the tenant; configure Row-Level Security

Revision Notes

Exam 70-778 Revision Notes: Securing Power BI Dashboards, Reports and Apps

Key Takeaways
  • Workspaces act as a container for the various components that form a Power BI Reporting solution. Within a Workspace, you will find all of the Dashboards, Reports, Workbooks and Datasets that developers have published content to. Each User has a Workspace created for them in Power BI when they first access the service. Additional Workspaces can be added through Office 365 Groups or by installing a Power BI App from AppSource. Dashboards and Reports created within your a Users Workspace are shareable to other Users, provided that your account has a Power BI Professional license assigned to it.
  • To help manage permissions to Dashboards/Reports in a more efficient manner, Administrators can create Security Groups on the same Office 365 Tenant where Power BI Online resides. These can contain multiple groups of Users, allowing administrators to minimise the amount of effort involved in managing Dashboard/Report access. Most crucially, this will also enable Users that do not have an Exchange Online mailbox to access Dashboards/Reports when they are shared out in this manner.
  • Administrators have a whole host of options available to them within the Tenant settings area of the Admin Portal. These include, but are not limited to:
    • Export and Sharing Settings
    • Enable/Disable Content Sharing
    • Enable/Disable Publish To Web
    • Enable/Disable Export Reports as PowerPoint Presentations
    • Enable/Disable Print Dashboards and Reports
    • Content Pack and App Settings
    • Integration Settings
    • Custom Visuals Settings
    • R Visuals Settings
    • Audit and Usage Settings
    • Dashboard Settings
    • Developer Settings
  • All of these settings can be enabled for a specific security group, the entire organisation (excepting specific security groups) or allowed for particular security groups, excluding all others in the organisation.
  • Row-Level Security (RLS) allows report developers to restrict data, based on Roles. Row-level DAX evaluation formulas are used to achieve this, which filters the data that is returned, depending on a TRUE/FALSE logic test. To utilise the feature, you must define both the Roles and DAX formulas for each query within your data model. Then, after deploying your Report to Power BI Online, you then assign Users or Security Groups to the Role(s) created within Power BI Desktop. It is possible to view the effect of a Role at any time, within Power BI Desktop or Online, via the View As Role functionality. With the wide-array of DAX formulas available, including specific ones that return the details for the current user accessing a Report, it is possible to define very granular filtering within a Power BI report, to suit particular security or access models.
Configure apps and apps workspaces.
Skills Measured

May include: Create and configure an app workspace; publish an app; update a published app; package dashboards and reports as apps

Revision Notes

Exam 70-778 Revision Notes: Working with Apps and App Workspaces

Key Takeaways
  • Workspaces act as a container for the various components that form a Power BI Reporting solution. Within a Workspace, you will find all of the Dashboards, Reports, Workbooks and Datasets that developers have published content to. Each User has a Workspace created for them in Power BI when they first access the service. It is also possible to create additional Workspaces, either through the Power BI Online interface or by creating an Office 365 Group. A new experience for creating Workspaces is currently in preview which, once released, would negate the need for each Workspace to have an associated Office 365 Group.
  • When creating a Workspace, you can define various settings such as the type of access each user has (read-only or ability to modify its content), its members and whether it requires assignment to a Power BI Premium node. It is not possible to change the access type for a Workspace after creation, but you can freely change its name or modify its membership at any time.
  • The contents of a Workspace can be published as an App, enabling you to expose your solution to a broader audience within or outside your organisation. Once published, users navigate to the Power BI AppSource store for their tenant, which lists all Apps available for installation. Once installed, they will then become visible from within the Apps area of the application. You can update content within an App at any time by republishing its corresponding Workspace. It is also possible to define individual properties within an App, such as its description, access rights and landing page. To install and use Apps, the user in question must have a Power BI Professional license.

Additional Preperation Resources

The official Microsoft exam reference book is a helpful learning aid for this exam, particularly given that it includes numerous exercises that you can work through to familiarise yourself with different Power BI functionality. There is also an online course available on the edX website which, despite not covering the whole exam syllabus, does provide a useful visual aid and includes a lot of the features you are expected to know for the exam. Finally, nothing beats actually working with the product itself and trying out the various features yourself. Power BI Desktop is a free download and, with access to one of the sample databases provided by Microsoft, you can very quickly provision an environment on your own home computer to enable you to experience everything that Power BI has to offer.

Exams are always a nightmarish experience, both when preparing for them and when you are sat there in the test centre. I hope that this post, and this whole series, proves to be useful in helping with your exam preparation and getting you ready to pass the exam with flying colours 🙂