User Experience Test with Power BI

As an entrepreneur who makes a living in analytics, it is crucial for my company to keep up with the best tools on the market. One of the most important tool categories for my work is data visualization. My tool of choice for the last six years has been Tableau.

That said, for the past 18 months, I have been occasionally testing Microsoft Power BI. You may have seen them appear in the upper right quadrant of the last Gartner report. Backed by the budget size that Microsoft can offer, and pricing themselves below Tableau, I can see it being a matter of time before Power BI competes head-to-head with Tableau.

Gartner Report – February 2017

With a change in position on the Gartner report, I felt it was time to yet again head into Power BI to test the user experience and capabilities. Keep in mind that the axes on the Gartner report are “completeness of vision” and “ability to execute.” It doesn’t measure the existence of certain capabilities and the customer experience. My benchmark for testing is my experience with Tableau Desktop Pro, Tableau Online and Tableau Server.

My previous tests of Power BI have always left me feeling very restricted in capabilities. In these cases, I was testing Power BI “Services” which is the online service tool. As an additional note, they also offer a desktop version.

Microsoft Power BI Services (Online Tool)

Heading back into the online service version a few days ago, I had renewed hope of capabilities based on the Gartner report. The first thing I noticed in comparison to Tableau is that the service connections with Power BI offer more on the social side. For example, Power BI says it can connect to a MailChimp account (email campaigns) to create a dashboard for email campaign statistics. I thought I’d start there.

Selecting the connection for MailChimp in the Power BI service was easy. I selected my simplest MailChimp account with one list in it. The connection was a success and to Power BI’s credit, it created a default dashboard and report. In Tableau, I would need to build my first visualizations from scratch.

I continued to Test Number 2 which was to switch the credentials (ID / password) of MailChimp to my Numerical Insights account. That account houses several lists, is much larger and has more activity (subscribes, unsubscribes, email bounces etc.). I switched the credentials and… Power BI still showed me the MailChimp data, report and dashboard from Test Number 1. I tried again and… a connection error.

I submitted this error to the Microsoft Power BI community and the response that came back was that you have to delete everything you have (MailChimp data, report and dashboard) and start over again. Did that work? Yes. Was it a good user experience? No. Do I consider that a “solution?” Definitely not.

Microsoft Power BI Desktop

The second response from the community was to use the Power BI desktop version. The community claimed that switching the credentials of the MailChimp account (ID / password) was possible on the desktop tool and that the desktop tool had greater capabilities. So, I headed in that direction.

About a year ago, I installed Power BI desktop on a Windows 7 laptop without any issues. I could easily connect to data and create basic dashboards. I could share dashboards with other people, as long as they didn’t have an email address which was designated as personal. The frustration here was that I couldn’t share a dashboard example with a small client who used as her official business email. I did have to laugh a bit since locking out is Microsoft locking out one of its own domains.

Today’s test was on a new Windows 10 laptop. I downloaded the desktop version and the installation produced no errors. However, launching the program was an entirely different story. A box opens, saying “Initializing model” and then I am left with a black box with some indecipherable text in the upper-right corner. At this point, the program is frozen and must be closed within the task manager.

The response from the Power BI community? It mentioned changing settings of GPU rendering within Internet Explorer (for some reason the program pulls its settings from this obsolete browser), stopping Nahimic for MSI if you’re running an MSI laptop (say what?), and installing the 32 bit version instead of the 64 bit.

I tried the first and third suggestions and managed to get a box to appear which allowed me to sign into my Power BI account. Sadly, it returned me to the same black screen. Clicking in the “blackness” at other locations revealed another menu. Sadly, I have to guess on this black screen where the menus might be. Clearly I have some sort of display issue.

I decided to attack this problem with the process of elimination. I am running two screens: one on my laptop and another with an external display. Each has a different screen resolution. Sure enough, if I disconnected the external display and then launched Power BI, it displayed correctly. So, the key to making Power BI work for me, is to unplug my monitor, launch Power BI and then plug my monitor back in. I suppose that with two different screen resolutions, some of my pain is self-inflicted, but surely I’m not the only customer with this set-up.

At this point, I headed back to my MailChimp test to run the same test that I had tried online. In the desktop version, a MailChimp connection is clearly labeled (beta). If I’d seen that label in the online version, I would have been more forgiving in my assessment of that tool.

I managed to connect to my simplest MailChimp tool. In the online version, I was spoiled by being handed a dashboard of my MailChimp information already made. In the desktop version, this doesn’t happen. No problem there since building visualizations from scratch is what I’m used to.

I headed into the test of changing the MailChimp credentials, i.e., changing the ID and password over to my more complicated MailChimp account. If you recall, the reason I decided to install the desktop version was to get around a lack of functionality in the online version.

I found where to change these credentials pretty easily and asked Power BI Desktop to refresh my data view. Unlike the online version, the desktop version was successful in this task. I could see a 5 year history of all of my email campaigns.

At this point, I had used up my allotted testing time. I have seen enough that I know that the online version is not something I would recommend to my clients. As for the desktop version, I do plan to continue testing and will track the results against Tableau.

I am convinced that Power BI will morph into something impressive, but I’ll need to give it more time. I promise to conduct more testing in the future now that I have resolved my desktop installation display issue.

Until next time,


Tracey Smith is an internationally recognized business author, speaker and analytics consultant. She is one of the most highly respected voices when it comes to business analytics and HR analytics. She is the author of multiple business books and hundreds of articles in a variety of publications. Tracey has worked with and advised organizations, both well-known and little-known, on how to use data analytics to impact the bottom line. If you would like to talk to Tracey about consulting work or speaking engagements, please visit or contact Tracey Smith through LinkedIn.

Visit Tracey Smith’s Amazon Author Page