Data Warehouse and BI Support Organization: Functional or by Client Area?

If your organization is like the one I grew up in, no doubt there is some undulation in organization structure. Maybe things started off in the old "strategic business unit" structure where every business unit had just about every function except finance and HR. Then over time, as your company became larger, went public, felt a budget pinch, or whatever, they decided to go to a more functional organization to eliminate redundancies in areas such as marketing and possibly even some IT functions.

I promised to stay business-focused with this blog and I will. But as business people we are often engaged in the debate over where the data warehouse effort will live. Perhaps historically in your company, anything that smelled like IT is functionally organized in IT and prioritized through a corporate prioritization process. My bet is that these resources are scheduled for the next 3 years and not able to respond to your business needs with the speed you would like. Many times what results from this situation is data efforts, large and small, sprouting up throughout the company, from little Access databases to full-scale data warehouses owned and supported by business areas. Necessity is the mother of invention, but eventually these efforts will capture the attention of IT management and the debate (war) will be on.

I will place a stake in the ground and say the best scenario is something of a compromise. It is best for a data warehousing group to be centralized within IT, but to have a "customer-centric" organization. There is much to be supported: hardware, nightly ETL loads, metadata, end users, security, training, etc. It is impossible to ensure "one version of the truth" with data marts distributed throughout the organization unless they are connected to an overall architecture and it is very difficult to support these functions within a business department or even a business division. While business people may have the vision for how the data will be leveraged, their IT counterparts bring the special knowledge of what is involved in supporting such a major initiative and should be at the table to offer this value.

That said, I think that somewhere between a fully-centralized, functional approach and a siloed, isolated data mart is an enterprise-wide effort that is organized to quickly address immediate needs of business clients. In order for a data warehouse to be supported by the business, it must remain more than relevant, its relevance must be on the order of oxygen for the business users. And in order to remain that relevant, the data warehouse must be able to address the business needs swiftly and nimbly.

Exactly what that organization will look like will vary from company to company, but generally speaking, the subject matter experts--managers and analysts in each business area--need some control over priorities for the business intelligence resources. They also need to be very involved as a group in the overall priorities for how the data will be developed at the warehouse level, so they understand how their priorities will fit into the overall plan.

Like I said, how deeply into the process this client-focused organization would go varies by company, but in most cases even down to the ETL development and support level there will be some datasets that are particular to a business client and should be organized to support that particular client, balancing swift attention to business priorities with ensuring that the data does not become siloed. The result is a sort of cross-functional, client-focused team that lives in the IT area, is managed by IT managers, but acts like part of the business customer's team.

Additionally, as much of the report development that can be handed off to business analysts should be. These are usually cheaper resources than BI developers found in IT and can be directly driven by their manager's priorities. The important thing is to ensure that reports are properly documented.

The advantages for the manager and business analyst are clear. The advantage of an organization like this for IT management is that they get to balance the immediate needs of their business customers (whose support they need) with building the enterprise data warehouse, which is a constantly moving target with new systems coming in, old systems being retired, and new overall strategic initiatives coming into play every day. Finding a balance that takes all these considerations into account requires political savvy and a deep organizational knowledge of what will work organically within your company.