The data world is booming. According to QuantumHub, about 67% of companies surveyed have been expanding their data science services. In 2020, job postings grew a total of 37%, compared to the previous year. Forbes, moreover, reported that 54% of enterprises saw cloud-based BI as vital to their current and future initiatives; and according to Select Hub, the market value for applications in business intelligence was $17.7 billion in 2020.
This shows that not only have more and more organizations started to realize how important data intelligence is (in certain industries data intelligence marks the difference between life and death), but they have also been actively working on expanding and improving their data intelligence capabilities. Gartner, for example, reports that more than a third of large organizations will have analysts practicing decision intelligence by 2023.
With the growing importance of having a robust Data and Analytics (D&A) department, organization leaders are looking for ways to organize and structure these departments.
In our experience, the organizations traditionally tackle this challenge in one of the following ways:
They create a separate D&A department;
Each department in the organization develops its own D&A capabilities; or
They structure their D&A requirements and capabilities according to a particular project.
Each approach has clear advantages. They also, however, have weaknesses that may be detrimental for the organization, depending on what they seek to achieve.
To help leaders facing the challenge of structuring their D&A capabilities, we will take a closer look at the approaches below in an effort to discover where they might prove most beneficial, if implemented correctly.
We will round off this examination with a proposal for a more efficient data-intelligence approach. This product-focused approach has already been proven to deliver positive results for organizations of various data-intelligence needs and expectations. It is agile, result focused, and manageable. This approach allows organizations to tackle the business-critical missions first, test various hypotheses quickly, and launch valuable and impactful data products in a matter of just a few months.
How to organize your D&A department?
Data-intelligence needs are growing in existing business departments spanning from finance, to supply chain, and even into HR. All of these departments require quick and reliable access to insight. The question, therefore, of how to structure data-intelligence capabilities often comes down to the question of which department data science should report to.
This question tends to be the determining factor when organizations structure their D&A teams. Determining reporting lines of newly-assembled data teams is critical, but locking the structure of this function prematurely can lead to bad results.
In addition to reporting lines, organizations typically consider two factors when structuring their D&A teams:
How large a D&A team needs to be; and
What the overarching business strategy for the organization is (going to be).
This has led to three different approaches to structuring D&A teams in organizations.
Separate D&A department
In our experience, this is one of the most frequently used approaches. A separate D&A department is usually built as an alternative or extension of the existing BI department by adding advanced analytics capabilities. The department then continues to report to the CIO organization.
The main benefit of this approach is that the department is well protected under the CIO organization which has a direct link to the CEO. This is especially important if process changes are needed in the company. Having the attention of the CEO and top leadership makes it easier to unlock funding and drive bigger projects with higher value potential. If there have been failed digital initiatives in the past, having a reputable Chief Analytics Officer from a renowned company can add credibility to the department.
Normally, a central D&A department has enough critical mass to attract the top talent in areas such as artificial intelligence and machine learning. A larger team, greater budget, and an expanded project portfolio make for an attractive proposition for many senior experts and leaders in the data industry today.
On the other hand, standalone data analytics departments can be problematic precisely because they tend to prioritize bigger, oftentimes structural projects. These projects need them to focus heavily on savings and ROI. As a result, they move incredibly slowly. Management in the operational parts of the company, however, needs fast and effective insights to deliver on the day-to-day requests from clients and other stakeholders.
Another challenge is scaling. Due to increasing demand for data intelligence across various business functions, centralized D&A teams are constantly grasping for resources. D&A therefore goes from being a solution, to hindering digital innovation. Self-service analytics and local teams, of which there are usually many, help scale data intelligence in an organization. Unfortunately, our experience shows that central D&A managers often prioritize their own projects, instead of supporting analytics teams with better processes, data and technology. In fact, we have even seen D&A teams running company-wide projects behaving like a competitor, rather than as a business partner.
For these reasons, we advise caution in implementing a stand-alone D&A team. We suggest delaying implementation of a team until there is a strong analytics community in place with a shared vision (more on this later).
Another approach we often see organizations take is to simply allow each department to organize their own data science and analytics efforts. This approach has two significant advantages (especially when the organization is at the beginning of their data-intelligence journey): speed and experimentation.
When analytics teams are close to the business, they have a deeper understanding of the needs and they can act faster. With trust and communication already in place, iterations can happen a lot faster. Since these teams are fully aligned with business, they also better understand the end goals and tend to spend more time figuring out and experimenting with solutions to best support their stakeholders’ objectives.
On the other hand, challenges with the laissez faire approach arise in two areas: 1) maintenance and 2) scaling.
Maintenance becomes a prominent issue after the initial stage of a project is completed. Line of Business (LOB) D&A teams tend to move quickly. For this reason, the teams don’t go into production with best practices. It is common for such projects to fall apart after a few years, usually when a key member of the team leaves.
Poor compliance with good practices coupled with a lack of funding give rise to the second challenge — scaling. Local teams often don’t have enough resources to invest either into proper adoption and training strategies, or into productionalization of their systems. Meaning that they are unable to accept waves of new users.
The Laissez-Faire approach is a good starting point and in most scenarios more adequate than heavy top-down structures. Its drawbacks, however, become pronounced when it comes to maintenance and company-wide impact of decision-intelligence projects.
This approach happens across the board and is not specific to D&A. It can have, however, an especially negative impact on D&A due to the exploratory nature of analytics. The strategy behind this approach is securing funding for a particular D&A project, based on estimated return on investment or other financial KPIs.
Any analytics team, no matter the size or position in the company hierarchy, could be subject to this approach. From this point of view it is the most democratic way of handing over budgets for analytics.
The challenge with funding projects is the lack of an overall vision for data intelligence and collaboration across departments and units. This way of approaching digital innovation could be compared to raising skyscrapers without any city planning. The skyscrapers might be innovative and modern, but the city would lack basic necessities, such as public transportation or a sewage system.
Potential Solution: Lean Approach
Although the aforementioned approaches can all deliver reliable and impactful results when implemented correctly, we’ve experienced time and time again that “proper-implementation” efforts can be strenuous for all parties involved. For that reason, we propose an approach to structuring D&A teams that most enterprises are already well-familiar with, one that has been proven to be successful and is integral to the production process.
The Lean Approach is based on an existing solution for innovation – product innovation. It takes the same principles used to build and sell products, and applies them to digital products intended to be used within the company.
About Lean Approach
The Lean Approach follows a trend that is revolutionizing the IT industry: Software as a Service (SaaS). SaaS companies are changing the industry with their simplified subscription pricing, scalability, and marketing. We see significant benefits to this approach and applying it to internal programs as well. The following aspects are especially relevant:
Product based approach, adding new features often in order to quickly learn what adds value and avoid lengthy theoretical debates.
Subtly adding AI into workflows over time in order to gain quick adoption. For inspiration, think of how easy it was to start using google maps recommendations.
Fixation on user experience to gain traction and adoption.
Well defined value proposition, effective copywriting, and strong marketing in order to clearly explain the benefits and unlock funding approvals.
Simple pricing which covers everything users don’t care about (for example, infrastructure) in order to focus on business value and avoid distracting debates or micromanagement.
Figure 1: At dyvenia, we follow Evaluation – Startup – Scale approach that entails the phases and outcomes described on the picture above.
In our experience, it is easier to structure D&A around these three phases. With these phases, D&A can quickly gain traction and reputation across the company.
In this approach, D&A acts more like a Center of Excellence than a single top-heavy department. D&A can still report into the CIO organization but it will keep one foot in the business as well. With this operating model, the strategic conversation will revolve around evaluation, startup and scale of data products, while being end-user centric.
A word on infrastructure. This model emphasizes data products because they drive traction, receive funding, and are easier to communicate well. IT and technologists, however, still have to work in the background to modernize infrastructure. Infrastructure can easily be modernized using Cloud and Infra as Code solutions. Yet, it still requires effort and resources. Our recommendation is to include all the infrastructure costs in the price of each data product to the business. Of course, this depends on company culture, but we generally find it more challenging to get infrastructure funding approved on its own.
Less organizational headaches. Under this approach, it is still advisable to have a central D&A department, but D&A’s role is to act as a portfolio owner and enabler. It is not supposed to compete with existing analytics teams, but rather to enhance collaboration success. Incentives are aligned because when a digital product succeeds and delivers, the local team gets a win and D&A receives a positive impact in their portfolio.
Experimentation done in the right way. Following “The Lean Startup” (link to book) methodology, the laissez-faire approach would not be a scaling bottleneck, but simply the “startup” phase of an MVP (Minimum Viable Product). During the MVP stage, the only goal is learning the needs of the end users as well as if / how they use the digital product.
Easier funding approvals. Finally, funding is a lot easier with (digital) products. Business managers can see a direct link between a digital product and its impact on their KPIs, so they will approve budgets more easily. Product development costs also decrease as scale and adoption rates increase. This generates savings to be re-invested in less visible but still important aspects of D&A, including data governance, quality and catalogue efforts.
There is no silver bullet but there is a silver direction
The Lean approach is not the only solution to all problems D&A face. There are situations where it makes sense to go with a traditional approach.
At the same time, if an enterprise has a history of central funding approvals, the product-driven approach might be difficult to implement at the beginning. In this case, it is usually better to start with a standard centralized budget approach to gain reputation for the team and then try to change the approach later.
Finally, the product view can make people forget the less visible initiatives that are still critical to support the growing portfolio of digital products. Infrastructure, data governance, and security are all things to be managed in parallel, but it would be a mistake to try to make business partners care about those. In all fairness, when was the last time someone asked about how their car was produced? It is assumed that all the “boring” things are included in the final price. It takes strong and savvy leadership to price products not just for the visible features like development hours, but also for all the other less visible aspects that still need to be properly built and managed.
Data & Analytics needs are growing across all companies and departments, but the industry is still fragmented and lacks established standards. In the search for solutions, company leaders are evaluating different options (some better than others). But as a whole, there is still a lack of cohesive vision for data.
Companies are familiar with products. After all, it’s what they sell to stay in business. So why not apply some of these familiar processes to D&A? Startups are achieving high growth at record speeds by following a simple product-based approach in developing their business.
There are challenges with the product-based approach for data-driven digital innovation, especially around challenging established practices such as centralized funding approval. The trend, however, favours well scoped, well defined, digital products, which is especially clear when looking at the SaaS industry.