Big data in the project process - effects on the project business of tomorrow

Picture of Helmut Schäfer

Helmut Schäfer

January 12, 2018

The digitalization of society and the economy is generating an immense stream of data from which a multitude of new types of information can be obtained. The term big data stands for the processing, analysis and evaluation of this mass and real-time data. The first companies have recognized this opportunity and are looking for ways to use the knowledge gained in a targeted manner. This article analyzes the role of big data in the project process and presents the - current and future - interaction between information and its potential. It concludes with a list and explanation of five theses on the influence on the project business and expected developments with regard to big data in the project process of the future.

Customized services from the management perspective

The characteristics of uniqueness and novelty set out in the general definition of a project are only partially correct for the project business. These two characteristics certainly apply to the majority of development projects, organizational or IT projects. However, different rules apply to customer projects. In companies that are active in the project business, a more or less large proportion of customer orders are provided through customized services in a project. Products, solutions and services are combined in such a way that the contractually agreed properties, deadlines and costs are met at the end of the project. The project goals of content/quality, deadlines and price are bindingly defined in the contract. Achieving a positive financial result is the top priority. In order to achieve this goal, efforts are made to standardize methods and processes at all levels and at the same time to make them more efficient.

With regard to process optimization in the project business, a distinction is made between the following two types of process:

  • The project process serves directly to specify and establish the scope of delivery and services of a project. The definition of this process depends on the type of service and usually also on the industry in which the company operates.
  • Project management processes are used to plan, implement and organize project work. The methods to be used in these processes are largely independent of the project's scope of delivery and services. Depending on the project category (e.g. simple, medium, complex), they can take different forms.

Increasing digitalization will affect both value chains. radically change. Both the project processes with the linked specialist processes and the Project management processes will be put to the test and redesigned if necessary.

Digital into the future - big data as a key driver

Even medium-sized projects leave behind an unmanageable amount of data. Today, everything is documented digitally and stored in structured form in the corresponding systems. However, this structured data only represents a fraction of all existing data. Far greater is the proportion of unstructured data that is generated in the form of log files, sensor data, images, audio or video recordings, emails, web texts, GPS data, etc. by the systems involved, such as intelligent networked products ("Internet of Things"). This data is often collected randomly or incidentally and deleted as soon as it is no longer needed. Big data is used to store this mass data in order to investigate it further with regard to possible causal relationships. The increasing use of mobile devices and their specialized apps in conjunction with technologies such as Cloud computing are causing this data to increase exponentially.

Digital innovations, information in interaction, classic company data, digital data world, computing power, Internet of Things, additive manufacturing, robotics, virtual reality, sensor technology, data analytics, networking, cloud computing, mobile internet, social networks, artificial intelligence, big data in the project process
Figure 1: The interaction of digital innovations. (Image: © Helmut Schäfer)

The last example shows how the development and, in particular, networking of innovative technologies is constantly creating new, completely unimagined possibilities that surpass everything that has gone before in terms of speed and impact.

Big data, i.e. the exponentially growing volumes of data and their handling, the constant further development of digital technologies and, last but not least, their networking form the core of the digital transformation. Figure 2 (see below) summarizes this situation.

If the project management of customer projects is viewed as the management of deviation, the development of big data analytics is particularly important. It creates completely new dimensions for data evaluation. The methods that contribute to gaining knowledge and providing a preview are particularly interesting for project management. These include methods such as

  • Diagnostic analytics ("What is happening right now?")
  • Predictive analytics ("What will happen?") and
  • Prescriptive analytics ("What should happen?").

On the basis of information obtained in real time, it will also be possible to make short-term forecasts. If you know at all times what is happening in the project and why it is happening, and can also make reliable predictions about how the project will develop, the planning and control processes in the project will change fundamentally. If you then consider that decisions can also be automated, it becomes clear why the importance of these approaches for the project business cannot be overestimated. Especially in the Project Management It has long been believed that many processes can only be automated to a limited extent due to the complicated decision-making situations. However, more and more examples from the field of artificial intelligence show that this could increasingly succeed.

Descriptive analytics, diagnostic analytics, predictive analytics, prescriptive analytics, big data in the project process
Figure 2: Big data analytics. (Source: Gartner)

On the other hand, there are new approaches for simulating and forecasting project events. New insights can be gained on the basis of patterns and dependencies, allowing project processes to be designed more effectively and efficiently.

Big data in the project process: practical examples

Two examples of big data analyses in the project process will illustrate this:

In the first example a research project at the Vienna University of Economics and Business Administration investigated the extent to which a project employee assigned to a specific task also acts in accordance with the defined project role. This involved analyzing what the employee communicates about their task in emails and which artefacts they have actually processed. The underlying processes were described and the associated event logs were determined using data mining.

In the second example the communication behavior of teams was investigated at MIT in Boston. To this end, they were equipped with electronic sensors that were used to analyze three factors:

  1. The frequency and nature of the exchange of ideas
  2. The integration of individual team members into the team
  3. The team's communication with other teams.

The results of the analysis show that the manner of communication is more important for team success than the content. Even if this data does not (yet) come from a big data analysis, it is easy to imagine that this will be possible in the near future.

The two examples were chosen because they deviate from the usual pattern of a project analysis and thus show the still undreamt-of possibilities that lie in the linking of structured and unstructured data.

Looking ahead - effects on the project business

If we look to the future in the course of the developments of big data in the project process, one question arises above all: What effects will the developments of the digitalization described above have on the implementation of the project business? The following five exemplary theses will be used to discuss this development.

1. the degree of complexity determines the approach.

The project business is characterized by the fact that the way in which projects are handled is standardized. In the course of the discussion about agile approaches, the realization is now gaining ground that the complexity of a project must be better taken into account. Complex problems must be solved differently than complicated ones. The "Cynefin framework" helps to select the right process model. This means that, depending on whether the project situation is complex or complicated, a decision is made on how the teams should be set up, what type of process model they should use and what tools they should use.

2. proactive action is becoming even more important.

Big data creates the conditions for a paradigm shift in project planning and management. As a result, the focus of project status is no longer on looking back, but increasingly on predicting future developments. Based on constantly updated information and predictive analysis methods, proactive action can be taken instead of just reacting to deviations.

3. real-time data intensifies decision-making processes.

Evaluations that took days or weeks in the past can be available within minutes. Continuous real-time information is visualized in dashboards. Critical project situations can be quickly identified and overcome using early warning systems. Thus the abilityThe ability to obtain information and react quickly has become an important differentiator in the project business.

4. automation will continue to increase.

The findings are largely obtained automatically and/or the knowledge is applied directly in automated processes. Machine learning often plays a central role here. Standard reports can be generated automatically or, in more complicated cases, supported by decision routines. The influence of pure expert estimates is declining.

5 Process management becomes a key concept.

In order to better understand the interrelationships, the methods of business process management (BPM) and workflow management are increasingly being used. Project management and process management are not seen as competing disciplines that are applied depending on the business case: Rather, these methods complement each other, whereby synergies can be created to a high degree. Project planning is based on process models that are selected, adapted and compiled in a project plan according to requirements. Process management ensures that the appropriate sub-processes are provided by the specialist disciplines involved and that the experience gained in the project flows back into the further development of the processes.

Conclusion: Big data in the project process poses challenges for future viability in all industries

To summarize the developments of big data in the project process, the following can be stated:

  • We are only at the beginning of the digital journey. It is safe to assume that everything that can be digitized will be digitized.
  • The speed of digital change will increase. The tipping point has long since been passed.
  • In order to remain competitive, the project business must keep pace with these developments. In some industries, this will mean that the existing project system will have to be completely rethought in the sense of a "project business reloaded".
  • Numerous new methods and approaches already exist in project management today, allowing for a variety of design options.
  • The realization of the "technology" is the least of the problems.

(Cover image: © tonefotografia |

About the author

Picture of Helmut Schäfer

Helmut Schäfer

Dr. Helmut Schäfer is Managing Partner for the areas "Project Management Excellence" and "Projects & Processes for Engineering" within TCI GmbH. He has also been an independent consultant, trainer and coach specializing in project, process and change management since 2005.

Share this article on social media

More blog articles

More from our blog

Harrlachweg 2

68163 Mannheim


Do you have an request? With pleasure!

© 2024 TCI - All rights reserved.