The client’s Procurement area was facing hardships to update their control panel, since consolidating data related to costs and the acquisition of materials from all their business units, distributed in nine different countries, was a cumbersome and complex process. Since each unit has its own characteristics in terms of how they store data, the manual consolidation of that data took a lot of time from the team, thus hindering their analysis capacity.
That made it difficult to create an assertive forecast on the acquisition of materials or to negotiate materials in a more proactive way with their suppliers.
Among the many challenges found to implement the project, we may mention the following:
- Mapping more than 100 different data sources, spread between many different systems, in addition to manually updated Excel spreadsheets;
- Data quality in their respective sources;
- The possibility of human error in the manually fed spreadsheets;
- Integrations between the systems and Data Lake.
In line with the company’s data-driven strategy, the purpose of this project was to implement full automation of the process necessary to consolidate their data, leading to a Business Intelligence tool that enabled more in-depth data analysis.
As in every project developed by Dojo, the CRISP-DM methodology was utilized, which allows the business area to increase the value of acquired data and its analysis. In order to automate the data consolidation process, we used Google’s Data Lake infrastructure, implemented at the client’s systems by DOJO.
Dojo followed, through the CRISP-DM methodology, the following steps to achieve the project’s objective:
- Mapping of all data sources;
- Data insertion with quality assurance;
- Data processing, that is, the creation of the entire data engineering pipeline using the best practices to avoid unnecessary processing costs, thus ensuring the best commercial performance for the solution;
- Monitoring and synchronization of the data pipeline based on their business requirements;
- Data assurance with the necessary quality to enable user analysis;
- Creation of the necessary visualization layer for the area’s specific analyses.
The full automation of the process has significantly increased the team’s analysis capacity. Today, the control panel is updated every day, if compared to the monthly updates seen before the project’s implementation. The elimination of the time spent with the operational work demanded by that process has enabled the team to focus on more in-depth analyses and allocate the business area to new initiatives, while being able to visualize all of their procurement negotiation possibilities in a more predictive manner.
As a consequence, the company has obtained millions worth of gains, due to greater agility and prediction skills, and higher assertiveness when making data-driven decisions.
