Centralized data lake and automated data pipelines drive real-time KPI monitoring through a Supply Chain Control Tower

Sigmoid optimized logistics management by building a centralized data lake with automated data pipelines to facilitate fast and error-free reporting and improve dashboard performance for crucial supply chain KPIs in real-time.

Business Challenges

An American multinational food manufacturing company wanted to enhance their supply chain reporting processes for essential KPIs such as delivery times, product availability, and order accuracy. Their legacy infrastructure contained numerous sources of data, scattered across the in-house analytics systems, each addressing different logistics use cases. The slow and inconsistent sales and logistics scorecard data affected the inventory management processes. The manual freight cost reconciliation process was prone to human error. Multiple independent workstreams created inefficiencies in logistics metrics. They also wanted to identify areas for improvement and enhance the overall performance of their dashboards in terms of access to insights and better user experience

Sigmoid Solution

Sigmoid guided the data discovery process and identified necessary datasets to ensure accurate reporting of sales and logistics metrics. We developed automated pipelines to bring data from staging to the business domain layer. The data processing workflow was executed by using a collection of Redshift Stored Procedures. These procedures were activated by calls from AWS Glue Jobs which were triggered on a cron (a time-based job scheduler). We scheduled the jobs using SQL scripts, stored procedures, Glue jobs, and Glue triggers. The reports/KPIs were generated as per pre-defined schedules using the data from the domain layer and the output was stored in Redshift’s datamart tier. This output was used by the visualization team to optimize dashboards for easy interpretation of the data.

Download case study

Business Impact

Sigmoid increased automation in the data pipelines to streamline the data processing workflow for the analytics platform, resulting in faster and error-free reporting. The solution identified data that previously eluded Finance analysts in the Freight Recon process, eliminating consistent discrepancies. Implementation of a usable, repeatable, and automated Single Source of Truth (SSOT) with data standardization and governance framework ensured consistency in sales and logistics scorecard data. Additionally, we established a strong foundation for data products that can effortlessly integrate new supply chain use cases.