Create Data Product for Procurement Domain
Let us now design the procurement domain product.
Figure 4-5. Procurement domain product
Figure 4-5 depicts the procurement domain product. There are different suppliers from whom the organization procures data. The data comprise procured product details, prices, and other necessary information. The data is directly ingested from the supplier system, and the data format varies from supplier to supplier. Cosmos DB database is used to store the supplier data. The Synapse pipeline reads the data from the supplier database, transforms the data into a consumable format, and then stores the data in
an Azure Data Lake Gen2. Synapse pipeline can use the Copy activity to fetch the data from HR database and put it in the Azure Data Lake Gen2 after processing the data.
Azure Data Lake Gen2 exposes the data to other domain products for consumption. Operational logs and metrics for Synapse are stored in Azure Monitor. Azure Monitor can be used to diagnose the run-time issues in the product. Azure Purview scans data capture and stores the data lineage, data schema, and other metadata information that can help in discovering the data.
Create Data Product for Sales Domain
Let us now design the sales domain product.
Figure 4-6. Sales domain product
Figure 4-6 depicts the sales domain product. There are different sales channels for the enterprise. There are distributors that sell the products. There are also B2C sales for the products in the enterprise e-commerce portal. The sales data from multiple channels are ingested into Azure Databricks through Event Hub. The volume of data that is getting ingested is huge. So, the data are stored in Azure Databricks. Data from the inventory domain droduct are consumed by the sales domain product. Spark pipelines transform the data and put the data in an Azure Data Lake Gen2, which exposes the data to other domain products for consumption. Operational logs and metrics for Synapse are stored in Azure Monitor. Azure Monitor can be used to diagnose the run-time issues in the product. Azure Purview scans data capture and stores the data lineage, data schema, and other metadata information that can help in discovering the data.
