PAB Lineage

Overview

• the master pipeline calls the orchestration and takes care of notification

• the orchestration pipeline calls the various pipelines (nav_to_bronze, etc.) and refreshes the semantic model

• there is only a schedule on the master pipeline

• the notification goes via an Azure Logic App. This is a simple and cheap component that performs some activities based on HTTP calls. So, in case of an error, the master pipeline calls a Web Activity. For now, there is just a single notification, without any details about where the error happened. This will have to be looked up in the Monitor section of Fabric.
  

  graph TD;
      A[Master Pipeline] -->|1-triggers| B[Orchestration]
      A -->|7-notifies| C[Azure-Logic-App]
      B -->|2-triggers| D[nav-to-bronze]
      B -->|3-triggers| E[nav-to-silver]
      B -->|4-triggers| F[sap-to-bronze]
      B -->|5-triggers| G[sap-to-silver]
      B -->|6-triggers| H[semantic-model]
  
  

Data Lineage

graph TD;
    A[NAV] -->|WS| B[Gateway]
    C((SAP)) -->|ODBC| B
    B --> D[nav_to_bronze]
    B --> E[sap_to_bronze]

    D --> F((LH nav_bronze))
    E --> G((LH sap_bronze))

    F --> H[nav_to_silver]
    G --> I[sap_to_silver]

    H --> J((LH nav_silver))
    I --> K((LH sap_silver))