Skip to content

Robaws DWH Azure Deployment

Basic steps:

  1. Refresh Token

    1. get a refresh token (see dhw-robaws-installation-authentication)

  2. Azure SQL database

    1. Create an Azure SQL Database Server if necessary or not already created
    2. Create an Azure SQL Database with DTU model, if necessary, and create the users. Keep the password of the db-readerfor later. See sql-server-installation for details.
    3. Create the initial configuration by running the update script: code/sql/update sql dwh.sql (see: product-dwh-robaws/code/sql/update sql dwh.sql at main · pwrp/product-dwh-robaws)
    4. add / change configurations for the client by modifying entries in config.robaws_Tables
    5. You should now have:
      1. a schema config (if not already existing from an additional DWH product installation)
      2. a schema robaws
      3. a table config.robaws_Tables

  3. Key Vault

    1. Create a key-vault if necessary. See key-vault-installation .
    2. Store the refresh token as a secret with name robaws-refresh-token

  4. Azure Function in Azure, create an Azure Function af-installation, specifically with

    1. service plan: B1 (to avoid timeouts but depends on customer Y1 also works)
    2. python 3.10 (11 doesn't contain SQL Server driver)
    3. name: xxx-pwrp-robaws-af
    4. Deploy the AF: pwrp/product-dwh-robaws/code/af (see: product-dwh-robaws/code/af at main · pwrp/product-dwh-robaws)

  5. ADF in Azure, create an Azure Data Factory adf-installation

    1. configure the Azure Function as a service
    2. configure the Azure SQL DWH as a service
    3. load the ARM template from pwrp/product-dwh-robaws/code/adf into ADF
    4. configure a trigger for the ADF pipeline
    5. publish

Additional important information!
Since the Robaws API can return inconsistent custom fields or additional nested structure fields for single entries within a response, every endpoint needs to be tested with the AF in VSCode to see if a column mismatch causes a failure when downloading. If this happens check the __init__.py file in github (see: product-dwh-robaws/code/af/CopyRobaws2SQL_http at main · pwrp/product-dwh-robaws) to see how to solve the issue!!