Posts

Showing posts from May, 2022

Copy data from an OData source by using Azure Data Factory

Image
  Hey guys, time to share something new that I have learned in the past few days.  I am going to share my thought process on how I viewed the problem statement and arrived at the solution.  So, let's get rolling, as part of the requirements, we were given an API endpoint and were told that we need to ingest data and show it on the power bi report.  I began brainstorming on how do we pull data from API either to ADLS storage in the form of CSV or txt files or to Azure SQL DB in the form of a managed table.  Disclaimer -  I am new to extracting information from API endpoints using Azure Data Factory, so had to read quite a few blogs. I discovered that we could either use Rest API connectors or HTTP connectors to talk to the API endpoint and pull out the information needed.  So I tried using both of the above connectors and found that it is becoming challenging to extract the information in the CSV file. Although I did get some success in fetching only the first row from the API endpo