Unable to connect to Big Query as a data source when adding data

This article discusses a fix for the error message that you get when you try to connect a production instance to a Big Query table in Adobe Experience Platform.

Description description

Environment

Adobe Experience Platform

Issue/Symptoms

When trying to connect a production instance to a Big Query table, you get the message:  An error occurred getting data from the source

All existing dataflows run successfully. Only adding new data seems to be causing the error.

Resolution resolution

The issue is in the Microsoft DataFactory API, which takes a long time to fetch a list of tables if there are large number of datasets in the GBQ project.

The fix for this was done by Microsoft, but it seems that it was released only for those affected at the time of the report.

Adobe Engineering is currently communicating with Microsoft for the fix to be released for everyone.

If you are facing this issue from the Adobe Experience Platform perspective, please raise a support ticket with the AEP team.