Unable to connect a to Big Query as a data source when adding data

Description description

Environment
Adobe Experience Platform

Issue
When trying to connect a Production instance to a Big Query table, you get an error:“An error occurred getting data from the source”

Existing dataflows run successfully.

Adding new data seems the to be issue cause.

Resolution resolution

Solution
The issue is in the Microsoft DataFactory API which takes a long time to fetch list of tables if there are a large number of datasets in GBQ project.

The fix for this was done by Microsoft, but it seems that it was released it only for affected customers at the time of the report.

Adobe Engineering is currently communicating with Microsoft for the fix to be released for everyone.

If facing this issue from the Adobe Experience Platform perspective, please raise a support ticket with the AEP team.

recommendation-more-help
3d58f420-19b5-47a0-a122-5c9dab55ec7f