I am trying to setup an Azure Data Factory Dataflow.
The source is a REST API from Monday.com which uses GraphQL. The API requires POST method, with 3 headers Content-Type "application/json", API-Version "2025-01" and Authorization: "API token"
The request body requires a query to pull the data. Heres my query example
{
"query":"query { boards(ids: someid) {items_page(limit: 1) {cursor items {id name column_values {column {title} text}}}}}"
}
When i run the dataflow it fails with the following
Error: Spark job failed: { "text/plain": "{\"runId\":\"7d4ed91d-1716-4d54-84a03c27cde7890a\",\"sessionId\":\"0e29d687-0201-4f4d-b18d-760767a23168\",\"status\":\"Failed\",\"payload\":{\"statusCode\":400,\"shortMessage\":\"com.microsoft.dataflow.broker.InvalidOperationException: DSL compilation failed: DF-DSL-001 - DSL stream has parsing errors\\nLine 8 Position 11: body: '{ \\nmismatched input ''' expecting {DECIMAL_LITERAL, HEX_LITERAL, OCT_LITERAL, BINARY_LITERAL, MAX_INT, MIN_INT, MAX_LONG, MIN_LONG, POSITIVE_INF, NEGATIVE_INF, '-', '!', '$', '~', ':', '(', '#', '[', '@(', '[]', FLOAT_LITERAL, HEX_FLOAT_LITERAL, STRING_LITERAL, REGEX_LITERAL, 'parameters', 'functions', 'stores', 'as', 'input', 'output', 'constant', 'expression', 'integer', 'short', 'long', 'double', 'float', 'decimal', 'boolean', 'timestamp', 'date', 'byte', 'binary', 'integral', 'number', 'fractional', 'any', IDENTIFIER, ANY_IDENTIFIER, META_MATCH, '$$', '$$$', '$#', OPEN_INTERPOLATE}\",\"detailedMessage\":\"Failure 2025-02-22 00:25:08.846 failed DebugManager.processJob, run=7d4ed91d-1716-4d54-84a0-3c27cde7890a, errorMessage=com.microsoft.dataflow.broker.InvalidOperationException: DSL compilation failed: DF-DSL-001 - DSL stream has parsing errors\\nLine 8 Position 11: body: '{ \\nmismatched input ''' expecting {DECIMAL_LITERAL, HEX_LITERAL, OCT_LITERAL, BINARY_LITERAL, MAX_INT, MIN_INT, MAX_LONG, MIN_LONG, POSITIVE_INF, NEGATIVE_INF, '-', '!', '$', '~', ':', '(', '#', '[', '@(', '[]', FLOAT_LITERAL, HEX_FLOAT_LITERAL, STRING_LITERAL, REGEX_LITERAL, 'parameters', 'functions', 'stores', 'as', 'input', 'output', 'constant', 'expression', 'integer', 'short', 'long', 'double', 'float', 'decimal', 'boolean', 'timestamp', 'date', 'byte', 'binary', 'integral', 'number', 'fractional', 'any', IDENTIFIER, ANY_IDENTIFIER, META_MATCH, '$$', '$$$', '$#', OPEN_INTERPOLATE}\"}}\n" } - RunId: 7d4ed91d-1716-4d54-84a0-3c27cde7890a
I have tried placing the query into a single line but it is being rejected by the API source with the following error:
at Source'source2':Failure to read most recent page request:DF-REST_001- Error response from server: Some(<html><head><title>400 Bad Request</title></head><body><center><h1>400 Bad Request</h1></center><hr><center>cloudflare</center></body></html>), Status code: 400. Please check your request url and body.(url:https://api.monday.com/v2/,request body: Some({ "query":"query { boards(ids:some id) {items_page(limit: 1) { cursor items { id name column_values { column { title } text } } } }}" }), request method: POST
The dataflow JSON is automatically created by ADF which shows this and as you can see it is placing , after the first { and last }
"source(allowSchemaDrift: true,"," validateSchema: false,"," inferDriftedColumnTypes: true,"," format: 'rest',"," timeout: 30,"," headers: ['Content-Type' -> 'application/json', 'API-Version' -> '2025-01', 'Authorization' -> 'Some API Token', 'Accept' -> 'application/json'],"," httpMethod: 'POST',"," body: '{ ","\"query\":\"query { boards(ids: some id) {items_page(limit: 1) { cursor items { id name column_values { column { title } text } } } }}\" ","}',"," paginationRules: ['supportRFC5988' -> 'true'],"," responseFormat: ['type' -> 'json', 'documentForm' -> 'arrayOfDocuments']) ~> source2">```