1

My Azure data Factory Pipeline has a COPY operation to COPY data from CSV to CosmosDB.
CSV file has 3 columns to import in CosmosDB.

CosmosDB Collection will receive:

Column1 in Partition Key
Column2 in Id
Column3 whole JSON Value

when I run COPY it copies the data in String format in CosmosDB for all 3 columns. Because there is no JSON or Map type available while we specify ColumnType.

enter image description here

What should I do to import JSON in value field, instead of String or TEXT. Below is the sample I am getting in CosmosDB: enter image description here

4
  • any other replies..? or If we can PUSH JSON Data directly from Notebook to CosmosDB? Commented Apr 3, 2020 at 12:29
  • hi,does my answer helps you? Commented Apr 7, 2020 at 3:27
  • @JayGong I am not allowed to use Azure Functions yet in our PROD. Commented Apr 9, 2020 at 7:20
  • @AnilKapoor did you find the solution? Commented Jul 14, 2021 at 20:50

1 Answer 1

1

Per my knowledge, no such features could help you convert string data to object format in adf cosmos db configuration.

So, as workaround, I suggest you using Azure Function Cosmos DB Trigger to process every document when they imported into database. Please refer to my function code:

using System.Collections.Generic;
using Microsoft.Azure.Documents;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Newtonsoft.Json.Linq;
using System;
using Microsoft.Azure.Documents.Client;

namespace TestADF
{
    public static class Function1
    {
        [FunctionName("Function1")]
        public static void Run([CosmosDBTrigger(
            databaseName: "db",
            collectionName: "item",
            ConnectionStringSetting = "documentdbstring",
            LeaseCollectionName = "leases")]IReadOnlyList<Document> input, TraceWriter log)
        {
            if (input != null && input.Count > 0)
            {
                log.Verbose("Start.........");
                String endpointUrl = "https://***.documents.azure.com:443/";
                String authorizationKey = "key";
                String databaseId = "db";
                String collectionId = "item";

                DocumentClient client = new DocumentClient(new Uri(endpointUrl), authorizationKey);

                for (int i = 0; i < input.Count; i++)
                {
                    Document doc = input[i];
                    if ((doc.GetPropertyValue<String>("Value") == null) || (!doc.GetPropertyValue<String>("Value")))
                    {                       
                        String V= doc.GetPropertyValue<String>("Value");
                        JObject obj = JObject.Parse(V);

                        doc.SetPropertyValue("Value", obj );

                        client.ReplaceDocumentAsync(UriFactory.CreateDocumentUri(databaseId, collectionId, doc.Id), doc);

                        log.Verbose("Update document Id " + doc.Id);
                    }

                }
            }
        }
    }
}
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.