0

The following here under is a Java program I am trying out to test JSON Processing using the org.fasterxml.jackson.core and the jackson-databind libraries in a streaming manner.

The goal is to learn how to process JSON and capture the information I want. For now, what I want to accomplish is the following:
1) The JSON that I post here under has a lot of data. The Java program that follows this JSON is my attempt to process this JSON and in particular, capture the "name": US SSN" element under "part1/myAnalysis/matches." and also "name": MasterCard Credit Card number", both elements falling under the the scope of "part1/myAnalysis/matches." ONLY.

Okay, for now, My Goal is: I just want my program to compile and atleast print out those two elements (the ones aforementioned) that interest me.

My attempt at compilation yielded the following results:

Unprocessed property: type
Unprocessed property: incidentTimestamp
Unprocessed property: numOfMatches
Unprocessed property: myReport
Unprocessed property: whatSetItOffEntry
Unprocessed property: seeRestrictedIds
Unprocessed property: status
Unprocessed property: timeStamps
Unprocessed property: count

So the JSON, the program is attempting to process is as under. If anyone can point out how to make this program compile and then print out the elements I want. That will be a great 2 step process Task.

{
    "type": "ImportantIncidentInfo",
    "incidentTimestamp": "2014-05-15T10:09:27.989-05:00",
    "numOfMatches": 4,
    "myReport": {
        "docReports": {
            "part1/.": {
                "path": [
                    "unknown"
                ],
                "myAnalysis": {
                    "matches": [
                        {
                            "id": {
                                "major": 1,
                                "minor": 0
                            },
                            "name": "US SSN",
                            "position": 13,
                            "string": " 636-12-4567 "
                        },
                        {
                            "id": {
                                "major": 3,
                                "minor": 0
                            },
                            "name": "MasterCard Credit Card Number",
                            "position": 35,
                            "string": " 5424-1813-6924-3685 "
                        }
                    ]
                },
                "cleanedUpData": [
                    {
                        "startPosition": 0,
                        "endPosition": 65,
                        "frameContent": ""
                    }
                ],
                "minedMetadata": {
                    "Content-Encoding": "ISO-8859-1",
                    "Content-Type": "text/html; charset=iso-8859-1"
                },
                "deducedMetadata": {
                    "Content-Type": "text/html; iso-8859-1"
                }
            },
            "part2/.": {
                "path": [
                    "unknown"
                ],
                "patternAnalysis": {
                    "matches": [
                        {
                            "id": {
                                "major": 1,
                                "minor": 0
                            },
                            "name": "SSN",
                            "position": 3,
                            "string": " 636-12-4567\r"
                        },
                        {
                            "id": {
                                "major": 3,
                                "minor": 0
                            },
                            "name": "MasterCard Credit Card Number",
                            "position": 18,
                            "string": "\n5424-1813-6924-3685\r"
                        }
                    ]
                },
                "cleanedUpData": [
                    {
                        "startPosition": 0,
                        "endPosition": 44,
                        "frameContent": ""
                    }
                ],
                "minedMetadata": {
                    "Content-Encoding": "windows-1252",
                    "Content-Type": "text/plain; charset=windows-1252"
                },
                "deducedMetadata": {
                    "Content-Type": "text/plain; iso-8859-1"
                }
            }
        }
    },
    "whatSetItOffEntry": {
        "action": "Log",
        "component": {
            "type": "aComponent",
            "components": [
                {
                    "type": "PatternComponent",
                    "patterns": [
                        1
                    ],
                    "not": false
                }
            ],
            "not": false
        },
        "ticketInfo": {
            "createIncident": true,
            "tags": [],
            "seeRestrictedIds": [
                {
                    "type": "userGroup",
                    "name": "SiteMasters",
                    "description": "Group for SiteMasters",
                    "masters": [
                        "04fb02a2bc0fba"
                    ],
                    "members": [],
                    "id": "04fade"
                }
            ]
        },
        "letmeknowInfo": {
            "createNotification": true,
            "contactNames": [
                "[email protected]"
            ]
        }
    },
    "seeRestrictedIds": [
        "04fade66c0"
    ],
    "status": "New",
    "timeStamps": [
        "2014-03-15T10:09:27.989-05:00"
    ],
    "count": 1
}

Task # 2
2) To process this JSON, I wrote the following Java program.

import com.fasterxml.jackson.databind.*;
import com.fasterxml.jackson.core.*;

import java.io.*;

public class ParseJson {

    public static void main(String[] args) {

        try{

                // TODO Auto-generated method stub
                JsonFactory f = new MappingJsonFactory();
                //JsonParser jp = f.createJsonParser(new File(args[0]));
                JsonParser jp = f.createParser(new File("C:/somepath /in/my/eclipse/project/jsonFormattedModified.json"));


                JsonToken current;

                current = jp.nextToken();
                if (current != JsonToken.START_OBJECT) {
                  System.out.println("Error: root should be object: quiting.");
                  return;
                }

                while (jp.nextToken() != JsonToken.END_OBJECT) {
                  String fieldName = jp.getCurrentName();
                  // move from field name to field value
                  current = jp.nextToken();
                  if (fieldName.equals("matches")) {
                    if (current == JsonToken.START_ARRAY) {
                      // For each of the records in the array
                      while (jp.nextToken() != JsonToken.END_ARRAY) {
                        // read the record into a tree model,
                        // this moves the parsing position to the end of it
                        JsonNode node = jp.readValueAsTree();
                        // And now we have random access to everything in the object
                        System.out.println("Name: " + node.get("name").asText());
                        System.out.println("POS: " + node.get("pos").asText());
                      }
                    } else {
                      System.out.println("Error: records should be an array: skipping.");
                      jp.skipChildren();
                    }
                  } else {
                    System.out.println("Unprocessed property: " + fieldName);
                    jp.skipChildren();
                  }
                }                
              } catch(IOException ie) {
                  ie.printStackTrace();

              } 

        }
}

Thanks for all the help.

3 Answers 3

2

I suggest you use a very helpful Google API Gson to deal with serialization and deserialization easily. So first of all, create all below classes that match your json structure .

Helper class:

class Helper {
    String type;
    String incidentTimestamp;
    int numOfMatches;
    Report myReport;
    WhatSetItOffEntry whatSetItOffEntry;
    List<String> seeRestrictedIds;
    String status;
    List<String> timeStamps;
    int count;
    //getters and setters
}

Report class:

class Report {
    DocsReport docReports;
    //getters and setters
}

DocsReport class:

class DocsReport {
    @SerializedName("part1/.")
    Part1 part1;
    Part2 part2;
    //getters and setters
}

Part1 class:

class Part1 {
    List<String> path;
    Analysis myAnalysis;
    List<CleanedUpData> cleanedUpData;
    MinedMetadata minedMetadata;
    DeducedMetadata deducedMetadata;
    //getters and setters
}

Analysis class:

class Analysis {
    List<Information> matches;
    //getters and setters
}

Information class:

class Information {
    Identifying id;
    String name;
    int position;
    String string;
    //getters and setters
}

Identifying class:

class Identifying {
    int major;
    int minor;
    //getters and setters
}

CleanedUpData class:

class CleanedUpData {
    int startPosition;
    int endPosition;
    String frameContent;
    //getters and setters
}

MinedMetadata class:

class MinedMetadata {
    @SerializedName("Content-Encoding")
    String contentEncoding;
    @SerializedName("Content-Type")
    String contentType;
    //getters and setters
}

DeducedMetadata class:

class DeducedMetadata {
    @SerializedName("Content-Type")
    String contentType;
    //getters and setters
}

Part2 class:

class Part2 {
    List<String> path;
    Analysis patternAnalysis;
    CleanedUpData cleanedUpData;
    MinedMetadata minedMetadata;
    DeducedMetadata deducedMetadata;
    //getters and setters
}

WhatSetItOffEntry class:

class WhatSetItOffEntry {
    String action;
    Component component;
    TicketInfo ticketInfo;
    LetmeknowInfo letmeknowInfo;
    //getters and setters
}

Component class:

class Component {
    String type;
    List<ComponentData> components;
    Boolean not;
    //getters and setters
}

ComponentData class:

class ComponentData {
    String type;
    List<Integer> patterns;
    Boolean not;
    //getters and setters
}

TicketInfo class:

class TicketInfo {
    Boolean createIncident;
    List<Object> tags;
    List<RestrictedIds> seeRestrictedIds;
    //getters and setters
}

RestrictedIds class:

class RestrictedIds {
    String type;
    String name;
    String description;
    List<String> masters;
    List<Object> members;
    String id;
    //getters and setters
}

LetmeknowInfo class:

class LetmeknowInfo {
    Boolean createNotification;
    List<String> contactNames;
    //getters and setters
}

Then get your two names as follow

Gson gson = new Gson();
Helper data = gson
        .fromJson(
               new BufferedReader(
                        new FileReader(
                                "C:/somepath/in/my/eclipse/project/jsonFormattedModified.json")),
                new TypeToken<Helper>() {
                }.getType());

String name1 = data.getMyReport().getDocReports().getPart1()
        .getMyAnalysis().getMatches().get(0).getName();
String name2 = data.getMyReport().getDocReports().getPart1()
        .getMyAnalysis().getMatches().get(1).getName();

System.out.println(name1+"\n"+name2);

Output:

US SSN
MasterCard Credit Card Number
Sign up to request clarification or add additional context in comments.

11 Comments

Thank you very much. I am going to try it and let you know how it goes.
I haver tested it here on my IDE and it works perfectly. Just don't forget to add gson jar after downloading it from the link that i have already provided to you in my answer.
I was just going to test it in my IDE. In just a few minutes I should see the result of your effort and labor.
I am going to need another 30 minutes to finish running this class. I will be right back with the results of my testing.
Which Gson Library are you using? For this piece of code: It complains on this line that the method getJson is not applicable for the arguments BufferedWriter and Type. I am using gson-2.2.4.jar Helper data = gson.fromJson( new BufferedWriter( new FileReader("C:/somepath/in/my/eclipse/project/jsonFormattedModified.json")), new TypeToken<Helper>() { }.getType());
|
0

Two things to consider:

  • If you want to keep streaming, calling jp.readValueAsTree() is not a good idea as that will allocate memory to create the tree with all data under the current node (in your code, everything under "matches"), unless you know for sure this is a small tree.
  • To get to a value in an array, there is no need to create an array first. The streaming API will let you walk to the value in the array you need.

As @MChaker shows in his answer, if you need most of the data provided in the JSON file, creating a model to hold the data will be beneficial. But if you just need a couple of values, Jackson will let you do that. Allthough, I must admit, getting data in a truly streaming fashion will require some creativity to find ways of keeping track of where you are and what data you expect.

Following code shows the easy way and the streaming way:

import java.io.*;
import java.util.*;

import com.fasterxml.jackson.core.*;
import com.fasterxml.jackson.databind.*;

/**
 * http://stackoverflow.com/q/30288878/3080094
 * Using jackson-databind-2.5.3 which uses
 * jackson-annotations-2.5.0 and
 * jackson-core-2.5.3
 * @author vanOekel
 */
public class JsonTest {

    public static void main(String[] args) {

        try {
            new JsonTest().getNames();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    ObjectMapper jsonMapper = new ObjectMapper();
    JsonFactory jsonFactory = new JsonFactory();

    void getNames() throws Exception {

        final String resourceName = "some.json";
        JsonNode jn;
        try (InputStream in = 
                Thread.currentThread().getContextClassLoader().getResourceAsStream(resourceName)
                ) {
            if (in == null) {
                throw new FileNotFoundException("File not found: " + resourceName);
            }
            jn = jsonMapper.readTree(in);
        } 
        findByPath(jn);

        try (InputStream in = 
                Thread.currentThread().getContextClassLoader().getResourceAsStream(resourceName)
                ) {
            if (in == null) {
                throw new FileNotFoundException("File not found: " + resourceName);
            }
            JsonParser jsonParser = jsonFactory.createParser(in);
            findInStream(jsonParser);
        } 
    }

    final String[] path = new String[] {"myReport", "docReports", "part1/.", "myAnalysis", "matches", "name"};

    void findByPath(JsonNode jn) {

        JsonNode matchesNamesNode = jn;
        for (int i = 0; i < path.length - 1; i++) {
            matchesNamesNode = matchesNamesNode.path(path[i]);
        }
        if (matchesNamesNode.isMissingNode()) {
            throw new RuntimeException("No node with names found.");
        }
        System.out.println("Tree names: " + matchesNamesNode.findValuesAsText(path[path.length - 1]));
    }

    void findInStream(JsonParser jp) throws Exception {

        int pathIndex = 0;
        List<String> names = new ArrayList<String>();
        boolean breakOnClose = false;

        while (jp.nextToken() != null) {
            final String fieldName = jp.getCurrentName();
            if (fieldName == null) {
                continue;
            }
            if (breakOnClose && fieldName.equals(path[path.length - 2])) {
                System.out.println("Stopping search at end of node " + fieldName);
                break;
            }
            if (jp.getCurrentToken() != JsonToken.FIELD_NAME) {
                continue;
            }
            // System.out.println("Field " + fieldName);
            if (pathIndex >= path.length - 1) {
                if (fieldName.equals(path[path.length - 1])) {
                    // move from field name to field value.
                    jp.nextToken(); 
                    String name = jp.getValueAsString();
                    if (name == null) {
                        throw new RuntimeException("No value exists for field " + fieldName);
                    }
                    names.add(name);
                    System.out.println("Found " + fieldName + " value: " + name);
                }
            } else if (fieldName.equals(path[pathIndex])) {
                System.out.println("Found node " + path[pathIndex]);
                pathIndex++;
                if (pathIndex >= path.length - 1) {
                    System.out.println("Looking for names ...");
                    breakOnClose = true;
                    // prevent breaking on "matches" value json-token.
                    jp.nextFieldName(); 
                }
            }
        }
        System.out.println("Streaming names: " + names);
    }

}

3 Comments

Sorry, I am looking at your answer only now. Yes. I like Chaker's solution, but I tend to like the streaming approach, given that I am trying to find a couple of pieces of data in the json. Let me try this and get back to you.
So, what you are doing is giving me two ways to get to the data I want. 1) findInStream 2) findInPath
Indeed, one "in memory", the other streaming. I would go for the "findByPath" (the "in memory") option as long as the file is less than (lets say) 32kB. It is much easier to use (and maintain) compared to the streaming option and I doubt you'll ever run out of memory. If you ever need to handle several files at the same time that are over 1MB, the extra effort for the streaming option might be needed. But I would only use the streaming option when I'm certain I need it to prevent running out of memory.
0
  String jsonRecord = "[
                 {
                   a:a,
                   b:
                     {
                      c:
                        [{d:d,e:e},{d:d1,e:e1}]
                     }
                  }
                ]";
 String value ="$.b.c[1].d";
 String str = JsonPath.read(jsonRecord, value);
 system.out.println(str);

it will print d1

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.