jq (JSON Query) Tool Manual

jq (JSON Query) is JSON processing tool.

Installation

 

Install jq JSON processor on Ubuntu 10.04 - Stack Overflow 

Help Page

$ jq --help
jq - commandline JSON processor [version 1.6]

Usage:    jq [options] <jq filter> [file...]
    jq [options] --args <jq filter> [strings...]
    jq [options] --jsonargs <jq filter> [JSON_TEXTS...]

jq is a tool for processing JSON inputs, applying the given filter to
its JSON text inputs and producing the filter's results as JSON on
standard output.

The simplest filter is ., which copies jq's input to its output
unmodified (except for formatting, but note that IEEE754 is used
for number representation internally, with all that that implies).

For more advanced filters see the jq(1) manpage ("man jq")
and/or https://stedolan.github.io/jq

Example:

    $ echo '{"foo": 0}' | jq .
    {
        "foo": 0
    }

Some of the options include:
  -c               compact instead of pretty-printed output;
  -n               use `null` as the single input value;
  -e               set the exit status code based on the output;
  -s               read (slurp) all inputs into an array; apply filter to it;
  -r               output raw strings, not JSON texts;
  -R               read raw strings, not JSON texts;
  -C               colorize JSON;
  -M               monochrome (don't colorize JSON);
  -S               sort keys of objects on output;
  --tab            use tabs for indentation;
  --arg a v        set variable $a to value <v>;
  --argjson a v    set variable $a to JSON value <v>;
  --slurpfile a f  set variable $a to an array of JSON texts read from <f>;
  --rawfile a f    set variable $a to a string consisting of the contents of <f>;
  --args           remaining arguments are string arguments, not files;
  --jsonargs       remaining arguments are JSON arguments, not files;
  --               terminates argument processing;

Named arguments are also available as $ARGS.named[], while
positional arguments are available as $ARGS.positional[].

See the manpage for more options.

Echoing Input (Verifying JSON string)

Let's use the following JSON (example taken from https://json.org/example.html):

$ myjson=''\

'{"web-app": {'\
' "servlet": ['\
'   {'\
'     "servlet-name": "cofaxCDS",'\
'     "servlet-class": "org.cofax.cds.CDSServlet",'\
'     "init-param": {'\
'       "configGlossary:installationAt": "Philadelphia, PA",'\
'       "configGlossary:adminEmail": "ksm@pobox.com",'\
'       "dataStoreConnUsageLimit": 100,'\
'       "dataStoreLogLevel": "debug",'\
'       "maxUrlLength": 500}},'\
'   {'\
'     "servlet-name": "cofaxEmail",'\
'     "servlet-class": "org.cofax.cds.EmailServlet",'\
'     "init-param": {'\
'     "mailHost": "mail1",'\
'     "mailHostOverride": "mail2"}},'\
'   {'\
'     "servlet-name": "cofaxAdmin",'\
'     "servlet-class": "org.cofax.cds.AdminServlet"},'\
'   {'\
'     "servlet-name": "fileServlet",'\
'     "servlet-class": "org.cofax.cds.FileServlet"},'\
'   {'\
'     "servlet-name": "cofaxTools",'\
'     "servlet-class": "org.cofax.cms.CofaxToolsServlet",'\
'     "init-param": {'\
'       "templatePath": "toolstemplates/",'\
'       "log": 1,'\
'       "adminGroupID": 4,'\
'       "betaServer": true}}],'\
' "servlet-mapping": {'\
'   "cofaxCDS": "/",'\
'   "cofaxEmail": "/cofaxutil/aemail/*",'\
'   "cofaxAdmin": "/admin/*",'\
'   "fileServlet": "/static/*",'\
'   "cofaxTools": "/tools/*"},'\
' "taglib": {'\
'   "taglib-uri": "cofax.tld",'\
'   "taglib-location": "/WEB-INF/tlds/cofax.tld"}}}'

 

 

To verify that JSON is well-formatted we can use jq . which echoes JSON input to output (as pretty-printed JSON):

$ echo $myjson | jq .
{
  "web-app": {
    "servlet": [
      {
        "servlet-name": "cofaxCDS",
        "servlet-class": "org.cofax.cds.CDSServlet",
        "init-param": {
          "configGlossary:installationAt": "Philadelphia, PA",
          "configGlossary:adminEmail": "ksm@pobox.com",
          "dataStoreConnUsageLimit": 100,
          "dataStoreLogLevel": "debug",
          "maxUrlLength": 500
        }
      },
      {
        "servlet-name": "cofaxEmail",
        "servlet-class": "org.cofax.cds.EmailServlet",
        "init-param": {
          "mailHost": "mail1",
          "mailHostOverride": "mail2"
        }
      },
      {
        "servlet-name": "cofaxAdmin",
        "servlet-class": "org.cofax.cds.AdminServlet"
      },
      {
        "servlet-name": "fileServlet",
        "servlet-class": "org.cofax.cds.FileServlet"
      },
      {
        "servlet-name": "cofaxTools",
        "servlet-class": "org.cofax.cms.CofaxToolsServlet",
        "init-param": {
          "templatePath": "toolstemplates/",
          "log": 1,
          "adminGroupID": 4,
          "betaServer": true
        }
      }
    ],
    "servlet-mapping": {
      "cofaxCDS": "/",
      "cofaxEmail": "/cofaxutil/aemail/*",
      "cofaxAdmin": "/admin/*",
      "fileServlet": "/static/*",
      "cofaxTools": "/tools/*"
    },
    "taglib": {
      "taglib-uri": "cofax.tld",
      "taglib-location": "/WEB-INF/tlds/cofax.tld"
    }
  }
}

If JSON is not formatted well, jq . will output parsing error:

$ myjson=''\
> '['\
> '    ['\
> '        "first"'\
> '    ],'\
> '    ['\
> '        "second"'\
> '    ],'\
> ']'
 

$ echo $myjson | jq .

parse error: Expected another array element at line 1, column 29 

The error here is a comma after the last array element. After removing it the error goes away.

Getting Values

 

To get the value of the key which has a dash (-) in name, we need to wrap the key name in double quotes:

$ echo $myjson | jq '."web-app"'
{
  "servlet": [
    {
      "servlet-name": "cofaxCDS",
...

To get values at the path of keys, key names should be separated with dots:

$ echo $myjson | jq '."web-app".taglib'
{
  "taglib-uri": "cofax.tld",
  "taglib-location": "/WEB-INF/tlds/cofax.tld"
}

To get elements of an array, use [] behind the array key name:

$ echo $myjson | jq '."web-app".servlet[]'
{
  "servlet-name": "cofaxCDS",
  "servlet-class": "org.cofax.cds.CDSServlet",
  "init-param": {
    "configGlossary:installationAt": "Philadelphia, PA",
    "configGlossary:adminEmail": "ksm@pobox.com",
    "dataStoreConnUsageLimit": 100,
    "dataStoreLogLevel": "debug",
    "maxUrlLength": 500
  }
}
{
  "servlet-name": "cofaxEmail",
  "servlet-class": "org.cofax.cds.EmailServlet",
  "init-param": {
    "mailHost": "mail1",
    "mailHostOverride": "mail2"
  }
}
{
  "servlet-name": "cofaxAdmin",
  "servlet-class": "org.cofax.cds.AdminServlet"
}
{
  "servlet-name": "fileServlet",
  "servlet-class": "org.cofax.cds.FileServlet"
}
{
  "servlet-name": "cofaxTools",
  "servlet-class": "org.cofax.cms.CofaxToolsServlet",
  "init-param": {
    "templatePath": "toolstemplates/",
    "log": 1,
    "adminGroupID": 4,
    "betaServer": true
  }
}

 

Let's create JSON with array at its root:

 

$ myjson=''\

'['\
'    ['\
'        "first"'\
'    ],'\
'    ['\
'        "second"'\
'    ]'\

']'

$ echo $myjson | jq .
[
  [
    "first"
  ],
  [
    "second"
  ]
]

To get elements of the root array:

$ echo $myjson | jq .[]
[
  "first"
]
[
  "second"
]

To get elements at index 0 we can pipe jq commands: .[] gives all elements of the root array (and these elements are arrays themselves) and then .[0] gives all first elements in each this sub-array:

$ echo $myjson | jq ".[] | .[0]"
"first"
"second"

---

Resources:

bash - Parsing JSON with Unix tools - Stack Overflow
json - jq not working on tag name with dashes and numbers - Stack Overflow

Exploring long JSON files with jq | Adam Young’s Web Log

Introduction to JSON

JSON (JavaScript Object Notation) is:

• lightweight data-interchange format
• open standard file format
• easy for humans to read and write
• easy for machines to parse and generate
• uses human-readable text to store and transmit data objects

JSON - Wikipedia

JSON

Arrays

Valid JSON: array of objects:

[
   {
"a": 1
   }, 
   {
"b": 2
   }
]

Valid JSON: array of arrays where each array has a single element:

[
[
  {
"a": {}
}
 ],
[
  {
"b": {}
}
 ]
]

How to represent an array of integers?

JSON for List of int

{
    "Id": "610",
    "Name": "15",
    "Description": "1.99",
    "ItemModList": [
                       0,
                       375,
                       668,
                       5,
                       6
                   ]
}

JSON (object) vs JSON string

JSON object is an instance of the data type (e.g. Object in JavaScript) which represents a JSON (structure) and which can be initialised with a JSON literal:

var jsonObject = { "prop": "val" };

JSON string is a string:

var jsonString = '{ "prop": "val" }';

JavaScript provides methods for converting JSON Object to string:

JSON.stringify to convert objects into JSON string.

JSON.parse to convert JSON string back into an object.

JSON methods, toJSON

Ben Alman » There's no such thing as a "JSON Object"

JSON Data Binding with JSON Spirit

JSON is, like XML, a human readable, text-based format for data representation.

JSON document represents either object or an array.

Object starts and ends with curly brackets ({ }) and contains zero or more key-value pairs. Key (or name) is a string and value can be another object, number, string, array or literal true, false or null. Key is separated from value with a single colon (:). Key-value pars are separated with commas (,).

Array starts and ends with square brackets ([ ]) and contains zero or more values.

I wrote a mini series of articles on XML Data Binding and in this article I want to show how to serialize C++ object to JSON document and how to perform reverse process - how to deserialize JSON document back to the object.

JSON parser I am using here is JSON Spirit. Here is the list of steps of how to get working example from the scratch:

1. Download and unpack master from http://gitorious.org/json-spirit/json-spirit/trees/master
2. Find and open provided Visual Studio solution file (convert it to VS2010 format if necessary). It contains projects: json_demo, json_headers_only_demo, json_map_demo, json_spirit_lib and json_test.
3. We will create test project that will be using JSON Spirit static library and its headers (Boost headers will be necessary as well). Build json_spirit_lib project which output is static library named json_spirit_lib.lib.
4. In Project Settings for our test project, add json_spirit_lib.lib to Additional Dependencies as well as path to JSON Spirit headers (path to json_spirit directory) and path to Boost.
5. Use main.cpp provided below which contains couple of functions that demonstrate JSON/C++ serialization/deserialization:

main.cpp:


Output:

Serialization result (generated JSON string):

{
   "name" : "Fyodor",
   "surName" : "Dostoyevsky"
}

Deserialization result:

name: Fyodor
surName: Dostoyevsky

Serialization result (generated JSON string):

{
   "author" : {
      "name" : "Fyodor",
      "surName" : "Dostoyevsky"
   },
   "isbn" : 123456789,
   "title" : "Crime and Punishment",
   "type" : 1
}

Deserialization result:

author name: Fyodor
author surName: Dostoyevsky
isbn: 123456789
title: Crime and Punishment
type: 1

Links and References:
The application/json Media Type for JavaScript Object Notation (JSON) - RFC 4627
Can a JSON start with [? (SO)