Thankyou to @pmf, that is the filter I was after (with a caveat). The problem I was having relates to the jq error: jq: error (at Security.json:12490): Cannot iterate over null (null) . I think the issue in question was not just how to manage an array within jq, but how it handles iterating over an array when handling nulls and you don't explicitly state the length of the array, eg: jq '.Event[].EventData.Data[] |= (select(.content != null))' Security.json throws the error.

I have found that explicitly declaring the length of the array to jq when handling the top-level array, eg: jq '.Event[0- ( .Event | length)] <further jq> or jq '.Event[0-N] <further jq> (for an array with N+1 variables) results in successful execution. I would speculate that jq's mode of iteration when the length of the array is not specified results in it checking index N+1 and returning null, then handing that down the pipe and throwing an error when it looks for null (null). I don't know if this is intentional behaviour for jq or a bug.

I've included the complete bash script (with filters) below for other people that want help with an error similar to this:

Filtering Nulls from the array

The code that works for me:

jq '.Event[0- ( .Event | length)].EventData.Data[] |= (select(.content != null))' Security.json

and this (although not dynamic):

jq '.Event[0-152].EventData.Data[] |= (select(.content != null))' Security.json

The code that doesn't work for me:

jq '.Event[].EventData.Data[] |= (select(.content != null))' Security.json

Reformatting data within .Data[] into "Name": "Variable" without removing nulls

The code that works for me:

jq '.Event[0- ( .Event | length)].EventData.Data[] |= {(.Name): .content}' Security.json | less

and this (although not dynamic):

jq '.Event[0-152].EventData.Data[] |= {(.Name): .content}' Security.json

The code that doesn't work for me:

jq '.Event[].EventData.Data[] |= {(.Name): .content}' Security.json