Structuring JSON data with the [dict] object in Max

{
	"perceivedComplexity": "beastly",
	"actualComplexity": "manageable",
}

Working with setting and getting content from dictionaries in Max seems straightforward enough, but trying to group data into well-structured form can be a little tricky.

Structured Data

Recently I had a need to create a way to store some fairly complex data in Max. I wanted to map out and find similarities in a bunch of audio files. I’m not a computer scientist or a real programmer, so I had no idea how I should do this or how I should store this information in a manageable way. [As it turns out, I needed to create a hash table.]

Traditionally, the coll object was the go-to object to do this kind of stuff (and still is to an extent). It’s a simple way to store a list of values at a numerical (or symbolic) index.

1, 100 72 64 forward 7.43 delay 85 0;
2, 60 160 62 forward 5.0 bypass 51 1;
3, 82 10 114 backward 0.2 delay 15 1;
4, 155 97 98 backward 8.2 delay 99 0;

Send the coll object a 2, and the corresponding data (60 160 62 forward 5.0 bypass 51 1) will come out the first outlet.

When trying to encode lots of data though, a more descriptive index would be more helpful.

While coll supports indexes that are symbols, I was keen to use something that allowed me to look up or retrieve particular ‘atoms’ of the information I was storing. With coll, if you request the data stored at an index, you retrieve all the data stored at that index. As coll data is stored as a list of values, the order of the data stored at that index is important, and it can be a little difficult to see what each value represents. Furthermore, as I was interested in storing and retrieving data based on some kind of shared similarity (ie. separate arrays of data that should be grouped under the same ‘index’) I wanted to store it in a more descriptive and extensible way. What I needed was something like an associative array.

Associative arrays store every piece of information as key and value pairs. This data structure goes by many differing names (dictionaries, hashes, maps, symbol tables, hash tables, collections). In the JavaScript world these kinds data storage structures are referred to as objects. I’ll refer to them as objects for the time being. (Just to confuse things more, key-value pairs are also sometimes termed name-value pairs, index-value pairs, and attribute-value pairs.)

The key would describe the bit of information I was interested in storing, and the value would be the number/setting/value representing that information.

Essentially, objects represent structured data like this:

{
	"name": "Alex",
	"sex": "male",
	"age": 35,
	"coffee": "espresso",
	"coffeeTimes": [7, 9, 11, 16]
}

An example of object notation.

keys sit on the left, values on the right. There’s a colon after each key, and a comma after the first to the penultimate key-value pair. keys are strings, and all strings are surrounded by quotes (eg. "coffee" or "espresso"), and arrays are a list of comma separated values in square brackets (eg. [7, 8, 11, 16]).

The cool thing about object notation is that values can be strings, numbers, lists/arrays, or even objects themselves. Even better is that you can insert a new key-value pair at any point within your object and it won’t break anything, because you retrieve values by their key (in contrast to coll where you’d have to keep track of where the data value you were storing was in the array of values stored at that index).

Combined with arrays, objects are very flexible ways to store and format data. Scott Murray’s D3 Tutorial chapter on Data Types illustrates the power of objects and arrays really well: “You can combine these two structures to create arrays of objects, or objects of arrays, or objects of objects or, well, basically whatever structure makes sense for your data set.”

What do ‘arrays of objects’, ‘objects of arrays’, and ‘objects of objects’ mean?

Well, many things. If an array is a list of items, and an object is a collection of named properties grouped together. You could combine them in ways to:

  • create a list of data structures that were all related in some way and assign them all to one keyed list (and access info about each one by its index in the array); or
  • nest specific bits of information within the context in which they are relevant; or
  • have a collection of properties that had their own groups of sub properties, and so on.

Example 1:

{
	"animals": [
		{ "name": "Alex", "sex": "male", "age": 35, "species": "human" },
		{ "name": "Benny", "sex": "male", "age": 3, "species": "cat" },
		{ "name": "Mench", "sex": "male", "age": 6, "species": "cat" }		
	]
}

The animals key contains an array of objects.

Example 2:

{
	"series1": [ 0, 1, 3, 7, 15, 31, 63 ],
	"series2": [ 1, 4, 9, 16, 25, 36, 49 ],
	"series3": [ 1, 2, 4, 7, 11, 16, 22 ],
	"series4": [ 1, 1, 2, 3, 5, 8, 13 ]
}

An object whose keys are all arrays.

Example 3:

{
	"name": "Alex",
	"sex": "male",
	"age": 35,
	"coffee": {
		"type": "espresso",
		"specs": {
			"shots": 2,
			"milk": 1,
			"sugar": 0
		}
	},
	"coffeeTimes": [ 7, 9, 11, 16 ]
}

Note that the ‘coffee’ key contains an object with two keys (‘type’ and ‘specs’), and the value of ‘specs’ itself is an object.

Essentially, [] indicates an array, and {} an object. In JavaScript, you access objects’ values by their key, and arrays’ values by appending their numerical index (starting at 0) in square brackets. If an object is contained within another object, you use ‘dot’ notation to indicate the ‘path’ to the desired named element.

age					// Returns 35
coffee.type			// Returns "espresso"
coffee.specs.shots	// Returns 2
coffeeTimes[2]		// Returns 11

Retrieving properties of keys and arrays in JavaScript.

JavaScript Object Notation

JavaScript Object Notation (or JSON) is a specific syntax for organising data as JavaScript objects. Essentially keys are wrapped in double quotes, as are the values if they are strings/symbols.

{
  "firstName": "John",
  "lastName": "Smith",
  "isAlive": true,
  "age": 25,
  "address": {
    "streetAddress": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postalCode": "10021-3100"
  },
  "phoneNumbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    }
  ],
  "children": [],
  "spouse": null
}

[From the JSON entry on Wikipedia

Note again that the value stored under ‘address’ is itself an object that contains its own key-value pairs, and that ‘phoneNumbers’ contains an array of objects.

Dictionaries in Max: The [dict] object

The dict object emerged in Max 6 as a way to store structured data like this. As the term ‘object’ in Max refers to elements within a patch that perform a function, object-like data structures are referred to as dictionaries in Max.

{
	"key": "value",
	"anotherKey": "anotherValue"
}

Why are dictionaries good?

Apart from the fact that data can be structured in a more meaningful and readable way, the order of the key-value data pairs they contain doesn’t matter. As alluded to above, in the coll object, changing the order of the values in an array would likely break something in your patch (as the position of the items in the array carries some kind of associative meaning), whereas in a dictionary the order doesn’t matter as you request the value stored at a key (as opposed to the nth item in a list).

In a coll:

1, 100 72 64 forward 7.43 delay 85 0;

… is different to:

1, 64 forward 100 72 7.43 delay 85 0;

Whereas in a dict:

{
	"key1": 54,
	"key2": 95,
	"key3": 8
}

…is equivalent to:

{
	"key1": 54,
	"key3": 8,
	"key2": 95
}

Building dictionary content

The dict object allows us to programmatically build up content in a JSON-like way. There are a few ways of setting content in a dict object.

set, append, and replace messages allow you to:

  • set a string (symbol), number (int/float), or array at a particular key;
  • append values to a specified key to turn it into an array (or insert the key and value pair if it does not existing within the dictionary); and
  • replace the value at an existing key (or insert the key and value pair if it does not existing within the dictionary).

For example, the message:

set tree 4

Creates the following in the dict:

{
	"tree": 4
}

…and sending the message: (if the dict already contained {"tree": 4})

append tree oak

… would result in:

{
	"tree": [4, "oak"]
}

(We’ve appended a value to the key ‘tree’, so it now contains an array of two items.)

Message:

replace tree none

… changes dict‘s content to:

{
	"tree": "none"
}

(Replace the value at key ‘tree’ with something else.)

Before we get to nesting dictionaries within dictionaries, let’s look at how to retrieve content.

Retrieving content from a dictionary

{
	"name": "Alex",
	"sex": "male",
	"age": 35,
	"coffee": {
		"type": "espresso",
		"specs": {
			"shots": 2,
			"milk": 1,
			"sugar": 0
		}
	},
	"coffeeTimes": [ 7, 9, 11, 16 ]
}

There are a few methods that allow you to get information from a dictionary: get, gettype, getsize, and getkeys. Given the dictionary above, the following is an example of what gets output with these get methods.

Method Example key Output (key and value)
get name name Alex
sex sex male
coffee coffee dictionary u504001192
coffee::type coffee::type espresso
coffeeTimes coffeeTimes 7 9 11 16
gettype name symbol
age int
coffee coffee dictionary
coffeetimes coffeeTimes array
getsize name name 1 [ie. 1 string]
age age 1 [ie. 1 int]
coffee coffee 1 [ie. 1 dictionary]
coffee::specs coffee::specs 1 [ie. 1 dictionary]
coffeeTimes coffeeTimes 4 [ie. 4 values in the array]
getkeys [outputs a list of all the top level keys]

Note that to access nested dictionary content (eg. ‘specs’), you use a double colon separator (::) — ie. get coffee::type.

So we can retrieve nested dictionary content, but how do we set it?

Setting key-value pairs is easy, but setting nested dictionary content (ie. a dictionary at a key, or an array of dictionaries at a key) requires a few little steps to do correctly. Let’s build a complex set of nested content like GeoJSON data as an example:

{
    "type": "FeatureCollection",
    "features": [
        {
            "type": "Feature",
            "geometry": {
                "type": "Point",
                "coordinates": [ 150.1282427, -24.471803 ]
            },
            "properties": {
                "type": "town"
            }
        }
    ]
}

[From Scott Murray’s Types of data]

The setparse message

There’s not very much about setparse in the Max help patches, but setparse is one of the most important messages when trying to construct dictionaries within dictionaries using Max messages.

setparse allows you to set content as a dictionary at a specified key.

Let’s go back to a simple example:

{
	"name": "Alex",
	"sex": "male",
	"age": 35
}

The syntax for setparse goes like this:

setparse coffee type: espresso

The first word after ‘setparse’ is the key at which you wish to add some dictionary value. If the second word has a trailing colon (eg. as in ‘type:’), it creates a dictionary with that key (‘type’) within the first key (‘coffee’). Re-read that if it didn’t make sense.

If you list a value after the second word (eg. ‘espresso’), it sets the value at the second word’s key (ie. the value of the nested dictionary’s key).

Namely, the dictionary would now look like this:

{
	"name": "Alex",
	"sex": "male",
	"age": 35,
	"coffee": {
		"type": "espresso"
	}
}

You can specify as many words with trailing colons as you like and it will create those keys, eg. the message:

setparse coffee origin: roast: age:

…would create:

{
	"name": "Alex",
	"sex": "male",
	"age": 35,
	"coffee": {
		"origin": "*",
		"roast": "*",
		"age": "*"
	}
}

…and Max will store placeholder text ("*") at those keys (if no value is listed after each key). Note though that the type key disappeared. When you set content (and this includes setparse), it overwrites existing content at that key. It is sometimes best to create a key with setparse:

{
	"name": "Alex",
	"sex": "male",
	"age": 35
}
setparse coffee type: espresso

… then append the elements one at a time like this:

append coffee::origin *
append coffee::roast *
append coffee::age *

This will retain the four keys (type:, origin:, roast:, and age:)

Making a key store an array of dictionaries.

Lastly, if you want an item stored at a key to be an array of dictionaries, there is a cool thing you can do to achieve this (that, as far as I can see is undocumented in the help patches).

Let’s try to create this structure:

{
    "type": "FeatureCollection",
    "features": [
        {
            "type": "Feature",
            "geometry": {
                "type": "Point",
                "coordinates": [ 150.1282427, -24.471803 ]
            },
            "properties": {
                "type": "town"
            }
        }
    ]
}

Here is a list of messages (with a comment explaining what each does):

set type FeatureCollection // create a key called 'type' and assign it the value 'FeatureCollection'
set features // create an 'empty key' called 'features'
append features // this is a crucial step - this turns features' value into an empty array
setparse features[0] type: geometry: properties: // creates an object with three keys under the first 'features' key of the array
set features[0]::type Feature // as with the last step, we need to ensure we address the items with square bracket notation now that it's an array


setparse features[0]::geometry type: coordinates: // add a key with a dictionary value (with its own two keys) to 'features'
set features[0]::geometry::type Point // set the value of 'type' within the geometry dictionary
set features[0]::geometry::coordinates 150.12825 -24.471804 // set the value of 'coordinates' within the geometry dictionary to an array of floats


setparse features[0]::geometry type: Point coordinates: 150.12825 -24.471804 // or the previous 3 lines all in one step

setparse features[0]::properties type: town // create a new key 'properties' and set its content as a dictionary

Optional: should you wish to extend the length of the ‘features’ array, try:

append features * // append some dummy data to the 'features' array, then...
setparse features[1] type: geometry: properties: // add the keys
append features * // again, extend the 'features' array
setparse features[2] type: geometry: properties: // add keys to the third item in the array
append features * // and again, extend the 'features' array
setparse features[3] type: geometry: properties: // ...you get the idea.

Building GeoJSON data example patch

A comprehensive tutorial (aside from this vignette) from Cycling ’74 is still very much desired, but in the meantime check out the help patch below for some examples of how to create complex dictionary structures.

5 comments

  4. Thank you so much and shame on Cycling 74 not providing any useful help on building and retrieving more complex structures. Your sample patch should be part of the official documentation.

    GOLD: I now understand how I get any substructure in its own dictionary.

  5. Absolutely fantastic work! Thank you so much for these incredibly clear and thorough explanation and patch.

Leave a Reply

Your email address will not be published. Required fields are marked *