# Access nested dictionary items via a list of keys?

Posted on

Solving problem is about exposing yourself to as many situations as possible like Access nested dictionary items via a list of keys? and practice these strategies over and over. With time, it becomes second nature and a natural way you approach any problems in general. Big or small, always start with a plan, use other strategies mentioned here till you are confident and ready to code the solution.
In this post, my aim is to share an overview the topic about Access nested dictionary items via a list of keys?, which can be followed any time. Take easy to follow this discuss.

Access nested dictionary items via a list of keys?

I have a complex dictionary structure which I would like to access via a list of keys to address the correct item.

``````dataDict = {
"a":{
"r": 1,
"s": 2,
"t": 3
},
"b":{
"u": 1,
"v": {
"x": 1,
"y": 2,
"z": 3
},
"w": 3
}
}
maplist = ["a", "r"]
``````

or

``````maplist = ["b", "v", "y"]
``````

I have made the following code which works but I’m sure there is a better and more efficient way to do this if anyone has an idea.

``````# Get a given data from a dictionary with position provided as a list
# Set a given data in a dictionary with position provided as a list
``````

Use `reduce()` to traverse the dictionary:

``````from functools import reduce  # forward compatibility for Python 3
import operator
``````

and reuse `getFromDict` to find the location to store the value for `setInDict()`:

``````def setInDict(dataDict, mapList, value):
``````

All but the last element in `mapList` is needed to find the ‘parent’ dictionary to add the value to, then use the last element to set the value to the right key.

Demo:

``````>>> getFromDict(dataDict, ["a", "r"])
1
2
>>> setInDict(dataDict, ["b", "v", "w"], 4)
>>> import pprint
{'a': {'r': 1, 's': 2, 't': 3},
'b': {'u': 1, 'v': {'w': 4, 'x': 1, 'y': 2, 'z': 3}, 'w': 3}}
``````

Note that the Python PEP8 style guide prescribes snake_case names for functions. The above works equally well for lists or a mix of dictionaries and lists, so the names should really be `get_by_path()` and `set_by_path()`:

``````from functools import reduce  # forward compatibility for Python 3
import operator
def get_by_path(root, items):
"""Access a nested object in root by item sequence."""
return reduce(operator.getitem, items, root)
def set_by_path(root, items, value):
"""Set a value in a nested object in root by item sequence."""
get_by_path(root, items[:-1])[items[-1]] = value
``````

And for completion’s sake, a function to delete a key:

``````def del_by_path(root, items):
"""Delete a key-value in a nested object in root by item sequence."""
del get_by_path(root, items[:-1])[items[-1]]
``````

It seems more pythonic to use a `for` loop.
See the quote from What’s New In Python 3.0.

Removed `reduce()`. Use `functools.reduce()` if you really need it; however, 99 percent of the time an explicit `for` loop is more readable.

``````def nested_get(dic, keys):
for key in keys:
dic = dic[key]
return dic
``````

Note that the accepted solution doesn’t set non-existing nested keys (it raises `KeyError`). Using the approach below will create non-existing nodes instead:

``````def nested_set(dic, keys, value):
for key in keys[:-1]:
dic = dic.setdefault(key, {})
dic[keys[-1]] = value
``````

The code works in both Python 2 and 3.

Using reduce is clever, but the OP’s set method may have issues if the parent keys do not pre-exist in the nested dictionary. Since this is the first SO post I saw for this subject in my google search, I would like to make it slightly better.

The set method in ( Setting a value in a nested python dictionary given a list of indices and value ) seems more robust to missing parental keys. To copy it over:

``````def nested_set(dic, keys, value):
for key in keys[:-1]:
dic = dic.setdefault(key, {})
dic[keys[-1]] = value
``````

Also, it can be convenient to have a method that traverses the key tree and get all the absolute key paths, for which I have created:

``````def keysInDict(dataDict, parent=[]):
return [tuple(parent)]
else:
[keysInDict(v,parent+[k]) for k,v in dataDict.items()], [])
``````

One use of it is to convert the nested tree to a pandas DataFrame, using the following code (assuming that all leafs in the nested dictionary have the same depth).

``````def dict_to_df(dataDict):
ret = []
v = np.array( getFromDict(dataDict, k), )
v = pd.DataFrame(v)
v.columns = pd.MultiIndex.from_product(list(k) + [v.columns])
ret.append(v)
return reduce(pd.DataFrame.join, ret)
``````

This library may be helpful: https://github.com/akesterson/dpath-python

A python library for accessing and searching dictionaries via
/slashed/paths ala xpath

Basically it lets you glob over a dictionary as if it were a
filesystem.

To get a value:

``````def getFromDict(dataDict, maplist):
first, rest = maplist[0], maplist[1:]
if rest:
# if `rest` is not empty, run the function recursively
else:
``````

And to set a value:

``````def setInDict(dataDict, maplist, value):
first, rest = maplist[0], maplist[1:]
if rest:
try:
# if the key is not a dict, then make it a dict
except KeyError:
# if key doesn't exist, create one
else:
``````

Solved this with recursion:

``````def get(d,l):
if len(l)==1: return d[l[0]]
return get(d[l[0]],l[1:])
``````

``````dataDict = {
"a":{
"r": 1,
"s": 2,
"t": 3
},
"b":{
"u": 1,
"v": {
"x": 1,
"y": 2,
"z": 3
},
"w": 3
}
}
maplist1 = ["a", "r"]
maplist2 = ["b", "v", "y"]
``````

Instead of taking a performance hit each time you want to look up a value, how about you flatten the dictionary once then simply look up the key like `b:v:y`

``````def flatten(mydict):
new_dict = {}
for key,value in mydict.items():
if type(value) == dict:
_dict = {':'.join([key, _key]):_value for _key, _value in flatten(value).items()}
new_dict.update(_dict)
else:
new_dict[key]=value
return new_dict
"a":{
"r": 1,
"s": 2,
"t": 3
},
"b":{
"u": 1,
"v": {
"x": 1,
"y": 2,
"z": 3
},
"w": 3
}
}
print flat_dict
{'b:w': 3, 'b:u': 1, 'b:v:y': 2, 'b:v:x': 1, 'b:v:z': 3, 'a:r': 1, 'a:s': 2, 'a:t': 3}
``````

This way you can simply look up items using `flat_dict['b:v:y']` which will give you `1`.

And instead of traversing the dictionary on each lookup, you may be able to speed this up by flattening the dictionary and saving the output so that a lookup from cold start would mean loading up the flattened dictionary and simply performing a key/value lookup with no traversal.

Pure Python style, without any import:

``````def nested_set(element, value, *keys):
if type(element) is not dict:
raise AttributeError('nested_set() expects dict as first argument.')
if len(keys) < 2:
raise AttributeError('nested_set() expects at least three arguments, not enough given.')
_keys = keys[:-1]
_element = element
for key in _keys:
_element = _element[key]
_element[keys[-1]] = value
example = {"foo": { "bar": { "baz": "ok" } } }
keys = ['foo', 'bar']
nested_set(example, "yay", *keys)
print(example)
``````

Output

``````{'foo': {'bar': 'yay'}}
``````