Access nested dictionary items via a list of keys?

Posted on

Solving problem is about exposing yourself to as many situations as possible like Access nested dictionary items via a list of keys? and practice these strategies over and over. With time, it becomes second nature and a natural way you approach any problems in general. Big or small, always start with a plan, use other strategies mentioned here till you are confident and ready to code the solution.
In this post, my aim is to share an overview the topic about Access nested dictionary items via a list of keys?, which can be followed any time. Take easy to follow this discuss.

Access nested dictionary items via a list of keys?

I have a complex dictionary structure which I would like to access via a list of keys to address the correct item.

dataDict = {
    "a":{
        "r": 1,
        "s": 2,
        "t": 3
        },
    "b":{
        "u": 1,
        "v": {
            "x": 1,
            "y": 2,
            "z": 3
        },
        "w": 3
        }
}
maplist = ["a", "r"]

or

maplist = ["b", "v", "y"]

I have made the following code which works but I’m sure there is a better and more efficient way to do this if anyone has an idea.

# Get a given data from a dictionary with position provided as a list
def getFromDict(dataDict, mapList):
    for k in mapList: dataDict = dataDict[k]
    return dataDict
# Set a given data in a dictionary with position provided as a list
def setInDict(dataDict, mapList, value):
    for k in mapList[:-1]: dataDict = dataDict[k]
    dataDict[mapList[-1]] = value
Asked By: kolergy

||

Answer #1:

Use reduce() to traverse the dictionary:

from functools import reduce  # forward compatibility for Python 3
import operator
def getFromDict(dataDict, mapList):
    return reduce(operator.getitem, mapList, dataDict)

and reuse getFromDict to find the location to store the value for setInDict():

def setInDict(dataDict, mapList, value):
    getFromDict(dataDict, mapList[:-1])[mapList[-1]] = value

All but the last element in mapList is needed to find the ‘parent’ dictionary to add the value to, then use the last element to set the value to the right key.

Demo:

>>> getFromDict(dataDict, ["a", "r"])
1
>>> getFromDict(dataDict, ["b", "v", "y"])
2
>>> setInDict(dataDict, ["b", "v", "w"], 4)
>>> import pprint
>>> pprint.pprint(dataDict)
{'a': {'r': 1, 's': 2, 't': 3},
 'b': {'u': 1, 'v': {'w': 4, 'x': 1, 'y': 2, 'z': 3}, 'w': 3}}

Note that the Python PEP8 style guide prescribes snake_case names for functions. The above works equally well for lists or a mix of dictionaries and lists, so the names should really be get_by_path() and set_by_path():

from functools import reduce  # forward compatibility for Python 3
import operator
def get_by_path(root, items):
    """Access a nested object in root by item sequence."""
    return reduce(operator.getitem, items, root)
def set_by_path(root, items, value):
    """Set a value in a nested object in root by item sequence."""
    get_by_path(root, items[:-1])[items[-1]] = value

And for completion’s sake, a function to delete a key:

def del_by_path(root, items):
    """Delete a key-value in a nested object in root by item sequence."""
    del get_by_path(root, items[:-1])[items[-1]]
Answered By: Martijn Pieters

Answer #2:

It seems more pythonic to use a for loop.
See the quote from What’s New In Python 3.0.

Removed reduce(). Use functools.reduce() if you really need it; however, 99 percent of the time an explicit for loop is more readable.

def nested_get(dic, keys):
    for key in keys:
        dic = dic[key]
    return dic

Note that the accepted solution doesn’t set non-existing nested keys (it raises KeyError). Using the approach below will create non-existing nodes instead:

def nested_set(dic, keys, value):
    for key in keys[:-1]:
        dic = dic.setdefault(key, {})
    dic[keys[-1]] = value

The code works in both Python 2 and 3.

Answered By: DomTomCat

Answer #3:

Using reduce is clever, but the OP’s set method may have issues if the parent keys do not pre-exist in the nested dictionary. Since this is the first SO post I saw for this subject in my google search, I would like to make it slightly better.

The set method in ( Setting a value in a nested python dictionary given a list of indices and value ) seems more robust to missing parental keys. To copy it over:

def nested_set(dic, keys, value):
    for key in keys[:-1]:
        dic = dic.setdefault(key, {})
    dic[keys[-1]] = value

Also, it can be convenient to have a method that traverses the key tree and get all the absolute key paths, for which I have created:

def keysInDict(dataDict, parent=[]):
    if not isinstance(dataDict, dict):
        return [tuple(parent)]
    else:
        return reduce(list.__add__,
            [keysInDict(v,parent+[k]) for k,v in dataDict.items()], [])

One use of it is to convert the nested tree to a pandas DataFrame, using the following code (assuming that all leafs in the nested dictionary have the same depth).

def dict_to_df(dataDict):
    ret = []
    for k in keysInDict(dataDict):
        v = np.array( getFromDict(dataDict, k), )
        v = pd.DataFrame(v)
        v.columns = pd.MultiIndex.from_product(list(k) + [v.columns])
        ret.append(v)
    return reduce(pd.DataFrame.join, ret)
Answered By: eafit

Answer #4:

This library may be helpful: https://github.com/akesterson/dpath-python

A python library for accessing and searching dictionaries via
/slashed/paths ala xpath

Basically it lets you glob over a dictionary as if it were a
filesystem.

Answered By: dmmfll

Answer #5:

How about using recursive functions?

To get a value:

def getFromDict(dataDict, maplist):
    first, rest = maplist[0], maplist[1:]
    if rest:
        # if `rest` is not empty, run the function recursively
        return getFromDict(dataDict[first], rest)
    else:
        return dataDict[first]

And to set a value:

def setInDict(dataDict, maplist, value):
    first, rest = maplist[0], maplist[1:]
    if rest:
        try:
            if not isinstance(dataDict[first], dict):
                # if the key is not a dict, then make it a dict
                dataDict[first] = {}
        except KeyError:
            # if key doesn't exist, create one
            dataDict[first] = {}
        setInDict(dataDict[first], rest, value)
    else:
        dataDict[first] = value
Answered By: xyres

Answer #6:

Solved this with recursion:

def get(d,l):
    if len(l)==1: return d[l[0]]
    return get(d[l[0]],l[1:])

Using your example:

dataDict = {
    "a":{
        "r": 1,
        "s": 2,
        "t": 3
        },
    "b":{
        "u": 1,
        "v": {
            "x": 1,
            "y": 2,
            "z": 3
        },
        "w": 3
        }
}
maplist1 = ["a", "r"]
maplist2 = ["b", "v", "y"]
print(get(dataDict, maplist1)) # 1
print(get(dataDict, maplist2)) # 2
Answered By: Poh Zi How

Answer #7:

Instead of taking a performance hit each time you want to look up a value, how about you flatten the dictionary once then simply look up the key like b:v:y

def flatten(mydict):
  new_dict = {}
  for key,value in mydict.items():
    if type(value) == dict:
      _dict = {':'.join([key, _key]):_value for _key, _value in flatten(value).items()}
      new_dict.update(_dict)
    else:
      new_dict[key]=value
  return new_dict
dataDict = {
"a":{
    "r": 1,
    "s": 2,
    "t": 3
    },
"b":{
    "u": 1,
    "v": {
        "x": 1,
        "y": 2,
        "z": 3
    },
    "w": 3
    }
}
flat_dict = flatten(dataDict)
print flat_dict
{'b:w': 3, 'b:u': 1, 'b:v:y': 2, 'b:v:x': 1, 'b:v:z': 3, 'a:r': 1, 'a:s': 2, 'a:t': 3}

This way you can simply look up items using flat_dict['b:v:y'] which will give you 1.

And instead of traversing the dictionary on each lookup, you may be able to speed this up by flattening the dictionary and saving the output so that a lookup from cold start would mean loading up the flattened dictionary and simply performing a key/value lookup with no traversal.

Answered By: OkezieE

Answer #8:

Pure Python style, without any import:

def nested_set(element, value, *keys):
    if type(element) is not dict:
        raise AttributeError('nested_set() expects dict as first argument.')
    if len(keys) < 2:
        raise AttributeError('nested_set() expects at least three arguments, not enough given.')
    _keys = keys[:-1]
    _element = element
    for key in _keys:
        _element = _element[key]
    _element[keys[-1]] = value
example = {"foo": { "bar": { "baz": "ok" } } }
keys = ['foo', 'bar']
nested_set(example, "yay", *keys)
print(example)

Output

{'foo': {'bar': 'yay'}}
Answered By: Arount

Leave a Reply

Your email address will not be published.