How to overcome “datetime.datetime not JSON serializable”?

Posted on

Solving problem is about exposing yourself to as many situations as possible like How to overcome “datetime.datetime not JSON serializable”? and practice these strategies over and over. With time, it becomes second nature and a natural way you approach any problems in general. Big or small, always start with a plan, use other strategies mentioned here till you are confident and ready to code the solution.
In this post, my aim is to share an overview the topic about How to overcome “datetime.datetime not JSON serializable”?, which can be followed any time. Take easy to follow this discuss.

How to overcome “datetime.datetime not JSON serializable”?

I have a basic dict as follows:

sample = {}
sample['title'] = "String"
sample['somedate'] = somedatetimehere

When I try to do jsonify(sample) I get:

TypeError: datetime.datetime(2012, 8, 8, 21, 46, 24, 862000) is not JSON serializable

What can I do such that my dictionary sample can overcome the error above?

Note: Though it may not be relevant, the dictionaries are generated from the retrieval of records out of mongodb where when I print out str(sample['somedate']), the output is 2012-08-08 21:46:24.862000.

Asked By: Rolando

||

Answer #1:

Updated for 2018

The original answer accommodated the way MongoDB “date” fields were represented as:

{"$date": 1506816000000}

If you want a generic Python solution for serializing datetime to json, check out @jjmontes’ answer for a quick solution which requires no dependencies.


As you are using mongoengine (per comments) and pymongo is a dependency, pymongo has built-in utilities to help with json serialization:
http://api.mongodb.org/python/1.10.1/api/bson/json_util.html

Example usage (serialization):

from bson import json_util
import json
json.dumps(anObject, default=json_util.default)

Example usage (deserialization):

json.loads(aJsonString, object_hook=json_util.object_hook)

Django

Django provides a native DjangoJSONEncoder serializer that deals with this kind of properly.

See https://docs.djangoproject.com/en/dev/topics/serialization/#djangojsonencoder

from django.core.serializers.json import DjangoJSONEncoder
return json.dumps(
  item,
  sort_keys=True,
  indent=1,
  cls=DjangoJSONEncoder
)

One difference I’ve noticed between DjangoJSONEncoder and using a custom default like this:

import datetime
import json
def default(o):
    if isinstance(o, (datetime.date, datetime.datetime)):
        return o.isoformat()
return json.dumps(
  item,
  sort_keys=True,
  indent=1,
  default=default
)

Is that Django strips a bit of the data:

 "last_login": "2018-08-03T10:51:42.990", # DjangoJSONEncoder 
 "last_login": "2018-08-03T10:51:42.990239", # default

So, you may need to be careful about that in some cases.

Answered By: jdi

Answer #2:

My quick & dirty JSON dump that eats dates and everything:

json.dumps(my_dictionary, indent=4, sort_keys=True, default=str)

default is a function applied to objects that aren’t serializable.
In this case it’s str, so it just converts everything it doesn’t know to strings. Which is great for serialization but not so great when deserializing (hence the “quick & dirty”) as anything might have been string-ified without warning, e.g. a function or numpy array.

Answered By: jjmontes

Answer #3:

Building on other answers, a simple solution based on a specific serializer that just converts datetime.datetime and datetime.date objects to strings.

from datetime import date, datetime
def json_serial(obj):
    """JSON serializer for objects not serializable by default json code"""
    if isinstance(obj, (datetime, date)):
        return obj.isoformat()
    raise TypeError ("Type %s not serializable" % type(obj))

As seen, the code just checks to find out if object is of class datetime.datetime or datetime.date, and then uses .isoformat() to produce a serialized version of it, according to ISO 8601 format, YYYY-MM-DDTHH:MM:SS (which is easily decoded by JavaScript). If more complex serialized representations are sought, other code could be used instead of str() (see other answers to this question for examples). The code ends by raising an exception, to deal with the case it is called with a non-serializable type.

This json_serial function can be used as follows:

from datetime import datetime
from json import dumps
print dumps(datetime.now(), default=json_serial)

The details about how the default parameter to json.dumps works can be found in Section Basic Usage of the json module documentation.

Answered By: jgbarah

Answer #4:

I have just encountered this problem and my solution is to subclass json.JSONEncoder:

from datetime import datetime
import json
class DateTimeEncoder(json.JSONEncoder):
    def default(self, o):
        if isinstance(o, datetime):
            return o.isoformat()
        return json.JSONEncoder.default(self, o)

In your call do something like: json.dumps(yourobj, cls=DateTimeEncoder) The .isoformat() I got from one of the answers above.

Answered By: lenny

Answer #5:

Convert the date to a string

sample['somedate'] = str( datetime.utcnow() )
Answered By: D.A

Answer #6:

For others who do not need or want to use the pymongo library for this.. you can achieve datetime JSON conversion easily with this small snippet:

def default(obj):
    """Default JSON serializer."""
    import calendar, datetime
    if isinstance(obj, datetime.datetime):
        if obj.utcoffset() is not None:
            obj = obj - obj.utcoffset()
        millis = int(
            calendar.timegm(obj.timetuple()) * 1000 +
            obj.microsecond / 1000
        )
        return millis
    raise TypeError('Not sure how to serialize %s' % (obj,))

Then use it like so:

import datetime, json
print json.dumps(datetime.datetime.now(), default=default)

output: 

'1365091796124'
Answered By: Jay Taylor

Answer #7:

Here is my solution:

import json
class DatetimeEncoder(json.JSONEncoder):
    def default(self, obj):
        try:
            return super().default(obj)
        except TypeError:
            return str(obj)

Then you can use it like that:

json.dumps(dictionnary, cls=DatetimeEncoder)
Answered By: Natim

Answer #8:

if you are using python3.7, then the best solution is using
datetime.isoformat() and
datetime.fromisoformat(); they work with both naive and
aware datetime objects:

#!/usr/bin/env python3.7
from datetime import datetime
from datetime import timezone
from datetime import timedelta
import json
def default(obj):
    if isinstance(obj, datetime):
        return { '_isoformat': obj.isoformat() }
    return super().default(obj)
def object_hook(obj):
    _isoformat = obj.get('_isoformat')
    if _isoformat is not None:
        return datetime.fromisoformat(_isoformat)
    return obj
if __name__ == '__main__':
    #d = { 'now': datetime(2000, 1, 1) }
    d = { 'now': datetime(2000, 1, 1, tzinfo=timezone(timedelta(hours=-8))) }
    s = json.dumps(d, default=default)
    print(s)
    print(d == json.loads(s, object_hook=object_hook))

output:

{"now": {"_isoformat": "2000-01-01T00:00:00-08:00"}}
True

if you are using python3.6 or below, and you only care about the time value (not
the timezone), then you can use datetime.timestamp() and
datetime.fromtimestamp() instead;

if you are using python3.6 or below, and you do care about the timezone, then
you can get it via datetime.tzinfo, but you have to serialize this field
by yourself; the easiest way to do this is to add another field _tzinfo in the
serialized object;

finally, beware of precisions in all these examples;

Answered By: Cyker

Leave a Reply

Your email address will not be published. Required fields are marked *