Solving problem is about exposing yourself to as many situations as possible like How to overcome “datetime.datetime not JSON serializable”? and practice these strategies over and over. With time, it becomes second nature and a natural way you approach any problems in general. Big or small, always start with a plan, use other strategies mentioned here till you are confident and ready to code the solution.
In this post, my aim is to share an overview the topic about How to overcome “datetime.datetime not JSON serializable”?, which can be followed any time. Take easy to follow this discuss.
I have a basic dict as follows:
sample = {}
sample['title'] = "String"
sample['somedate'] = somedatetimehere
When I try to do jsonify(sample)
I get:
TypeError: datetime.datetime(2012, 8, 8, 21, 46, 24, 862000) is not JSON serializable
What can I do such that my dictionary sample can overcome the error above?
Note: Though it may not be relevant, the dictionaries are generated from the retrieval of records out of mongodb
where when I print out str(sample['somedate'])
, the output is 2012-08-08 21:46:24.862000
.
Answer #1:
Updated for 2018
The original answer accommodated the way MongoDB “date” fields were represented as:
{"$date": 1506816000000}
If you want a generic Python solution for serializing datetime
to json, check out @jjmontes’ answer for a quick solution which requires no dependencies.
As you are using mongoengine (per comments) and pymongo is a dependency, pymongo has built-in utilities to help with json serialization:
http://api.mongodb.org/python/1.10.1/api/bson/json_util.html
Example usage (serialization):
from bson import json_util
import json
json.dumps(anObject, default=json_util.default)
Example usage (deserialization):
json.loads(aJsonString, object_hook=json_util.object_hook)
Django
Django provides a native DjangoJSONEncoder
serializer that deals with this kind of properly.
See https://docs.djangoproject.com/en/dev/topics/serialization/#djangojsonencoder
from django.core.serializers.json import DjangoJSONEncoder
return json.dumps(
item,
sort_keys=True,
indent=1,
cls=DjangoJSONEncoder
)
One difference I’ve noticed between DjangoJSONEncoder
and using a custom default
like this:
import datetime
import json
def default(o):
if isinstance(o, (datetime.date, datetime.datetime)):
return o.isoformat()
return json.dumps(
item,
sort_keys=True,
indent=1,
default=default
)
Is that Django strips a bit of the data:
"last_login": "2018-08-03T10:51:42.990", # DjangoJSONEncoder
"last_login": "2018-08-03T10:51:42.990239", # default
So, you may need to be careful about that in some cases.
Answer #2:
My quick & dirty JSON dump that eats dates and everything:
json.dumps(my_dictionary, indent=4, sort_keys=True, default=str)
default
is a function applied to objects that aren’t serializable.
In this case it’s str
, so it just converts everything it doesn’t know to strings. Which is great for serialization but not so great when deserializing (hence the “quick & dirty”) as anything might have been string-ified without warning, e.g. a function or numpy array.
Answer #3:
Building on other answers, a simple solution based on a specific serializer that just converts datetime.datetime
and datetime.date
objects to strings.
from datetime import date, datetime
def json_serial(obj):
"""JSON serializer for objects not serializable by default json code"""
if isinstance(obj, (datetime, date)):
return obj.isoformat()
raise TypeError ("Type %s not serializable" % type(obj))
As seen, the code just checks to find out if object is of class datetime.datetime
or datetime.date
, and then uses .isoformat()
to produce a serialized version of it, according to ISO 8601 format, YYYY-MM-DDTHH:MM:SS (which is easily decoded by JavaScript). If more complex serialized representations are sought, other code could be used instead of str() (see other answers to this question for examples). The code ends by raising an exception, to deal with the case it is called with a non-serializable type.
This json_serial function can be used as follows:
from datetime import datetime
from json import dumps
print dumps(datetime.now(), default=json_serial)
The details about how the default parameter to json.dumps works can be found in Section Basic Usage of the json module documentation.
Answer #4:
I have just encountered this problem and my solution is to subclass json.JSONEncoder
:
from datetime import datetime
import json
class DateTimeEncoder(json.JSONEncoder):
def default(self, o):
if isinstance(o, datetime):
return o.isoformat()
return json.JSONEncoder.default(self, o)
In your call do something like: json.dumps(yourobj, cls=DateTimeEncoder)
The .isoformat()
I got from one of the answers above.
Answer #5:
Convert the date to a string
sample['somedate'] = str( datetime.utcnow() )
Answer #6:
For others who do not need or want to use the pymongo library for this.. you can achieve datetime JSON conversion easily with this small snippet:
def default(obj):
"""Default JSON serializer."""
import calendar, datetime
if isinstance(obj, datetime.datetime):
if obj.utcoffset() is not None:
obj = obj - obj.utcoffset()
millis = int(
calendar.timegm(obj.timetuple()) * 1000 +
obj.microsecond / 1000
)
return millis
raise TypeError('Not sure how to serialize %s' % (obj,))
Then use it like so:
import datetime, json
print json.dumps(datetime.datetime.now(), default=default)
output:
'1365091796124'
Answer #7:
Here is my solution:
import json
class DatetimeEncoder(json.JSONEncoder):
def default(self, obj):
try:
return super().default(obj)
except TypeError:
return str(obj)
Then you can use it like that:
json.dumps(dictionnary, cls=DatetimeEncoder)
Answer #8:
if you are using python3.7, then the best solution is using
datetime.isoformat()
and
datetime.fromisoformat()
; they work with both naive and
aware datetime
objects:
#!/usr/bin/env python3.7
from datetime import datetime
from datetime import timezone
from datetime import timedelta
import json
def default(obj):
if isinstance(obj, datetime):
return { '_isoformat': obj.isoformat() }
return super().default(obj)
def object_hook(obj):
_isoformat = obj.get('_isoformat')
if _isoformat is not None:
return datetime.fromisoformat(_isoformat)
return obj
if __name__ == '__main__':
#d = { 'now': datetime(2000, 1, 1) }
d = { 'now': datetime(2000, 1, 1, tzinfo=timezone(timedelta(hours=-8))) }
s = json.dumps(d, default=default)
print(s)
print(d == json.loads(s, object_hook=object_hook))
output:
{"now": {"_isoformat": "2000-01-01T00:00:00-08:00"}}
True
if you are using python3.6 or below, and you only care about the time value (not
the timezone), then you can use datetime.timestamp()
and
datetime.fromtimestamp()
instead;
if you are using python3.6 or below, and you do care about the timezone, then
you can get it via datetime.tzinfo
, but you have to serialize this field
by yourself; the easiest way to do this is to add another field _tzinfo
in the
serialized object;
finally, beware of precisions in all these examples;