Python and NumPy: Time and Date (datetime and timedelta64)

This guide is obsolete, it was written during the early stages when I was learning python. A more up to date guide is available here:

Other python guides including installation of python are available in:

For timing NumPy has the inbuilt function datetime64 which can be used to express the date and time and the function timedelta64 which is used to specify the difference between two times or duration of an event. A list (vector) of dates or time differences can also be created.

Tutorial Video

Spyder Variable Editor Notes

Currently Spyder does not have the means to display datetime64, timedelta64, or arrays of these types properly in the variable editor.

They show up but one cannot view their contents unless one uses the command print. Double clicking times, the datetime64[D] array object yields the following error:

Years

The year isn't a SI unit, it is defined as the time it takes the earth to rotate around the sun.

We can specify time as a year by using:

2019

We can have a look at its data type using:

datetime64[Y]

-1 years

Here diffyear is negative because year2 was after year1. In other words year2 was a year ago respect to year1. We can look at the datatype of diffyear:

timedelta64[Y]

We see that this difference in type has a different data type, timedelta64. Both the year and the difference between years have the format in years. The difference can be added to year2 to give a new variable year3

Again we are adding a negative difference in time or duration to year 2. This gives:

2017

We can have a look at the datatype using:

datetime64[Y]

A timedelta64 variable can also directly be created for example:

We should see this has a data type of timedelta64:

timedelta64

Once again we can add this timedelta64 duration to a datetime64 date to get a new date:

2021
datetime64[Y]

Years-Months (YYYY-MM)

The month once again isn't an SI unit and is one of the most messed up units that is still commonly used currently.

Metric time has never fully been fully adopted and the conventions used for time adopts a combination of two ancient counting systems, the decimal system, which is base ten used by the Romans and the base 12 and base 60 which were used by the Babylonians. Both these counting systems relate to the way one counts using their fingers.

Historically the duration of a year was divided up into 10 months by using the decimal system by King Romulus. His 10 month calendar is as follows. As the number of perceived days in a year 304 did not divide perfectly by 10 some months had 30 days and some other months had 31 days.

Unfortunately his 304 day year calendar was found to be short when it came to the alignment of seasonal events such as the weather patterns from year to year. This lead to an update by King Numa who made a 355 day calendar by adding an additional 2 months. This 355 day calendar was much more accurate than the 304 day calendar but he failed to keep up with the two conventions from the original calendar. Instead of just making some months 35 days and others 36 days, he removed the decimal convention of assigning 10 months and instead added 2 additional months. Moreover instead of adding the 2 new months to the end of the calendar, he placed self-importance in his own work and placed them at the start of the calendar year. This broke the mapping between the month name and the meaning of the month name. An Octopus for example has 8 legs, a Octogon has 8 sides but October is the 10 month (it was originally the 8th month).

12 months relates to the base 12 and base 60 counting system used for smaller time units by the Babylonians. Instead of counting each finger and thumb individually, they counted using only their fingers but noted that each finger was divided into 3 segments which gave 12 for one hand.

This is also seen on an analogue clock where the clock shows 12 hours (half a day) and two half days (two clock cycles) make up a day. Although King Numa was more accurate 355 days in a year vs 304 days, he was ultimately still out by 10 and a quarter days. The Calendar was improved by Julius Caesar and Augustus who found that it took 365.25 days for the earth to rotate around the sun and they renamed two of the months names after themselves. This is the basis of the calendar we use today:

The month is in essence a failed attempt to divide the year up into 10 and later 12 equally spaced parts however there is a variation of 3 days. February in a non-leap year has 28 days and January has 31 days giving an approximate 10 percent error in terms of number of days per month.

To get the months we use a dash – to separate the year and the month other symbols such as / do not work in this case. The month has to expressed as a 2 digit number. Using 1 opposed to 01 for January will throw up an error:

2019-01

Once again we can look at the duration or difference between two months and have a look at the data types:

2019-08
datetime64[M]
2018-02
datetime64[M]
18 months
timedelta64[M]

24 months
timedelta64[M]

17 months
timedelta64[M]

Years-Months-Days (YYYY-MM-DD)

The day also isn't an SI unit, it is defined as the time it takes the earth to spin around its own axis.

2019-08-19
datetime64[D]
2018-02-20
datetime64[D]
545 days
timedelta64[D]

Inconsistencies in Time Measurements

Note if a duration is expressed in terms of months and days or years and days the following error message shows up:

TypeError: Cannot get a common metadata divisor for NumPy datetime metadata [M] and [D] because they have incompatible nonlinear base time units

This is due to the inconsistency of the number of days in a month and the number of days in a year… as mentioned above. In other words the units years are (365±1 days) and months (30±2 days) don't have the same accuracy as the units days. This inconsistency is reflected in the rhyme:

Thirty days hath September,
April, June, and November, 
all the rest have thirty-one. 
February has twenty-eight, 
but leap year coming one in four
February then has one day more.

As a result of the inaccuracy of Months and Years in terms of Days, Python cannot accurately sum the former two units alongside Days as they don't have the accuracy of the unit Day. Essentially the duration should be expressed exclusively in days alone which have a fixed unit.

397 days
timedelta64[D]

Years-Months-Days-TimeHours (YYYY-MM-DDThh)

The day also isn't an SI unit. The number of hours in a day relates to the base 12 and base 60 counting system used for smaller time units by the Babylonians. Instead of counting each finger and thumb individually, they counted using only their fingers but noted that each finger was divided into 3 segments which gave 12 for one hand.

If we use both hands to count in this manner we instead get 2×12 hour clocks which give 24 hours, the number of hours in a day.

The date is separated from the time by using a T. i.e. in the format 'YYYY-MM-DDThh'

2019-08-19T20
datetime64[h]
2019-08-18T23
datetime64[h]
21 hours
timedelta64[h]

Note that for hours, the dtype is lower case h opposed to the upper case Y, M and D for years, months and days. This distinction using upper case for the date and lower case for the time is deliberate to distinguish months abbreviated M from minutes abbreviated m. A duration can be expressed in terms of Days and Hours as a Day is exactly 24 Hours.

25 hours
timedelta64[h]

Years-Months-Days-TimeHours:Minutes (YYYY-MM-DDThh:mm)

The minute also isn't an SI unit. The number of minutes in an hour once again relates to the base 12 and base 60 counting system used for smaller time units by the Babylonians. Recalling that one can count to 12 using their left hand.

Instead of using ones second hand to count to 24, they can instead use each finger on their second hand to count a multiple of 12. 4 fingers in the right hand corresponding to 12 gives 48, this when added to the 12 counted on the left hand sums to 60.

The Babylonians used the base 60 to split an hour into minutes and also to split a minute into seconds.

The Babylonians used the base 60 to split an hour into minutes and also to split a minute into seconds. To specify a time to the precision of minutes, we use a : to split the hours hh and the minutes mm

-4 minutes
timedelta64[m]

A duration of Days, hours and minutes can be created.

1501 minutes
timedelta64[m]

Years-Months-Days-TimeHours:Minutes:Seconds (YYYY-MM-DDThh:mm:ss)

The minute is divided up into 60 seconds and the second is an SI unit. To specify a time in seconds, we use a colon : to split the hours hh and the minutes mm and then another colon : to split the minutes mm and seconds ss.

2019-08-19T20:01:01
datetime64[s]
2019-08-19T20:05:02
datetime64[s]
-241 seconds
timedelta64[s]

A duration consisting of Days, hours, minutes and seconds can be created.

90061 seconds
timedelta64[s]

Date Times Specified to Units Below a Second

Unlike all other time units discussed, the second is a SI unit and units below 1 second are expressed in SI form using the decimal point. For example:

2019-08-19T20:01:01.001
datetime64[ms]
2019-08-19T20:05:02
datetime64[s]
-240999 milliseconds

In scientific notation every 1 thousand of a unit has its own abbreviation. For time these abbreviations are as follows.

abbreviation	name	time in s
ms	millisecond	0.001
us	microsecond	0.000 001
ns	nanosecond	0.000 000 001
ps	picosecond	0.000 000 000 001
fs	femtosecond	0.000 000 000 000 001
as	attosecond	0.000 000 000 000 000 001

These abbreviations can be used within duration

90061001001 microseconds
timedelta64[us]

Creating a List (Vector) of Dates

The function arange can be used with the start set to the start date, the stop set to the last date (bare in mind 0 order indexing is used, so we go up to but don't include the stop date) and the step should be set to a duration (not a date). For example if we want every date from today (the date I wrote this) until the 31st of October but not including the 31st of October itself we can use:

['2019-08-21' '2019-08-22' '2019-08-23' '2019-08-24' '2019-08-25'
 '2019-08-26' '2019-08-27' '2019-08-28' '2019-08-29' '2019-08-30'
 '2019-08-31' '2019-09-01' '2019-09-02' '2019-09-03' '2019-09-04'
 '2019-09-05' '2019-09-06' '2019-09-07' '2019-09-08' '2019-09-09'
 '2019-09-10' '2019-09-11' '2019-09-12' '2019-09-13' '2019-09-14'
 '2019-09-15' '2019-09-16' '2019-09-17' '2019-09-18' '2019-09-19'
 '2019-09-20' '2019-09-21' '2019-09-22' '2019-09-23' '2019-09-24'
 '2019-09-25' '2019-09-26' '2019-09-27' '2019-09-28' '2019-09-29'
 '2019-09-30' '2019-10-01' '2019-10-02' '2019-10-03' '2019-10-04'
 '2019-10-05' '2019-10-06' '2019-10-07' '2019-10-08' '2019-10-09'
 '2019-10-10' '2019-10-11' '2019-10-12' '2019-10-13' '2019-10-14'
 '2019-10-15' '2019-10-16' '2019-10-17' '2019-10-18' '2019-10-19'
 '2019-10-20' '2019-10-21' '2019-10-22' '2019-10-23' '2019-10-24'
 '2019-10-25' '2019-10-26' '2019-10-27' '2019-10-28' '2019-10-29'
 '2019-10-30']

datetime64[D]

Creating a List (Vector) of Durations

It is also possible to create a List (Vector) of Durations by having, the start, stop and step all set as durations. For example if we want an array from 0 s up to 15 s but not including 15 (0 order indexing) we can use:

[ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14]

timedelta64[s]

Get Current Date or Time

It is possible to get the current date, or current time using the input arguments today or now in datetime64:

2019-08-21
2019-08-21T16:19:34

To get a higher precision for the time a second input argument can be obtained.

2019-08-21T16:22:08.000