Python and NumPy: Time and Date (datetime and timedelta64)

For timing NumPy has the inbuilt function datetime64 which can be used to express the date and time and the function timedelta64 which is used to specify the difference between two times or duration of an event. A list (vector) of dates or time differences can also be created.

Tutorial Video

Spyder Variable Editor Notes

Currently Spyder does not have the means to display datetime64, timedelta64, or arrays of these types properly in the variable editor.

They show up but one cannot view their contents unless one uses the command print. Double clicking times, the datetime64[D] array object yields the following error:

Years

The year isn’t a SI unit, it is defined as the time it takes the earth to rotate around the sun.

Python

We can specify time as a year by using:

Python
2019

We can have a look at its data type using:

Python
datetime64[Y]
Python
-1 years

Here diffyear is negative because year2 was after year1. In other words year2 was a year ago respect to year1. We can look at the datatype of diffyear:

Python
timedelta64[Y]

We see that this difference in type has a different data type, timedelta64. Both the year and the difference between years have the format in years. The difference can be added to year2 to give a new variable year3

Python

Again we are adding a negative difference in time or duration to year 2. This gives:

2017

We can have a look at the datatype using:

Python
datetime64[Y]

A timedelta64 variable can also directly be created for example:

Python

We should see this has a data type of timedelta64:

Python
timedelta64

Once again we can add this timedelta64 duration to a datetime64 date to get a new date:

Python
2021
datetime64[Y]

Years-Months (YYYY-MM)

The month once again isn’t an SI unit and is one of the most messed up units that is still commonly used currently.

Metric time has never fully been fully adopted and the conventions used for time adopts a combination of two ancient counting systems, the decimal system, which is base ten used by the Romans and the base 12 and base 60 which were used by the Babylonians. Both these counting systems relate to the way one counts using their fingers.

The Romans adopted Base 10 which is based around the fact than one possesses 10 fingers. For Base 10 there are 10 characters 0,1,2,3,4,5,6,7,8,9 but one typically counts using 1st order indexing 1,2,3,4,5,6,7,8,9. Base 10 is also known as the Decimal System and is the basis of the metric system.

Historically the duration of a year was divided up into 10 months by using the decimal system by King Romulus. His 10 month calendar is as follows. As the number of perceived days in a year 304 did not divide perfectly by 10 some months had 30 days and some other months had 31 days.

King Romulus Calendar consisted of 10 months and 304 Days.

Unfortunately his 304 day year calendar was found to be short when it came to the alignment of seasonal events such as the weather patterns from year to year. This lead to an update by King Numa who made a 355 day calendar by adding an additional 2 months. This 355 day calendar was much more accurate than the 304 day calendar but he failed to keep up with the two conventions from the original calendar. Instead of just making some months 35 days and others 36 days, he removed the decimal convention of assigning 10 months and instead added 2 additional months. Moreover instead of adding the 2 new months to the end of the calendar, he placed self-importance in his own work and placed them at the start of the calendar year. This broke the mapping between the month name and the meaning of the month name. An Octopus for example has 8 legs, a Octogon has 8 sides but October is the 10 month (it was originally the 8th month).

King Numa’s Calendar consisted of 12 months and 355 Days.

12 months relates to the base 12 and base 60 counting system used for smaller time units by the Babylonians. Instead of counting each finger and thumb individually, they counted using only their fingers but noted that each finger was divided into 3 segments which gave 12 for one hand.

Using one hand to count to count to 12 which relates to 12 hours on an analogue clock.

This is also seen on an analogue clock where the clock shows 12 hours (half a day) and two half days (two clock cycles) make up a day. Although King Numa was more accurate 355 days in a year vs 304 days, he was ultimately still out by 10 and a quarter days. The Calendar was improved by Julius Caesar and Augustus who found that it took 365.25 days for the earth to rotate around the sun and they renamed two of the months names after themselves. This is the basis of the calendar we use today:

The Julian Calendar consisted of 12 months and 365.25 days

The month is in essence a failed attempt to divide the year up into 10 and later 12 equally spaced parts however there is a variation of 3 days. February in a non-leap year has 28 days and January has 31 days giving an approximate 10 percent error in terms of number of days per month.

To get the months we use a dash – to separate the year and the month other symbols such as / do not work in this case. The month has to expressed as a 2 digit number. Using 1 opposed to 01 for January will throw up an error:

Python
2019-01

Once again we can look at the duration or difference between two months and have a look at the data types:

Python
2019-08
datetime64[M]
2018-02
datetime64[M]
18 months
timedelta64[M]
Python
24 months
timedelta64[M]
Python
17 months
timedelta64[M]

Years-Months-Days (YYYY-MM-DD)

The day also isn’t an SI unit, it is defined as the time it takes the earth to spin around its own axis.

Python
2019-08-19
datetime64[D]
2018-02-20
datetime64[D]
545 days
timedelta64[D]

Inconsistencies in Time Measurements

Note if a duration is expressed in terms of months and days or years and days the following error message shows up:

Python
TypeError: Cannot get a common metadata divisor for NumPy datetime metadata [M] and [D] because they have incompatible nonlinear base time units

This is due to the inconsistency of the number of days in a month and the number of days in a year… as mentioned above. In other words the units years are (365±1 days) and months (30±2 days) don’t have the same accuracy as the units days. This inconsistency is reflected in the rhyme:

Thirty days hath September,
April, June, and November, 
all the rest have thirty-one. 
February has twenty-eight, 
but leap year coming one in four
February then has one day more.

As a result of the inaccuracy of Months and Years in terms of Days, Python cannot accurately sum the former two units alongside Days as they don’t have the accuracy of the unit Day. Essentially the duration should be expressed exclusively in days alone which have a fixed unit.

Python
397 days
timedelta64[D]

Years-Months-Days-TimeHours (YYYY-MM-DDThh)

The day also isn’t an SI unit. The number of hours in a day relates to the base 12 and base 60 counting system used for smaller time units by the Babylonians. Instead of counting each finger and thumb individually, they counted using only their fingers but noted that each finger was divided into 3 segments which gave 12 for one hand.

Using one hand to count to count to 12 which relates to 12 hours on an analogue clock.

If we use both hands to count in this manner we instead get 2×12 hour clocks which give 24 hours, the number of hours in a day.

Using both hands to count to 24

The date is separated from the time by using a T. i.e. in the format ‘YYYY-MM-DDThh’

Python
2019-08-19T20
datetime64[h]
2019-08-18T23
datetime64[h]
21 hours
timedelta64[h]

Note that for hours, the dtype is lower case h opposed to the upper case Y, M and D for years, months and days. This distinction using upper case for the date and lower case for the time is deliberate to distinguish months abbreviated M from minutes abbreviated m. A duration can be expressed in terms of Days and Hours as a Day is exactly 24 Hours.

Python
25 hours
timedelta64[h]

Years-Months-Days-TimeHours:Minutes (YYYY-MM-DDThh:mm)

The minute also isn’t an SI unit. The number of minutes in an hour once again relates to the base 12 and base 60 counting system used for smaller time units by the Babylonians. Recalling that one can count to 12 using their left hand.

Using one hand to count to count to 12 which relates to 12 hours on an analogue clock.

Instead of using ones second hand to count to 24, they can instead use each finger on their second hand to count a multiple of 12. 4 fingers in the right hand corresponding to 12 gives 48, this when added to the 12 counted on the left hand sums to 60.

Babylonian Base 60 Counting Using Both Hands

The Babylonians used the base 60 to split an hour into minutes and also to split a minute into seconds.

The Babylonians used the base 60 to split an hour into minutes and also to split a minute into seconds. To specify a time to the precision of minutes, we use a : to split the hours hh and the minutes mm

Python
-4 minutes
timedelta64[m]

A duration of Days, hours and minutes can be created.

Python
1501 minutes
timedelta64[m]

Years-Months-Days-TimeHours:Minutes:Seconds (YYYY-MM-DDThh:mm:ss)

The minute is divided up into 60 seconds and the second is an SI unit. To specify a time in seconds, we use a colon : to split the hours hh and the minutes mm and then another colon : to split the minutes mm and seconds ss.

Python
2019-08-19T20:01:01
datetime64[s]
2019-08-19T20:05:02
datetime64[s]
-241 seconds
timedelta64[s]

A duration consisting of Days, hours, minutes and seconds can be created.

Python
90061 seconds
timedelta64[s]

Date Times Specified to Units Below a Second

Unlike all other time units discussed, the second is a SI unit and units below 1 second are expressed in SI form using the decimal point. For example:

Python
2019-08-19T20:01:01.001
datetime64[ms]
2019-08-19T20:05:02
datetime64[s]
-240999 milliseconds

In scientific notation every 1 thousand of a unit has its own abbreviation. For time these abbreviations are as follows.

abbreviationnametime in s
msmillisecond0.001
usmicrosecond0.000 001
nsnanosecond0.000 000 001
pspicosecond0.000 000 000 001
fsfemtosecond0.000 000 000 000 001
asattosecond 0.000 000 000 000 000 001

These abbreviations can be used within duration

Python
90061001001 microseconds
timedelta64[us]

Creating a List (Vector) of Dates

The function arange can be used with the start set to the start date, the stop set to the last date (bare in mind 0 order indexing is used, so we go up to but don’t include the stop date) and the step should be set to a duration (not a date). For example if we want every date from today (the date I wrote this) until the 31st of October but not including the 31st of October itself we can use:

Python
['2019-08-21' '2019-08-22' '2019-08-23' '2019-08-24' '2019-08-25'
 '2019-08-26' '2019-08-27' '2019-08-28' '2019-08-29' '2019-08-30'
 '2019-08-31' '2019-09-01' '2019-09-02' '2019-09-03' '2019-09-04'
 '2019-09-05' '2019-09-06' '2019-09-07' '2019-09-08' '2019-09-09'
 '2019-09-10' '2019-09-11' '2019-09-12' '2019-09-13' '2019-09-14'
 '2019-09-15' '2019-09-16' '2019-09-17' '2019-09-18' '2019-09-19'
 '2019-09-20' '2019-09-21' '2019-09-22' '2019-09-23' '2019-09-24'
 '2019-09-25' '2019-09-26' '2019-09-27' '2019-09-28' '2019-09-29'
 '2019-09-30' '2019-10-01' '2019-10-02' '2019-10-03' '2019-10-04'
 '2019-10-05' '2019-10-06' '2019-10-07' '2019-10-08' '2019-10-09'
 '2019-10-10' '2019-10-11' '2019-10-12' '2019-10-13' '2019-10-14'
 '2019-10-15' '2019-10-16' '2019-10-17' '2019-10-18' '2019-10-19'
 '2019-10-20' '2019-10-21' '2019-10-22' '2019-10-23' '2019-10-24'
 '2019-10-25' '2019-10-26' '2019-10-27' '2019-10-28' '2019-10-29'
 '2019-10-30']

datetime64[D]

Creating a List (Vector) of Durations

It is also possible to create a List (Vector) of Durations by having, the start, stop and step all set as durations. For example if we want an array from 0 s up to 15 s but not including 15 (0 order indexing) we can use:

Python
[ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14]

timedelta64[s]

Get Current Date or Time

It is possible to get the current date, or current time using the input arguments today or now in datetime64:

Python
2019-08-21
2019-08-21T16:19:34

To get a higher precision for the time a second input argument can be obtained.

Python
2019-08-21T16:22:08.000
Advertisements

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.