Python:DataTypes

Object Classes

Lots of things to tell about strings, they have there own Python:Strings page.

For different number types we have Python:Numbers.

Also for numpy (module for scientific arithmetic) there is a special Numpy page.

Objects are iterable if they can contain more than 1 ordered objects (string, list, tuple, dict).

Objects are mutable if their content can be changed (list, set, dict)

isinstance(<obj>, <class>)

Boolean (returns True or False) to check if <obj> is an instance of <class>. Classnames are str, int, dict, list, set, tuple, range, ...

locals() and globals() are dictionaries holding the global and local variables so you can do:

if var in locals()

Check if a variable exists as local

if var in globals()

Check if a variable exists and is global.

globals().update(adict)

updates the globals with the dict. You can now address the keys as variable.

NOTE: Variables are pointers to objects, not the object itself. dict2 = dict1 does not make a new dictionary. Both variables point to the same dictionary. To make a copy use dict2 = dict(dict1). Same for lists, sets and tuples.

list

Class of iterable, mutable objects. Lists can be compared to arrays in other languages. Lists can contain a mixture of all kind of objects.

lst1 = []

Initialize an empty list

lst1 = [0] * 10

Create a list with 10 elements 0

lst1.append(2)

Add the '2' object to the end of lst1

lst1.extend(list2)

lst1 + lst2

Add the elements of lst2 to the end of lst1

lst1.pop(n)

Remove and return n^th element from lst1. Last element if n is not specified.

lst1.remove(x)

Remove object x from the list.

lst1 = list(object)

Convert object to a list (object is e.g. set, tuple or string)

count = lst1.count[x]

Return the number of occurrences of x in lst1

lst1.index(x)

Return the first position in lst1 where x is found

len(lst1) - 1 - lst1[::-1].index(x)

Return the last position in lst1 where x is found

bisect(lst1, x)

Return index of first value higher than x. bisect needs to be imported as from bisect import bisect

lst1.sort()

Sort lst1 and return 'None' object

lst2 = sorted(iterable, key=int)

Return the iterable object sorted as list.

The optional key= part converts the elements to int before sorting. Other functions can be used too.

srtlist = sorted(alist, key = lambda x: (x[4], x[0]))

Return alist sorted on the third and the first element of each list in alist (alist is a list of lists)

More on sorting.

random.shuffle(alist)

Randomize the list element order. The list itself is changed, nothing is returned.

print (','.join(map(str, alist)))

Concatenate the elements of alist to <element1>,<element2>,... and print

print ('\n'.join(map(str, alist)))

Print alist with each element on another line

list(map(int, alist))

Return a list with all elements in alist mapped to integer.

Put items matching a regular expression in newlist.

More on regular expressions in Python:Strings#Regular_Expressions_(regexp)

NOTE: filter applies a function (regexp matching in this case) on the object (list1). The object is modified. A filter object is returned. A filter object is not a list. However, when you iterate over it, it does return the matching elements.

import re
filterobject = filter(re.compile(<regular expression>).search,list1)    # This returns a filter object you can iterate over but not slice like filterobject[0]
newlist = list(filter(re.compile(<regular expression>).search,list1)))  # This returns a real list

Select from a list of lists.

newist = [i for i in lists if 'SERACHSTRING' in str(i)]

Check if some values are in a list (works for sets too)

if [ i for i in alist if i in [value1,value2]]:

sum(aListWithNumbers)

Total of all values in the list

set

Class of iterable, mutable objects. Objects added to sets are hashed. Therefor:

• Only immutable objects can be added to a set. So not dictionaries or lists
• Sets cannot hold duplicate objects (adding an object again does not change the set).
• Checking if a set holds an object is very fast.

A set is iterable but has no order, therfor:

• You can loop over a set like for a in set:
• You cannot take a slice from a set.

set1 = set()

Initialize an empty set

set1 = set([<values>])

set1 = {<val1>,<val2>}

Initialize a set with objects. Note the list-format of <values>.

set2 = set1.copy()

Make a copy of set1. set2 = set1 will make set2 point to the same set, changing set1 will change set2 (as it is the same object).

set1.add(2)

Add the '2' object to set1. You can add only 1 object at a time.

set1.discard(2)

Remove the '2' object from set1 (returns None object)

set1.update(itterable)

Update the set with the elements in itterable (list, set, )

unionset = set1.union(set2)

Combine the sets into a new set (e.g.to remove duplicates)

intersectset = set1.intersection(set2)

intersectset = set1 & set2

Put all elements that exist in both sets into a new set

diffset = set1 - set2

diffset = set1.difference(set2)

diffset will have all elements of set1 that are not in set2

Augmented assignment (update the left hand object instead of creating a new one) are:

set1 |= set2

Update set1 with set2

set1 -= set2

Removed set2 from set1

set1 &= set2

set1 will have the elements that exist in both sets (intersection)

set1 ^= set2

set1 will have the elements that exist in only 1 of the sets

Tuple

Class of iterable, immutable objects. It acts as an immutable list. Multiple return values from functions are tuples and e.g. results from database queries are by default returned as tuple.

tpl1 = ()

Initialize an empty tuple (pretty useless action)

Dictionary or dict

Class of iterable, mutable objects. Dictionary's can be compared to perl hashes. Check the Python:JSON page too.

dict1 = {}

Initialize an empty dictionary.

dict1 = { column1: value1, column2: value2 }

Initialize dictionary with data

The collections defaultdict module can be used for dictionairies with automatic key creation.

WARNING; In a dictionary created with defaultdict a key will be added when you try to read a non-existing lower key.

from collections import defaultdict
adict = defaultdict(lambda: defaultdict())
print(adict[key1][key2])  # This will create adict[key1]

adict = defaultdict(<type>)

Create a dictionary key of the provided type automatically when it is used (see WARNINIG above). Use this to avoid checking if a key already exists before you populate it. If you do not provide <type> you can put in anything.

bdict = dict(adict)

bdict will be an ordinary dict.

adict = defaultdict(lambda: defaultdict(lambda: defaultdict()))

Use lambda function to handle multilevel dictionaries

NOTE: For 2 levels, and 2nd level is a dictionary you can use defaultdict(dict) too.

if key in dict

Test if key exists in dict. if dict[key]: will throw a keyerror if it does not exist.

dict1.keys()

Iterable of keys in dict1

dict1.values()

Iterable of values

dict1.items()

iterable of all key-value pairs in dict1 as tuples

sum(adict.values())

Total of all values in a dict if all are numbers

dict2 = dict1.copy()

dict2 = dict(dict1)

Make a copy of dict1. dict2 = dict1 does not make a copy dict2 points to the same object as dict1

dict1.update(dict2)

Add dict2 to dict1. Duplicate keys are overwritten in dict1.

Overwriting happens in multilevel dictionary's too obviously, use something like this

for key in bdict:
    for key2 in bdict[key]:
        adict[key][key2] = bdict[key][key2]

dict1.pop(key,None)

Remove key from dict1, return dict1[key] if successful, None if key does not exist in dict1.

Code example recursively search a key:

def search_dict(data,skey=None):
    result = None
    if skey in data:
        result = data[skey]
    else:
        for key in data:
            if isinstance(data[key],dict):
                result = search_dict(data[key],skey)
    return result

Create a list of dict values where the key matches a string (List comprehension for dicts)

[value for key, value in d.items() if 'searchstring' in key]
[value for key, value in d.items() if re.search(pattern,key)]
[value for key, value in d.items() if key == 'keyname']
# List of keys that match
[key for key in d.keys() if key == 'keyname']

This very non-intuitive syntax is the same as:

list1 = list[]
for key,value in d.items():
    if 'searchstring' in key():
        list1.append(value)
return list1

sorted(dict1.items(), key=lambda x: x[1], reverse=False)

Return a list of tuples (key,value) sorted on value

sorted(adict.keys(), key=adict.get)

Return a list of keys sorted on values

intersectionkeys = dicta.keys() & dictb.keys()

intersectionkeys = set(dicta.keys()) & set(dictb.keys())

As dictionary keys are a set-like objects in python3 you can make intersctions like.

Second format is for python2

None

The None object is returned e.g. if nothing is found in a re.search. The None object is not an empty string

Range

Constructor of immutable sequences of integers. Use with list, set, tuple to create the desired object, or for loops.

range(start,stop,step)

Generic format. If you leave out step, step = 1. If only 1 parameter is provided, it is the stop number, start = 0, step = 1

for i in range (2,5,1):
    print(i)
2
3
4

alist = [*range(5)]

Unpack the range immediately (that is what * does). alist = [0, 1, 2, 3, 4]

Module itertools can be used to loop integers to infinity

import itertools

for i in itertools.count(step=15):
    if breakcondition:
        break
   print(i)

DateTime

from datetime import datetime
timestamp = datetime.now().strftime("%y%m%d-%H%M%S")

Other formats:

%s - Seconds since epoc (January 1, 1970 00:00:00)

%f - Nanoseconds or Milliseconds depends on what your system supports

All formats can be found at W3schools[1]

datetime.datetime(year, month, day, hour, minute, second).strftime('%s')

Date to epoc; hour, minute and second are optional.

Get an unique integer. Time is cheaper than datetime.

import time as t
# Use UTC-time 1 tick per second
nonce = int(t.mktime(t.gmtime()))
# This has greater precision (nanoseconds) but is in local time, not unique when the clock is set back.
nonce = int(t.time()*10000)
# I use datetime as that is loaded most time anyhow for other timestamps
from datetime import datetime
timestamp = datetime.utcnow().strftime("%s%f")

Convert datetime strings to a datetime object. The format indicates where the elements can be found (as in unix date formats ('%Y%m%d_%H%M'))

from datetime import datetime
dt = datetime.strptime(<datetimestring>, <format>)

For unixtime however you can not use strptime with %s

dt = datetime.fromtimestamp(<unixtime>)

To convert to a date object

from datetime import datetime
dateobj = datetime.strptime(<datetimestring>, <format>).date()

The parser is pretty smart, but it can be confused for day numbers under 13. It accepts e.g. 1-jan-1991 12:30 and YYYYMMDD-HHMM format.

from dateutil.parser import parse
datetimeobject = parse(datestring)

Calculations (7 days before now)

from datetime import date
from datetime import timedelta
newdate = date.today() - timedelta(days=7)

timddiff = datetimeobject - datetimeobject2
print(timediff.days)
print(timediff.seconds) # max 1 day
print(timediff.microseconds)  # max 1 second

Slicing

You can address all iterable datatypes partly or in a difference sequence.

object[b:e:s]

Generic format where b=Begin (counting starts at 0), e=End, s=Stepsize (negative stepsize starts counting at the end)

Examples:

'last element'[-1]
'All elements except the last'[:-1]
'All elements in reversed order'[::-1]

'All elements from the second'[1:]
'Second until 5th element (element 1,2,3 and 4)'[1:5]  
'Elements in reversed order (element 5,4,3 and 2)'[5:1:-1]
'Element 1,3 and 5'[1:6:2]