Python filter vs ifilter

Python filter is very handy python built-in function to filter iterables (e.g. lists) in python. But it returns a list which may consume huge memory when data behind iterable is huge. In such case using itertools.ifilter is a better choice. itertools.ifilter takes a lazy approach and calls the filter function only when needed. Here are quick examples for both approaches.

filter example

Here filter function is called for all values the moment filter is called.

a = [1,101, 2,3]
def is_lt_100 (x):
  print "is_lt_100 called"
  return x < 100
b = filter(is_lt_100, a)
for x in b:
  print "x=%s" % (x)

is_lt_100 called
is_lt_100 called
is_lt_100 called
is_lt_100 called
x=1
x=2
x=3

itertools.filter example

Here ifilter function is called only when we use the iterator returned by ifilter.

import itertools
a = [1,2,101,3]
def is_lt_100 (x):
  print "is_lt_100 called"
  return x < 100
b = itertools.ifilter(is_lt_100, a)
for x in b:
  print "x=%s" % (x)

is_lt_100 called
x=1
is_lt_100 called
x=2
is_lt_100 called
is_lt_100 called
x=3