Python filter vs ifilter

Python filter is very handy python built-in function to filter iterables (e.g. lists) in python. But it returns a list which may consume huge memory when data behind iterable is huge. In such case using itertools.ifilter is a better choice. itertools.ifilter takes a lazy approach and calls the filter function only when needed. Here are quick examples for both approaches.
python-filter-vs-ifilter

filter example

Here filter function is called for all values the moment filter is called.

a = [1,101, 2,3]
def is_lt_100 (x):
  print "is_lt_100 called"
  return x < 100
b = filter(is_lt_100, a)
for x in b:
  print "x=%s" % (x)
is_lt_100 called
is_lt_100 called
is_lt_100 called
is_lt_100 called
x=1
x=2
x=3
Env: Python 2.7.6

itertools.filter example

Here ifilter function is called only when we use the iterator returned by ifilter.

import itertools
a = [1,2,101,3]
def is_lt_100 (x):
  print "is_lt_100 called"
  return x < 100
b = itertools.ifilter(is_lt_100, a)
for x in b:
  print "x=%s" % (x)
is_lt_100 called
x=1
is_lt_100 called
x=2
is_lt_100 called
is_lt_100 called
x=3
Env: Python 2.7.6

Share this article: share on Google+ share on facebook share on linkedin tweet this submit to reddit

Comments

Click here to write/view comments