Python filter is very handy python built-in function to filter iterables (e.g. lists) in python. But it returns a list which may consume huge memory when data behind iterable is huge. In such case using itertools.ifilter is a better choice. itertools.ifilter takes a lazy approach and calls the filter function only when needed. Here are quick examples for both approaches.
Here filter function is called for all values the moment filter is called.
a = [1,101, 2,3] def is_lt_100 (x): print "is_lt_100 called" return x < 100 b = filter(is_lt_100, a) for x in b: print "x=%s" % (x)
is_lt_100 called is_lt_100 called is_lt_100 called is_lt_100 called x=1 x=2 x=3
Here ifilter function is called only when we use the iterator returned by ifilter.
import itertools a = [1,2,101,3] def is_lt_100 (x): print "is_lt_100 called" return x < 100 b = itertools.ifilter(is_lt_100, a) for x in b: print "x=%s" % (x)
is_lt_100 called x=1 is_lt_100 called x=2 is_lt_100 called is_lt_100 called x=3