Nov. 7, 2020

Enumerate and custom counters like skip and loop

Enumerate adds a counter variable to track iteration status. Custom linear or cyclic index variables can be created using generators.


Page contents

Add a counter

Iterating with enumerate is simple.

Python 3: Syntax

enumerate( iterable, start=0 )

Enumerate adds a counter to a for loop quite elegantly. It provides an index along with sequential values from an iterable list.

Python 3: Enumerate a list

periodic = ['H','He','Li','Be','B','C','N','O','F','Ne']

for atomic, element in enumerate(periodic, 1):
    print(atomic, element)
#= 1 H
#  2 He
#  3 Li
#  4 Be
#  5 B
#  6 C
#  7 N
#  8 O
#  9 F
#  10 Ne

C style looping

Creating an index variable and tracking element positions is old style.

Python 3: Conventional loops and counters

periodic = ['H','He','Li','Be','B','C','N','O','F','Ne']

for element in range(0, len(periodic)):
    #   atomic number starts from 1
    atomic = element + 1
    #   list index starts at zero
    name = periodic[element]
    print(atomic, name)
#= 1 H
#  2 He
#  3 Li
#  4 Be
#  5 B
#  6 C
#  7 N
#  8 O
#  9 F
#  10 Ne

Alternate mixed way of using a Python loop to get elements, and count independently using variable increment.

Python 3: Python loops and counters

periodic = ['H','He','Li','Be','B','C','N','O','F','Ne']

#   atomic is counter variable
atomic = 1
for element in periodic:
    print(atomic, element)
    atomic += 1
#= 1 H
#  2 He
#  3 Li
#  4 Be
#  5 B
#  6 C
#  7 N
#  8 O
#  9 F
#  10 Ne

Notes: Using counters properly

The use of enumerate is not a mandatory feature in Python, but it is elegant. One does not have to keep track of the counter within a loop. Each time a value is requested from the iterable, the enumerate method does the counting. Indexing and slicing values from an iterable, although possible, is less efficient than a for loop directly providing the next value. It's added burden and does slow down code. For super efficient coding where the loop is in high demand, enumerate is sure to provide efficiency and clean coding.

Book-keeping with lazy iteration

Creating an iterator and requesting values on demand.

Python 3: Book-keeping using iterators

periodic = ['H','He','Li','Be','B','C','N','O','F','Ne']

#   iterator object
elements = enumerate(periodic, 1)
print(elements)
#= <enumerate object at 0x00000213DDB8AF30>


#   request 2 elements
print('First row of periodic table')
print(next(elements))
print(next(elements))
#= First row of periodic table
#  (1, 'H')
#  (2, 'He')


#   request 8 elements
print('Second row of periodic table')
for row1 in range(0,8):
    print(next(elements))
#= Second row of periodic table
#  (3, 'Li')
#  (4, 'Be')
#  (5, 'B')
#  (6, 'C')
#  (7, 'N')
#  (8, 'O')
#  (9, 'F')
#  (10, 'Ne')

Notes: Iteration on demand

When an enumerate object is created, an iterator comes into existence. A for loop queries the iterator for values, one at a time. Each iteration, the enumerate object returns an object count, and a value from the list being iterated on.

It is perfectly fine to directly use the iterator as a lazy generator, which waits patiently consuming very little memory for a request to release the next value. Here the programmer does not have to keep track of the index variable, or the count. Each time a new value is required, the next method, like for, requests it from enumerate.

Iterating over dictionary

To iterate and return both key and value of a dictionary, the items method is used to create the enumerate iterator.

Py3: Iterating over a dictionary

#   dictionary of symbol:name
periodic = {'H':'Hydrogen', 'He':'Helium', 'Li':'Lithium'}

#   iterate object is created with both key and value
for item in enumerate(periodic.items(), 1):
    print(item)
#= (1, ('H', 'Hydrogen'))
#  (2, ('He', 'Helium'))
#  (3, ('Li', 'Lithium'))


#   separate variables
for count, (symbol, name) in enumerate(periodic.items(), 1):
    print(count, symbol, name)
#= 1 H Hydrogen
#  2 He Helium
#  3 Li Lithium

Notes: Three values from the iterator

Creating the enumerate iterator using items from a dictionary returns three items every cycle. The first is the counter value, next comes the key, and then the value from the dictionary. The key and value tuple can be expanded into separate variables along with the counter, or returned as a single nested tuple.

Custom skip counter

In Python, nothing is totally special. A custom iterator can be created which increments by two or any skip value instead of one.

Python 3: Custom iterator function

#   custom iterator with double increment
#   counter is second element
#-------------------------------------------------------
def count_skip( iterable, start=1, skip=1):
    counter = start
    for item in iterable:
        #   return item from iterable
        #   along with counter
        yield item, counter
        counter += skip
#-------------------------------------------------------

#   iterable list
oddlist = ['three','five','seven','nine']

#   iterator object
odds = count_skip(oddlist, 3, 2)

#   request single elements
print(next(odds))
print(next(odds))
#= ('three', 3)
#  ('five', 5)

#   continue requesting using for loop
for odd in odds:
    print(odd)
#= ('seven', 7)
#  ('nine', 9)

Notes: Custom skip counter

Enumerate can be used to create a generator that returns single items from an iterable along with a counter that increments every cycle. If a custom generator is needed to increment by a different value, we can always scale the enumerate counter value, or create a custom generator. The custom generator works the same way as enumerate, showing the magic of how the internals of the object function.

The custom generator shown here has configurable start value and skip value.

Cyclic counter

Instead of a monotonic counter, a cyclic custom counter can be very handy. These can be used for row formatting in a table to create a pattern.

Python 3: Cyclic iterator function

#   cyclic counter function
#-------------------------------------------------------
def count_cycle(itrble, start, stop, step=1):
    #   configure cyclic reset check function
    #   using lambda for count ascending or
    #   descending
    if (stop>start) & (step>0):
        #   stop > start, step is positive
        reset_chk = lambda x: x >= stop
    elif (stop<start) & (step<0):
        #   stop < start, step is negative
        reset_chk = lambda x: x <= stop
    else:
        #   zero steps, or if start and stop are crossed
        raise ValueError('Parameters not compatible')

    #   start yielding values from itrble
    #   keep a cyclic counter going
    n = start
    for item in itrble:
        yield n, item
        n += step
        #   check cyclic variable
        #   reset_chk is a function
        if reset_chk(n):
            n = start
#-------------------------------------------------------

#   configure to iterate over letters from 'cyclic'
#   counter epoch is 0, and reset at third letter
cyc = count_cycle('cyclic', 0, 3)

#   show the type of object created
print(type(cyc))
#= <class 'generator'>

#   perform iteration
for item in cyc:
    print(item, end=' ')
#= (0,'c') (1,'y') (2,'c') (0,'l') (1,'i') (2,'c')

Here the cyclic iterator can count in any direction with appropriate parameters.

Python 3: Application of cyclic iterator

result = [(e,v) for e, v in count_cycle('cycle', 0, -2, -1)]
#= [(0,'c'), (-1,'y'), (0,'c'), (-1,'l'), (0,'e')]

Notes: Cyclic generators

We can create our own rules of counting, and create a cyclic counter. These can be very useful to keep track of how many items are being sent for processing. An example can be a packet generator which requires 3 letters. The beauty of creating a generator is that it keeps track of the accounting, while consuming very little memory.

Custom repeater

Instead of calculating labels, a list of custom labels can be provided to be repeated for each loop value.

Python 3: Label repeater with loop count

#   cyclic label repeater
#   iterable is provided along with a list
#   of labels
#-------------------------------------------------------
def count_repeat(itrble, labels):
    nloop = [1]
    def repeater():
        while True:
            for lv in labels:
                yield lv
            nloop[0] += 1
    rep = repeater()

    for item in itrble:
        yield item, next(rep), nloop[0]
#-------------------------------------------------------

#  custom labels are provided as a list
for item, r, n in count_repeat('repeat', ['start',0,0]):
    #   show item from iterable
    #   along with label (r) and loop count (n)
    print(item, r, n)

#=  r start 1
#   e 0 1
#   p 0 1
#   e start 2
#   a 0 2
#   t 0 2

Notes: Repeating labels

The implementation of repeating labels uses a nested generator. Instead of having to calculate counter values, it may be useful to provide a list of values to pick the repeating values. The loop is controlled by the main iterable until it is consumed. The label generator is configured as an infinite loop, but stops when the main iterable is exhausted.

Moving average calculators

For a list of numbers, a counter can be converted into a calculator to generate rolling statistics on the numbers being iterated on. A simple moving average, and an exponential moving average calculator are shown here.

Python 3: SMA and EMA

#   calculate simple and exponential moving
#   averages for a list of numbers
#   rolling controls how many sequence items
#   are averaged
#-------------------------------------------------------
def calc_averages(numbers, rolling):
    #   establish fifo pipe
    pipe = [0]*rolling
    #   exponential moving average config    
    ema = 0
    k = 2/(1+rolling)

    # loop over elements
    for e, item in enumerate(numbers, 1):
        #   remove first element
        #   add new to right end
        pipe.pop(0)
        pipe.append(item)

        #   calculate average
        sma = sum(pipe)/rolling

        #   exponential moving average
        ema = item * k + ema * (1-k)

        #   return averages if valid
        if e >= rolling:
            yield item, sma, ema
        else:
            ema = sma
            yield item, None, None
#-------------------------------------------------------

#   moving averages for odd numbers from 1 to 11
#   in rolling groups of 3
for item, sma, ema in calc_averages(range(1,12,2), 3):
    print(item, sma, ema)

#=  1  None  None
#   3  None  None
#   5  3.0  3.1666666666666665
#   7  5.0  5.083333333333333
#   9  7.0  7.041666666666666
#   11  9.0  9.020833333333332

Notes: Moving statistics

Instead of simple counters, the generators can be used to send updated statistics on numeric sequences.

Simple moving average is calculated using a first-in-first-out fifo pipeline, and is valid for items greater or equal to the length of rolling average. An exponential moving average may take more time to settle down, but is generally initialized with SMA.

This implementation does not return SMA or EMA until they are valid.

Summary of counters

Built-in and customized generators and counters discussed.

Method/Statement Property
enumerate built-in serial counter
count_skip() custom increment counter
count_cycle() cyclic counter
count_repeat() cyclic label element repeater
calc_averages() moving average calculator

References

Popular content

python programming

Read and write lists with Pickle, Array and Pandas

python programming

Flatten nested list or generate blocks of nested lists

python programming

For loop and control statements

python programming

Clear list using inplace and standard methods

python programming

List comprehension with nested conditions

python programming

Concatenate list elements using add, append, extend

python programming

Enumerate and custom counters like skip and loop

python programming

Count number of elements, and memory allocated

python programming

Remove duplicate list elements

python programming

Statistics with numeric lists of integers, fractions, and decimals.

New content

python programming

Read and write lists with Pickle, Array and Pandas

python programming

Is element in list?

python programming

Dictionary merge common key groups

python programming

Packaging loops with zip

python programming

Concatenate list elements using add, append, extend

python programming

List comprehension with nested conditions

python programming

Flatten nested list or generate blocks of nested lists

python programming

Enumerate and custom counters like skip and loop

python programming

Range integer sequences

python programming

For loop and control statements