Concepts - Iterators

What is an iterator?

Very simply, an iterator is a function that "iterates" over a certain thing using a for block until it cannot anymore! An example of an iterator that you'll learn quickly is the pairs iterator function.

Example

local some_table = { 1, 2, nil, 4, 5 } for k, v in pairs(some_table) do print(v) end --- --- Outputs: --- 1 --- 2 --- 4 --- 5

How do I write my own?

Iterators work through a little bit of for magic and a function that returns another function. They are divided into two steps.

Lets say for this example, you want to iterate over every character in a string

Step 1 - Initialization

First, we define our generator function that will act as our iterator, like pairs!

function strchars(str) --- This is our "wrapper" or "generator" function, what generates new iterations --- Our work goes here end

Next, we have two ways of going about things, we can either handle our own "state" or allow Lua to handle the iteration state for us.

State here is relating to the data we store to remember where the iterator currently is

Step 2.a - Handling our own state

If we want to handle our own state, we define it in our wrapper function locally as such.
Handling our own state comes with some benefits and some cons, one of the biggest benefits is our state doesn't have to be the first value returned by our generator.

function strchars(str) --- First, we declare all our state local index = 1 --- String indices start at [1], so we start here --- Next, we return a new function that will get us our data --- Return values from this are passed to the arguments in our for loop --- If we return nil, the loop will be stopped immediately return function() index = index + 1 if str[index] == "" then -- Since indexing a string never returns nil, this is checking if we've reached the end return end --- Notice that here we're returning the value first and index second! --- This is reversed from how pairs() and ipairs() does it --- We can only do this with managing our own state return str[index], index end end for char, i in strchars("abc") do print(char, i) end --- --- Outputs: --- a 1 --- b 2 --- c 3

Step 2.b - Lua handling our state

If we want to let Lua handle our state for us, we need to remember that the first value returned from our generator must be our state, otherwise it will be discarded.

function strchars(str) --- Here, we declare nothing in the wrapper and just generate a new function --- Lua assumes that the first returned value from our generator function is our state value --- so it has features built in to make this easier return function(_, index) --- The first argument passed here is elaborated on in "Esoteric Information", it is our invariant state index = index + 1 if str[index] == "" then return end --- Notice that here we're returning the value first and index second! --- This is reversed from how pairs() and ipairs() does it --- We can only do this with managing our own state return index, str[index] end end for i, char in strchars("abc") do print(i, char) end --- --- Outputs: --- 1 a --- 2 b --- 3 c

Esoteric Information

There are extra values that are able to be passed after the iterator function call in for in loops.

"Invariant State"

Following the iterator function in a for in loop, you can provide something referred to as the "invariant state". This is essentially the constant (as in "invariant") data to be passed as the first argument to the function the iterator generator returns.
This is useful for a couple reasons, primarily, we can skip creating a generator entirely if we have a well defined function.

--- Since this function is built for being called directly and not from a generator --- We accept two arguments, some data, and some index --- This function is practically identical to the generated code in Step 2.b, except that we let Lua handle our data as well as our state index function strchars(str, index) index = index or 0 --- Index starts nil, so we default it to 0 index = index + 1 --- Next, we add one to it if str[index] == "" then return end --- Cancel once were done return index, str[index] end --- Instead of calling a function for the first value, we directly use our unwrapped generator function --- We also provide a second expression after strchars, this is our invariant state for i, ch in strchars, "abc" do print(i, ch) end --- --- Outputs: --- 1 a --- 2 b --- 3 c

"Control Variable"

After our invariant state, we can provide a "control variable", this is our initial index variable!

--- Note that were using strchars from "Invariant State" --- Like above, we call our unwrapped generator function with some invariant state of a string --- However, we also pass another, final parameter of 1 --- This is passed into strchars as our initial index upon the first call, essentially where we start for i, ch in strchars, "abc", 1 do print(i, ch) end --- --- Outputs: --- 2 b --- 3 c