Generators and Channels in JavaScript
Covering the Basics
Introduction
The following should be an introduction into generators and channels. If you know about promises, generators, coroutines and channels jump to the Using Generators and Channels with React section. While the examples might not be suited for the real world, it should be seen as a starting point, to experiment with the possibilities that might arise by taking this approach.
Just think about the following listen function for a second.
const listen = (el, type) => {
const ch = chan()
el.addEventListener(type, e => putAsync(ch, e))
return ch
}It will turn any event on a DOM element into a channel. Let’s begin with figuring out the basics.
Why Generators and Channels?
Before we jump into generators, coroutines and channels, let’s take a look at regular promises.
function getUsers() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve({
users: [{id: 1, name: 'test'}]
})
}, 1000)
})
}getUsers returns a promise and when successful, resolves the promise with needed data once we’re finished. This also works nicely when dealing with more than a single promise. Either one promise relies on another promise or an operation needs all promises to have been resolved to start. Both situations are covered with the standard promise implementations. Think chaining promises with then for the first situation and Promise.all for the latter.
Promises can be seen as a cleaner alternative to the raw callback approach, also known as callback hell. Just in case you can’t imagine or can’t remember what that actually means, here’s a reminder sans the actual implementation code.
asyncCallOne(() => {
asyncCallTwo(() => {
asyncCallThree(() => {
asyncCallFour(() => {
asyncCallFive(() => {
// do something here...
})
})
})
})
})Nesting callbacks seems fine when we’re dealing with non-complex code, but doesn’t scale well as seen in the example above. It’s better to avoid nesting callbacks from the get-go. Promises fit in well here, they enable us to avoid nesting and simplify exception handling.
asyncCallOne(() => { // do some something... } )
.then(asyncCallTwo)
.then(asyncCallThree)
.then(asyncCallFour)
.then(asyncCallFive)
.catch(() => {
// handle any errors that happened a long the way
})Resolving all asynchronous functions then continuing on is no problem either.
Promise.all([
asyncCallOne,
asyncCallTwo,
asyncCallThree
]).then(values => {
// do something with the values...
});Now that we revisited the basics 0n callbacks and promises, let’s take a look at generators, which were introduced in ES6.
Generators
Before jumping into the what, why and how let’s take a look at a short definition of what generators actually are.
“Generators are functions that can be paused and resumed, which enables a variety of applications.”
To quickly summarize Generator functions, they enable us to produce a sequence of values by using yield and retrieving values by calling next on the iterator object.
The most basic example to demonstrate a generator function is the following piece of code.
function* getNumbers() {
yield 1
yield 5
yield 10
}// retrieving
const getThoseNumbers = getNumbers()
console.log(getThoseNumbers.next()) // {value:1, done:false}
console.log(getThoseNumbers.next()) // {value:5, done:false}
console.log(getThoseNumbers.next()) // {value:10, done:false}
console.log(getThoseNumbers.next()) // {value:undefined, done:true}
We can iterate over generator function, use them to observe data and are suited for lazy evaluation and control flow.
Here are a couple of examples on how to retrieve from a generator function, with the last example even showing how to access the data using reduce.
// iterate
for (let i of getNumbers()) {
console.log(i) // 1 5 10
}// destructering
let [a, b, c] = getNumbers()
console.log( a, b, c) // 1 5 10// spread operator
let spreaded = [...getNumbers()]
console.log(spreaded) // [1, 5, 10]// even works with reduce
// Ramda reduce for example
const reducing = reduce((xs, x) => [...xs, x], [], getNumbers())
console.log(reducing) // [1, 5, 10]
Furthermore Generators enable us to send data via next, with the important exception that the first next will only start the iteration. This feature only works starting via the second next. The example below should illustrate this interesting fact.
function* setGetNumbers() {
const input = yield
yield input
}const setThoseNumbers = setGetNumbers()
console.log(setThoseNumbers.next(1)) //{value:undefined, done:false}
console.log(setThoseNumbers.next(2)) //{value: 2, done: false}
console.log(setThoseNumbers.next()) //{value: undefined, done: true}
As you can see from the output above, the first next is ignored and only the second next is taken into account.
Terminating the generator is as simple as defining a return inside the generator function.
Before we move on, here’s one more nice feature, that we will build on in the following sections. Generator functions can call other generator functions. Take this example into account f.e.
function* callee() {
yield 1
}function* caller() {
while (true) {
yield* callee();
}
}const callerCallee = caller()
console.log(callerCallee.next()) // {value: 1, done: false}
console.log(callerCallee.next()) // {value: 1, done: false}
console.log(callerCallee.next()) // {value: 1, done: false}
console.log(callerCallee.next()) // {value: 1, done: false}
We should have an understanding of generators by now.
For a more detailed writeup and walkthrough on generator functions in ES6 read the comprehensive ES6 Generators in depth article by Axel Rauschmayer.
Generators, Promises and Coroutines
Now that we have Promises and Generators covered, let’s see how the two combine.
function fetchUsers() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve({
users: [{id: 1, title: 'test'}]
})
}, 1000)
})
}function* getData() {
const data = yield fetchUsers()
yield data
}
Obviously we will need some mechanism to make sure that we don’t have to manually run the loop. This is where coroutines come into play. They enable us to write generator functions that can handle async actions including promises, thunks and other operations. co is a library that handles this exact case.
Although the following code is only a very naive implementation of the real co code, it should shed some light into what happens when we run a co function.
function co(fn) {
const obj = fn()
return new Promise((resolve, reject) => {
const run = result => {
const { value, done } = obj.next(result) // check if done and return if finished
if (done) return resolve(result) // retrieve the promise and call next with the result value
.then(res => run(res))
.catch(err => obj.throw(err))
}
// start
run()
})
}
Here’s the previous example using the simplified co function.
function fetchUsers() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve({
users: [{id: 1, name: 'test'}]
})
}, 1000)
})
}
function fetchOtherData() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve({
other: [{id: 2, title: 'other data'}]
})
}, 1000)
})
}
const get = co(function* getData() {
const getAll = yield Promise.all([fetchOtherData(), fetchUsers()])
// do something else...
return getAll
}).then(data => console.log(data))We can see that coroutines enables us to write code that looks synchronous by yielding one line after the other, as demonstrated by the following.
const get = co(function* getData() {
const otherData = yield fetchOtherData()
console.log('fetched other data: ', otherData)
const users = yield fetchUsers(otherData)
console.log('fetched users: ', users)
return users
}).then(data => console.log(data))So we’ve seen a way to combine generators and promises by using coroutines. This is all fine, but we can do even better. Let’s see the possibilities, when we start combining generator functions with channels.
Generators and Channels
Don’t combine Generators with Promises, combine them with Channels!
David Nolen
(http://swannodette.github.io/2013/08/24/es6-generators-and-csp)
While all the previous approaches on handling asynchronous code are well-known, it’s interesting to note that channels seem to be an afterthought in JavaScript. Clojure with core.async and also Go, with goroutines have championed channels for quite some time now.
There are a couple of great posts on the topic in JavaScript though, most prominently Taming the Asynchronous Beast with CSP Channels in JavaScript by James Longster. Please read the aforementioned article for a deeper understanding of channels. It does a better job in delivering insights on the topic than this post could ever accomplish to do.
I will just quote directly from the previously mentioned:
Typically channels are useful for coordinating truly concurrent tasks that might run at the same time on separate threads. They are actually just as useful in a single-threaded environment because they solve a more general problem of coordinating anything asynchronous, which is everything in JavaScript.
James Longster
When you read the word channels, you also usually see CSP come up, meaning Communicating Sequential Processes. I recommend reading Communicating Sequential Processes by David Nolen for a better understanding on the topic.
We will be using the js-csp library for demonstrating how we can leverage the benefits that come with using channels in JavaScript.
Processes communicate via channels. Typical channels offer a set of functions, but we will only need to know about put and take for now. While we can push to the queue via put, there is a process on the other side waiting via take. We will see this in more clear detail very soon. All we need to think about to begin with, is that we have a channel and a consumer. Take the following simplified example from the js-csp documentation f.e.
const ch = csp.chan(1);
yield csp.put(ch, 42);
yield csp.take(ch); // 42
ch.close()
yield csp.take(ch); // csp.CLOSEDWe’re creating a new Channel with a buffer size of 1, then we’re calling put, prefixed by yield, and passing the channel and the value 42. Then we take the value from the channel and finally close the channel . The next yield will have no affect as the channel is already closed.
Here’s another example lifted straight from the js-csp documentation.
var ch = go(function*(x) {
yield timeout(1000);
return x;
}, [42]);
console.log((yield take(ch)));By calling the go function we spawn a goroutine, which will immediately return a channel, enabling us to retrieve any values from the channel via take.
To understand how this all fits with the UI, take a look at the following listen function.
const listen = (el, type) => {
const ch = chan()
el.addEventListener(type, e => putAsync(ch, e))
return ch
}We can convert an element into a channel by using listen. This opens up the possibility to listen to changes via take on the channel. Now whenever we type something into the input box, we can retrieve any changes via the channel and update the display element.
go(function*() {
const input = document.getElementById('title')
const display = document.getElementById('display')
const ch = listen(input, 'keyup')
while(true) {
const e = yield take(ch)
display.innerHTML = `From Input: ${e.target.value}`
}
})Using Generators and Channels with React
Now that we have covered the basics and have a high-level idea of why channels and generators make sense in JavaScript, let’s put our acquired knowledge on channels and generators to work next.
First off, let’s begin with a classic Counter example. Although very basic, we’re only incrementing/decrementing the counter and displaying the current number on the screen, it will help us to gain a clearer insight into how we can render React components with this approach.
Below you can find the complete code sans the Counter component, which has been taken from fantastic React/Elm-Architecture tutorials by Stefan Oestreicher. The counter code can be found here.
AppStart should handle the initial rendering of the top-level React component and start a goroutine, which awaits any new updates on the AppChannel. The AppChannel itself is a regular channel without any buffering or other specialities. So all we do is put on the AppChannel as soon as an event has triggered an action. Nothing too exciting, but it works.
// basic example demonstrating the power of channels and generators
import React from 'react'
import { render } from 'react-dom'
import { chan, go, take, put, putAsync } from 'js-csp'
import { curry } from 'ramda'
import Counter from './Counter'// helper
const createRender = curry((node, app) => render(app, node))// create one channel for now
const AppChannel = chan()
const doRender = createRender(document.getElementById('mountNode'))// let start
const AppStart = ({ init, update, view }) => {
let model = 0
const signal = action => () => {
model = update(action, model)
putAsync(AppChannel, model)
}
// initial render...
putAsync(AppChannel, init(model))
go(function* () {
while(true) {
doRender(view(signal, yield take(AppChannel)))
}
})
}
// start
AppStart(Counter)
You can also find the code here.
Now that we have a basic example up and running, it’s time to do more complex things, like a fetch operation for example. We can build a simple list that fetches data from an external source and re-renders the results as soon as a state change has occurred. For this task, we will also want to present a loading information to the user, for a better user experience.
Let’s take a second to think about the requirements. We will probably need to separate the actions and channels for one, and also be required to handle the side-effects in a clean and well-structured manner, isolated from the regular synchronous running code.
Building the App…
Effectively we want to be able to define a function that handles the fetching and notifies when the loading has started and finished.
const getItems = () => {
go(function* () {
yield put(isLoading, true)
const fetchedItems = yield* fetchItems()
yield put(items, fetchedItems)
yield put(isLoading, false)
})
}Let’s start off we the writing a function that takes care of creating channels
const createChannel = (action, store) => {
const ch = chan()
go(function* () {
while(true) {
const value = yield take(ch)
yield put(AppChannel, action(store.get(), value));
}
})
return ch
}
// helper function for passing an object and getting channels
const createChannels = (actions, store) =>
mapObjIndexed(fn => createChannel(fn, store), actions)Now that we have createChannels in place, we might also write a couple of actions.
const Actions = {
isLoading: (model, isLoading) =>
assoc('isLoading', isLoading, model), items: (model, items) => assoc('items', items, model), addItem: (model, title) =>
assoc('items',
[ ...prop('items', model),
{title, id: getNextId(prop('items', model))}
],
model),
}
Let’s write our App component. Nothing too special here. Only a list of items and two buttons as well as an input field, one for fetching the list and the other for adding the text from the input field.
const App = ({ items, isLoading }) => {
if (isLoading) return (<p>loading...</p>)
return (
<div>
<h2>Random Items List</h2>
<ul>
{items.map(item => (
<li key={item.id} >{item.title}</li>
))}
</ul>
<input type='text' id='add' />
<button onClick={() => putAsync(addItem, findText())}>
Add Item
</button>
<button onClick={() => getItems()}>LoadItems</button>
</div>
)
}We have almost everything in place now. Our AppStart function is similar to the one we had previously created, except that it expects a component and a type of store now. The store is just a trivial object with getter and setter.
const AppStart = (Component, store) => {
// initial render...
putAsync(AppChannel, store.get())
go(function* () {
while(true) {
store.set(yield take(AppChannel))
doRender(<Component {...store.get() } />)
}
})
}Now all we need to do is create our channels for the previously defined Actions.
const { isLoading, items, addItem } = createChannels(Actions, store)We get an isLoading, an items and an addItem channel in return, with which we can update the state via channels now. Also important to note is that the AppChannel is being called with a sliding buffer to only process the latest value.
// create App channel... and render function
const AppChannel = chan(buffers.sliding(1))
const doRender = createRender(document.getElementById('mountNode'))All we need to do now is call AppStart now.
AppStart(App, store)Here is the full code if you’re interested.
This was just a quick experiment on how to combine React and Channels and needs more time and thinking to see what benefits might come from this approach.
Thinking about the listen function in the previous section and how we can turn an element into a channel — opens up a couple of ideas, including reacting to the window and changing the structure, styling or layout of your app.
The provided examples can be seen as a starting point for experimenting with the possibilities.
Outro
This should have been an introduction into generators and channels. We’re still missing some important aspects like transducers. A follow-up post covering channels and transducers including more advanced examples using React might make sense. Let me know if there is interest in more examples.
In case you have any questions or feedback leave a comment here or leave feedback via twitter.
Update:
Introduction into Channels and Transducers in JavaScript, a follow up post, is now online.
Links
ES6 Generators Deliver Go Style Concurrency
Taming the Asynchronous Beast with CSP in JavaScript
Why coroutines won’t work on the web
No promises: asynchronous JavaScript with only generators
CSP and transducers in JavaScript
