🪴\zansh.in\blog

6 min read

JavaScript Pipelining Techniques

Pipelining using 5 different techniques, current and future.

We'll refactor two chunks of code lifted from the TC39 Pipeline Operator Proposal:

const envarString = Object.keys(envars)
  .map(envar => `${envar}=${envars[envar]}`)
  .join(' ')
const consoleText = `$ ${envarString}`
const coloredConsoleText = chalk.dim(consoleText, 'node', args.join(' '))
console.log(coloredConsoleText)

const keys = Object.keys(values)
const uniqueKeys = Array.from(new Set(keys))
const items = uniqueKeys.map(item => <li>{item}</li>)
const unorderedList = <ul>{items}</ul>
return unorderedList

Each has a "chain" of operations used one after the other against the previous value.

The first chunk logs the final value, the second returns it:

• envars > envarString > consoleText > coloredConsoleText > log
• values > keys > uniqueKeys > items > unorderedList > return

In both cases, the final value is the only one we're really interested in, so this makes them candidates for pipelining!

Let's start with...

The simplest way to drop those temporary variables is to declare a single mutable let tmp and continuously reassign it:

let tmp = envars
tmp = Object.keys(tmp)
tmp = tmp.map(envar => `${envar}=${envars[envar]}`)
tmp = tmp.join(' ')
tmp = `$ ${tmp}`
tmp = chalk.dim(tmp, 'node', args.join(' '))
console.log(tmp)

It'll work, but maybe there are less error-prone ways of achieving the same thing. Also, mutable variables aren't exactly en vogue these days.

We can use Promise.resolve and a sequence of then's to keep the scope of each temporary variable under control:

Promise.resolve(envars)
  .then(_ => Object.keys(_))
  .then(_ => _.map(envar => `${envar}=${envars[envar]}`))
  .then(_ => _.join(' '))
  .then(_ => `$ ${_}`)
  .then(_ => chalk.dim(_, 'node', args.join(' ')))
  .then(_ => console.log(_))

No polluting the enclosing scope with tmp here! A Promise carries the idea of "piping" from envars all the way to logging the final colourised output without overwriting a temporary variable.

Not quite how we'd typically use Promise perhaps, but since many of us are familiar with how they chain together it's a useful jumping-off point for understanding pipelining for those not already familiar.

By the way, we could have used Object.keys and console.log first-class:

Promise.resolve(envars)
  .then(Object.keys) // instead of: _ => Object.keys(_)
  .then(console.log) // instead of: _ => console.log(_)

But I'll avoid using this "tacit" style here.

I'm also intentionally avoiding:

Promise.resolve(
  Object.keys(envars)
    .map(envar => `${envar}=${envars[envar]}`)
    .join(' ')
)
  .then(_ => `$ ${_}`)
  .then(_ => chalk.dim(_, 'node', args.join(' ')))
  .then(console.log)

Instead I'll try to keep the first level of indentation equal, as I think it helps convey the full pipelined-operation a bit better.

Anyway, using a Promise isn't ideal if we want a synchronous side-effect.

Popping an await before the whole chain is possible of course, but only if the pipeline sits inside an async function itself, which might not be what we want.

So let's try some synchronous pipelining techniques!

With this magic spell:

function pipe(x, ...fns) {
  return fns.reduce((g, f) => f(g), x)
}

...we can have:

pipe(
  envars,
  _ => Object.keys(_),
  _ => _.map(envar => `${envar}=${envars[envar]}`),
  _ => _.join(' '),
  _ => `$ ${_}`,
  _ => chalk.dim(_, 'node', args.join(' ')),
  _ => console.log(_)
)

We dropped all those .then()'s and left the lambdas (arrow-functions) behind as arguments to pipe that will run in sequence, with the first argument providing the starting value to the first lambda.

Handy!

If you're using Babel or living in a future where the TC39 pipeline proposal has landed, you can use Hack-pipes:

envars
  |> Object.keys(%)
  |> %.map(envar => `${envar}=${envars[envar]}`)
  |> %.join(' ')
  |> `$ ${%}`
  |> chalk.dim(%, 'node', args.join(' '))
  |> console.log(%)

Terse! And starting to look like an actual pipe on the left there, no?

Notice that a token % acts as our "previous value" variable when we use |>, just like when we used _ or tmp previously.

Let's cast another magic spell:

const Box = x => ({
  map: f => Box(f(x))
})

...and make a pipeline with it:

Box(envars)
  .map(_ => Object.keys(_))
  .map(_ => _.map(envar => `${envar}=${envars[envar]}`))
  .map(_ => _.join(' '))
  .map(_ => `$ ${_}`)
  .map(_ => chalk.dim(_, 'node', args.join(' ')))
  .map(_ => console.log(_))

Looks suspiciously like the Promise pipeline, except then is replaced with map, and this version is of course synchronous.

So that's 5 different pipelining techniques!

We'll apply them now in reverse-order for...

Here's the reference code again as a reminder:

const keys = Object.keys(values)
const uniqueKeys = Array.from(new Set(keys))
const items = uniqueKeys.map(item => <li>{item}</li>)
const unorderedList = <ul>{items}</ul>
return unorderedList

To start, let's first make Box a monad:

const Box = x => ({
  map: f => Box(f(x)),
  chain: f => f(x) // there we go
})

By adding chain we can return the JSX at the end of a pipeline without transforming it into yet another Box (which didn't really matter in the side-effect chunk since we weren't returning anything):

return Box(values)
  .map(_ => Object.keys(_))
  .map(_ => Array.from(new Set(_)))
  .map(_ => _.map(item => <li>{item}</li>))
  .chain(_ => <ul>{_}</ul>)

Kinda feels like the Promise.resolve pipeline if it had an await at the beginning, eh? Instead it's a Box with a chain at the end.

And synchronous too, like pipe()!

Speaking of which, let's go back and use it now:

return pipe(
  values,
  _ => Object.keys(_),
  _ => Array.from(new Set(_)),
  _ => _.map(item => <li>{item}</li>),
  _ => <ul>{_}</ul>
)

Fairly similar to the side-effect chunk, except to reveal that yes, pipe will indeed give us back the value returned by the last lambda in the chain. (That lovely <ul /> in this case.)

Alright, so if we were really going "backwards" this would appear before the above pipe example:

return values
  |> Object.keys(%)
  |> Array.from(new Set(%))
  |> %.map(item => <li>{item}</li>)
  |> <ul>{%}</ul>;

Hack-pipes are expressions, so the resultant value can be immediately returned or put into a variable.

Back in the land of async, does it make sense to return JSX from a Promise in this way? It's not a dynamic import(), after all.

I'll leave that up to you, but here it is anyway:

return await Promise.resolve(values)
  .then(_ => Object.keys(_))
  .then(_ => Array.from(new Set(_)))
  .then(_ => _.map(item => <li>{item}</li>))
  .then(_ => <ul>{_}</ul>)

(await thrown-in just to communicate intention, but it's not required.)

Lastly, let's bring it right back to let tmp:

let tmp = values
tmp = Object.keys(tmp)
tmp = Array.from(new Set(tmp))
tmp = tmp.map(item => <li>{item}</li>)
tmp = <ul>{tmp}</ul>
return tmp

And that's where we came in!

All in all we covered 5 different ways of pipelining: A way of transforming one value into another in a sequence of steps without worrying about what to call the bits in between.

• let tmp
• Promise#then
• pipe(startingValue, ...throughTheseFunctions)
• Hack |> pipes(%)
• Identity Functor/Monad (Box#map/chain)

A version of this article has been published on Dev.to.

4 Nov, 2021