run multiple tasks in parallel

[dagda1]

Is there anyway to run multiple tasks in parallel like is possible with concurrency in node?

[bartlomieju]

You can create a "wrapper" task that will spawn other tasks in parallel, like so:

{
  "tasks": {
    "foo": "echo 1",
    "bar": "echo 2",
    "parallel": "deno task foo & deno task bar"
  }
}

deno task parallel

[dagda1]

@bartlomieju thank you. Would the same thing work in node or is there something more going on here?

[bartlomieju]

AFAIK Node uses system shell, so it would work on Linux and Mac, but not on Windows

[konsumer]

A common thing I do with node is add npm-run-all to dev-deps, so I can do this:

{
  "scripts": {
    "start:frontend": "node frontend.js",
    "start:backend": "node backend.js",
    "start": "run-p start:frontend start:backend"
  }
}

If you Ctrl-C, it kills both of them, and run-p is cross-plaform, so on node it works fine everywhere.

There is a bit about a single & for async here but in normal shells, that would background the first process and run the next, which is not quite the same (the first process would be straggling in the background). It's not really the native unix-shell, though, it's a cross-platform subset of sh that deno runs (so it works same on windows), so it works as-is on all platforms with &.

I was looking into this to combine a webview wrapper with a an ultra backend (including live-reloading) and it seems to work great:

{
  "tasks": {
    "dev:frontend": "deno run -A --no-check --watch ./server.jsx",
    "dev:wrapper": "deno run -Ar --unstable deckagog.js",
    "dev": "deno task dev:frontend & deno task dev:wrapper"
  }
}

Ctrl-C kills them both, closing the web window leaves the server running (which can be Ctrl-C'd.)

I love that this is built-in!

[johndeighan]

So, it sounds like the original suggestion results in 2 processes, only one of which is killed with Ctrl-C. That makes it less than useful. I've used npm-run-all in NodeJS and would like to see something similar in Deno, though I suppose I could use the npm version of npm-run-all in Deno (but I'm on Windows, so maybe that would not work?). Does anyone know of a library in jsr that does this?

[johndeighan]

I just found the following on this page: https://docs.deno.com/runtime/reference/cli/task_runner/

Unlike in most shells, the first async command to fail will cause all the other commands to fail immediately. In the example above, this would mean that if the client command fails then the server command will also fail and exit. You can opt out of this behavior by adding || true to the end of a command, which will force a 0 exit code.

So, if you do this:

task 1 & task 2 & task 3 & task 4

Tasks 1, 2 and 3 will run in the background, task 4 will run in the foreground. The text above implies that if any of tasks 1, 2 or 3 fail, the other 2 will immediately fail. That's great, but I think it does NOT mean that if task 4 fails (e.g. by entering Ctrl-C), that the first 3 tasks will also fail.

Still, maybe it means that if the original example at the top of this page ran "deno task foo & deno task bar &", it would achieve the desired result, i.e. run 2 tasks in parallel in such a way that if one failed, the other would also terminate (note the trailing &)

[konsumer]

I don't think think it works like that, but maybe I am misunderstanding what you are saying. Here is an example. If you Ctrl-C before the 5s timeout, both sub-tasks exit.

Task p deno task a & deno task b
Task b deno b.js
Task a deno a.js
b: Hi!
a: Hi!
^C

If you let them both run:

Task p deno task a & deno task b
Task b deno b.js
Task a deno a.js
a: Hi!
b: Hi!
before goodbye
before goodbye
a: Bye!
b: Bye!

Here, a will run, and b, and both exit on Ctrl-C.

If I run the same thing in a normal shell (I am using zsh here) it backgrounds a, then starts b, and Ctrl-C only effects b:

deno task a & deno task b
[1] 240231
Task b deno b.js
Task a deno a.js
a: Hi!
b: Hi!
^C
❯  before goodbye
a: Bye!

[1]  + 240231 done       deno task a

[EarthlingDavey]

I'm trying to run 2 deno servers with one command.

deno task -r dev was only starting one of the tasks.

---

Looking at how task dependencies work:

{
  //   a
  //  / \
  // b   c
  //  \ /
  //   d
  "tasks": {
    "a": {
      "command": "deno run a.js",
      "dependencies": ["b", "c"]
    },
    "b": {
      "command": "deno run b.js",
      "dependencies": ["d"]
    },
    "c": {
      "command": "deno run c.js",
      "dependencies": ["d"]
    },
    "d": "deno run d.js"
  }
}

I came up with this workaround:

{
  "workspace": ["./service", "./server"],
  "tasks": {
    "dev": {
      "command": "echo 'No dev task defined'",
      "dependencies": ["server:dev", "service:dev"]
    },
    "server:dev": "cd server && deno task dev",
    "service:dev": "cd service && deno task dev"
  }
}

it ain't pretty, but it does run both tasks in parallel, and shut them both down with ctrl + c.

[konsumer]

This seems like another cool way to do it, but did you try with "dev": "deno task server:dev & deno task service:dev"? It should do same (single ampersand)

[EarthlingDavey]

Thanks @konsumer ... it was a bad assumption and skim reading on my part, I guessed that the & way would not log both processes to stout ☺️

Having re-read your comment, & is of course the cleaner way to do this 💯

[konsumer]

I think both are cool. It's nice to have options.