dask_intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Dask"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Attribution**: The following notebook is based on the great notebook created by Ryan Abernathey in the Earth and Environmental Data Science book ([link](https://earth-env-data-science.github.io/lectures/dask/dask_arrays.html))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is Dask?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dask is a Python library that provides flexible parallel computing and distributed data processing capabilities. It is designed to handle larger-than-memory computations and parallelize tasks across multiple cores and even distributed clusters. Dask's primary goal is to make it easier for developers to scale their data analysis and processing workflows while maintaining a familiar and Pythonic interface.\n",
    "\n",
    "Dask is composed of two components: 1) Task Scheduler, and 2) “Big Data” collections. These collections include parallel arrays, dataframes, and lists that extend common interfaces like NumPy, Pandas, or Python iterators to larger-than-memory or distributed environments. These parallel collections run on top of dynamic task schedulers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Dask Overview](https://docs.dask.org/en/stable/_images/dask-overview.svg)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dask overview (source: [Dask Documentation](https://docs.dask.org/en/stable/))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Dask Arrays\n",
    "\n",
    "A dask array looks and feels a lot like a NumPy array.\n",
    "However, a dask array doesn't directly hold any data.\n",
    "Instead, it symbolically represents the computations needed to generate the data.\n",
    "Nothing is actually computed until the actual numerical values are needed.\n",
    "This mode of operation is called \"lazy\"; it allows one to build up complex, large calculations symbolically before turning them over to the scheduler for execution.\n",
    "\n",
    "Dask arrays coordinate many NumPy arrays arranged into a grid. These arrays may live on disk or on other machines."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Dask Array](https://docs.dask.org/en/stable/_images/dask-array.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(source [Dask Documentation](https://docs.dask.org/en/stable/array.html))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we want to create a NumPy array of all ones, we do it like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 1., 1., ..., 1., 1., 1.],\n",
       "       [1., 1., 1., ..., 1., 1., 1.],\n",
       "       [1., 1., 1., ..., 1., 1., 1.],\n",
       "       ...,\n",
       "       [1., 1., 1., ..., 1., 1., 1.],\n",
       "       [1., 1., 1., ..., 1., 1., 1.],\n",
       "       [1., 1., 1., ..., 1., 1., 1.]])"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "shape = (1000, 4000)\n",
    "ones_np = np.ones(shape)\n",
    "ones_np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This array contains ~ 30 MB of data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "30.517578125"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ones_np.nbytes / (1024 * 1024)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's create the same array using dask's array interface."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <td>\n",
       "            <table style=\"border-collapse: collapse;\">\n",
       "                <thead>\n",
       "                    <tr>\n",
       "                        <td> </td>\n",
       "                        <th> Array </th>\n",
       "                        <th> Chunk </th>\n",
       "                    </tr>\n",
       "                </thead>\n",
       "                <tbody>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Bytes </th>\n",
       "                        <td> 30.52 MiB </td>\n",
       "                        <td> 30.52 MiB </td>\n",
       "                    </tr>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Shape </th>\n",
       "                        <td> (1000, 4000) </td>\n",
       "                        <td> (1000, 4000) </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <th> Dask graph </th>\n",
       "                        <td colspan=\"2\"> 1 chunks in 1 graph layer </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <th> Data type </th>\n",
       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
       "                    </tr>\n",
       "                </tbody>\n",
       "            </table>\n",
       "        </td>\n",
       "        <td>\n",
       "        <svg width=\"170\" height=\"92\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"0\" y1=\"42\" x2=\"120\" y2=\"42\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"42\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"42\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,42.89879552186203 0.0,42.89879552186203\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Text -->\n",
       "  <text x=\"60.000000\" y=\"62.898796\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >4000</text>\n",
       "  <text x=\"140.000000\" y=\"21.449398\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(-90,140.000000,21.449398)\">1000</text>\n",
       "</svg>\n",
       "        </td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "dask.array<ones_like, shape=(1000, 4000), dtype=float64, chunksize=(1000, 4000), chunktype=numpy.ndarray>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import dask.array as da\n",
    "ones = da.ones(shape)\n",
    "ones"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "32000000"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ones.nbytes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dask array representation reveals the concept of \"chunks\". \"Chunks\" describes how the array is split into sub-arrays. We did not specify any chunks, so Dask just used one single chunk for the array. This is not much different from a NumPy array at this point.\n",
    "\n",
    "### Specifying Chunks\n",
    "\n",
    "However, we could have split up the array into many chunks.\n",
    "\n",
    "There are [several ways to specify chunks](https://dask.pydata.org/en/latest/array-chunks.html).\n",
    "In this lecture, we will use a block shape."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <td>\n",
       "            <table style=\"border-collapse: collapse;\">\n",
       "                <thead>\n",
       "                    <tr>\n",
       "                        <td> </td>\n",
       "                        <th> Array </th>\n",
       "                        <th> Chunk </th>\n",
       "                    </tr>\n",
       "                </thead>\n",
       "                <tbody>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Bytes </th>\n",
       "                        <td> 30.52 MiB </td>\n",
       "                        <td> 7.63 MiB </td>\n",
       "                    </tr>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Shape </th>\n",
       "                        <td> (1000, 4000) </td>\n",
       "                        <td> (1000, 1000) </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <th> Dask graph </th>\n",
       "                        <td colspan=\"2\"> 4 chunks in 1 graph layer </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <th> Data type </th>\n",
       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
       "                    </tr>\n",
       "                </tbody>\n",
       "            </table>\n",
       "        </td>\n",
       "        <td>\n",
       "        <svg width=\"170\" height=\"92\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"0\" y1=\"42\" x2=\"120\" y2=\"42\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"42\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"30\" y1=\"0\" x2=\"30\" y2=\"42\" />\n",
       "  <line x1=\"60\" y1=\"0\" x2=\"60\" y2=\"42\" />\n",
       "  <line x1=\"90\" y1=\"0\" x2=\"90\" y2=\"42\" />\n",
       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"42\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,42.89879552186203 0.0,42.89879552186203\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Text -->\n",
       "  <text x=\"60.000000\" y=\"62.898796\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >4000</text>\n",
       "  <text x=\"140.000000\" y=\"21.449398\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(-90,140.000000,21.449398)\">1000</text>\n",
       "</svg>\n",
       "        </td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "dask.array<ones_like, shape=(1000, 4000), dtype=float64, chunksize=(1000, 1000), chunktype=numpy.ndarray>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chunk_shape = (1000, 1000)\n",
    "ones = da.ones(shape, chunks=chunk_shape)\n",
    "ones"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that we just see a symbolic representation of the array, including its shape, dtype, and chunksize.\n",
    "No data has been generated yet.\n",
    "When we call `.compute()` on a dask array, the computation is triggered and the dask array becomes a NumPy array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 1., 1., ..., 1., 1., 1.],\n",
       "       [1., 1., 1., ..., 1., 1., 1.],\n",
       "       [1., 1., 1., ..., 1., 1., 1.],\n",
       "       ...,\n",
       "       [1., 1., 1., ..., 1., 1., 1.],\n",
       "       [1., 1., 1., ..., 1., 1., 1.],\n",
       "       [1., 1., 1., ..., 1., 1., 1.]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ones.compute()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to understand what happened when we called `.compute()`, we can visualize the dask _graph_, the symbolic operations that make up the array. The argument `optimize_graph=True` helps to optimize the visualization of the graph based on your window size. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4fc4c811177e497d924d0e1232f2fedb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "CytoscapeWidget(cytoscape_layout={'name': 'dagre', 'rankDir': 'BT', 'nodeSep': 10, 'edgeSep': 10, 'spacingFact…"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ones.visualize(optimize_graph=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our array has four chunks. To generate it, dask calls `np.ones` four times and then concatenates this together into one array.\n",
    "\n",
    "Rather than immediately loading a dask array (which puts all the data into RAM), it is more common to want to reduce the data somehow. For example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "eed46647119d4d298edbf0b92f0424fc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "CytoscapeWidget(cytoscape_layout={'name': 'dagre', 'rankDir': 'BT', 'nodeSep': 10, 'edgeSep': 10, 'spacingFact…"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum_of_ones = ones.sum()\n",
    "sum_of_ones.visualize(optimize_graph=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we see dask's strategy for finding the sum. This simple example illustrates the beauty of dask: it automatically designs an algorithm appropriate for custom operations with big data. \n",
    "\n",
    "If we make our operation more complex, the graph gets more complex."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1878eaa199f243ba96814d02b2e34be8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "CytoscapeWidget(cytoscape_layout={'name': 'dagre', 'rankDir': 'BT', 'nodeSep': 10, 'edgeSep': 10, 'spacingFact…"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fancy_calculation = (ones * ones[::-1, ::-1]).mean()\n",
    "fancy_calculation.visualize(optimize_graph=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A Bigger Calculation\n",
    "\n",
    "The examples above were toy examples; the data (32 MB) is nowhere nearly big enough to warrant the use of dask.\n",
    "\n",
    "We can make it a lot bigger!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <td>\n",
       "            <table style=\"border-collapse: collapse;\">\n",
       "                <thead>\n",
       "                    <tr>\n",
       "                        <td> </td>\n",
       "                        <th> Array </th>\n",
       "                        <th> Chunk </th>\n",
       "                    </tr>\n",
       "                </thead>\n",
       "                <tbody>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Bytes </th>\n",
       "                        <td> 5.96 GiB </td>\n",
       "                        <td> 7.63 MiB </td>\n",
       "                    </tr>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Shape </th>\n",
       "                        <td> (200000, 4000) </td>\n",
       "                        <td> (1000, 1000) </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <th> Dask graph </th>\n",
       "                        <td colspan=\"2\"> 800 chunks in 1 graph layer </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <th> Data type </th>\n",
       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
       "                    </tr>\n",
       "                </tbody>\n",
       "            </table>\n",
       "        </td>\n",
       "        <td>\n",
       "        <svg width=\"79\" height=\"170\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"29\" y2=\"0\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"0\" y1=\"6\" x2=\"29\" y2=\"6\" />\n",
       "  <line x1=\"0\" y1=\"12\" x2=\"29\" y2=\"12\" />\n",
       "  <line x1=\"0\" y1=\"18\" x2=\"29\" y2=\"18\" />\n",
       "  <line x1=\"0\" y1=\"25\" x2=\"29\" y2=\"25\" />\n",
       "  <line x1=\"0\" y1=\"31\" x2=\"29\" y2=\"31\" />\n",
       "  <line x1=\"0\" y1=\"37\" x2=\"29\" y2=\"37\" />\n",
       "  <line x1=\"0\" y1=\"43\" x2=\"29\" y2=\"43\" />\n",
       "  <line x1=\"0\" y1=\"50\" x2=\"29\" y2=\"50\" />\n",
       "  <line x1=\"0\" y1=\"56\" x2=\"29\" y2=\"56\" />\n",
       "  <line x1=\"0\" y1=\"63\" x2=\"29\" y2=\"63\" />\n",
       "  <line x1=\"0\" y1=\"69\" x2=\"29\" y2=\"69\" />\n",
       "  <line x1=\"0\" y1=\"75\" x2=\"29\" y2=\"75\" />\n",
       "  <line x1=\"0\" y1=\"81\" x2=\"29\" y2=\"81\" />\n",
       "  <line x1=\"0\" y1=\"88\" x2=\"29\" y2=\"88\" />\n",
       "  <line x1=\"0\" y1=\"94\" x2=\"29\" y2=\"94\" />\n",
       "  <line x1=\"0\" y1=\"100\" x2=\"29\" y2=\"100\" />\n",
       "  <line x1=\"0\" y1=\"106\" x2=\"29\" y2=\"106\" />\n",
       "  <line x1=\"0\" y1=\"113\" x2=\"29\" y2=\"113\" />\n",
       "  <line x1=\"0\" y1=\"120\" x2=\"29\" y2=\"120\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"120\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"7\" y1=\"0\" x2=\"7\" y2=\"120\" />\n",
       "  <line x1=\"14\" y1=\"0\" x2=\"14\" y2=\"120\" />\n",
       "  <line x1=\"21\" y1=\"0\" x2=\"21\" y2=\"120\" />\n",
       "  <line x1=\"29\" y1=\"0\" x2=\"29\" y2=\"120\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"0.0,0.0 29.030629010473877,0.0 29.030629010473877,120.0 0.0,120.0\" style=\"fill:#8B4903A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Text -->\n",
       "  <text x=\"14.515315\" y=\"140.000000\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >4000</text>\n",
       "  <text x=\"49.030629\" y=\"60.000000\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(-90,49.030629,60.000000)\">200000</text>\n",
       "</svg>\n",
       "        </td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "dask.array<ones_like, shape=(200000, 4000), dtype=float64, chunksize=(1000, 1000), chunktype=numpy.ndarray>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bigshape = (200000, 4000)\n",
    "big_ones = da.ones(bigshape, chunks=chunk_shape)\n",
    "big_ones"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6103.515625"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "big_ones.nbytes / (1024 * 1024)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This dataset is 3.2 GB, rather MB! This is probably close to or greater than the amount of available RAM than you have in your computer. Nevertheless, dask has no problem working on it.\n",
    "\n",
    "_Do not try to `.visualize()` this array!_\n",
    "\n",
    "When doing a big calculation, dask also has some tools to help us understand what is happening under the hood"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[########################################] | 100% Completed | 809.76 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from dask.diagnostics import ProgressBar\n",
    "\n",
    "big_calc = (big_ones * big_ones[::-1, ::-1]).mean()\n",
    "\n",
    "with ProgressBar():\n",
    "    result = big_calc.compute()\n",
    "result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reduction \n",
    "\n",
    "All the usual NumPy methods work on dask arrays (Check this section on Dask Documentation to learn what NumPy methods work in Dask which ones don't).\n",
    "You can also  the NumPy function directly to a dask array, and it will stay lazy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <td>\n",
       "            <table style=\"border-collapse: collapse;\">\n",
       "                <thead>\n",
       "                    <tr>\n",
       "                        <td> </td>\n",
       "                        <th> Array </th>\n",
       "                        <th> Chunk </th>\n",
       "                    </tr>\n",
       "                </thead>\n",
       "                <tbody>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Bytes </th>\n",
       "                        <td> 31.25 kiB </td>\n",
       "                        <td> 7.81 kiB </td>\n",
       "                    </tr>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Shape </th>\n",
       "                        <td> (4000,) </td>\n",
       "                        <td> (1000,) </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <th> Dask graph </th>\n",
       "                        <td colspan=\"2\"> 4 chunks in 8 graph layers </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <th> Data type </th>\n",
       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
       "                    </tr>\n",
       "                </tbody>\n",
       "            </table>\n",
       "        </td>\n",
       "        <td>\n",
       "        <svg width=\"170\" height=\"75\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"0\" y1=\"25\" x2=\"120\" y2=\"25\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"25\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"30\" y1=\"0\" x2=\"30\" y2=\"25\" />\n",
       "  <line x1=\"60\" y1=\"0\" x2=\"60\" y2=\"25\" />\n",
       "  <line x1=\"90\" y1=\"0\" x2=\"90\" y2=\"25\" />\n",
       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"25\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,25.412616514582485 0.0,25.412616514582485\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Text -->\n",
       "  <text x=\"60.000000\" y=\"45.412617\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >4000</text>\n",
       "  <text x=\"140.000000\" y=\"12.706308\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,140.000000,12.706308)\">1</text>\n",
       "</svg>\n",
       "        </td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "dask.array<mean_agg-aggregate, shape=(4000,), dtype=float64, chunksize=(1000,), chunktype=numpy.ndarray>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "big_ones_reduce = (np.cos(big_ones)**2).mean(axis=0)\n",
    "big_ones_reduce"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plotting also triggers computation, since we need the actual values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAckAAAGsCAYAAAC2BlyYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAllUlEQVR4nO3df1DU953H8ReiLAzKXowIEpFyNok/ILbAHS7VWFtFqSHSZE50MgR7ybWkktGaTEYKRmrNYGcaG68jJuYybUgbpB2N6VQSQ3pRcfDHFbExkWTo6AVCQAKT7FJSIcrn/nDynVvxY3YRBevzMfOdCd/97Pf73u+kffaL+60hxhgjAAAwwKjhHgAAgJGKSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAIvRwz3A9dTf36+PPvpI48aNU0hIyHCPAwAYBsYYdXd3Ky4uTqNGXfle8aaK5EcffaT4+PjhHgMAMAK0tLRo8uTJV1xzU0Vy3Lhxki5emKioqGGeBgAwHHw+n+Lj450mXMlNFckvfsUaFRVFJAHgJhfIH7vxxR0AACyIJAAAFkQSAAALIgkAgAWRBADAgkgCAGBBJAEAsCCSAABYEEkAACyIJAAAFkQSAAALIgkAgAWRBADAgkgCAGBBJAEAsCCSAABYEEkAACyIJAAAFkQSAAALIgkAgAWRBADAgkgCAGBBJAEAsCCSAABYEEkAACyIJAAAFkQSAAALIgkAgAWRBADAYlCRLC8vV2JiosLDw5Wamqra2lrr2kOHDukb3/iGbr31VkVERGjatGn6xS9+MWDdrl27NGPGDLlcLs2YMUOvvPKK3+ulpaUKCQnx22JjYwczPgAAAQk6klVVVVqzZo2Ki4vV0NCguXPnKisrS83NzZddHxkZqcLCQh08eFCNjY0qKSlRSUmJduzY4aw5fPiwcnNzlZeXp7/85S/Ky8vTsmXLdPToUb9jzZw5U21tbc528uTJYMcHACBgIcYYE8wb0tPTlZKSou3btzv7pk+frpycHJWVlQV0jPvuu0+RkZF66aWXJEm5ubny+Xx67bXXnDWLFy/WLbfcosrKSkkX7yT37NmjEydOBDOuH5/PJ7fbLa/Xq6ioqEEfBwBw4wqmBUHdSfb19am+vl6ZmZl++zMzM1VXVxfQMRoaGlRXV6d58+Y5+w4fPjzgmIsWLRpwzKamJsXFxSkxMVHLly/X6dOnr3iu3t5e+Xw+vw0AgEAFFcnOzk5duHBBMTExfvtjYmLU3t5+xfdOnjxZLpdLaWlpWrVqlR5++GHntfb29i89Znp6uioqKrRv3z49//zzam9vV0ZGhrq6uqznLCsrk9vtdrb4+PhgPi4A4CY3qC/uhISE+P1sjBmw71K1tbX685//rGeffVbPPPOM82vUQI+ZlZWl+++/X8nJyVqwYIH27t0rSXrxxRet5ywqKpLX63W2lpaWgD4fAACSNDqYxRMmTFBoaOiAu8aOjo4Bd4KXSkxMlCQlJyfr7NmzKi0t1YoVKyRJsbGxQR8zMjJSycnJampqsq5xuVxyuVxXnAsAAJug7iTDwsKUmpqqmpoav/01NTXKyMgI+DjGGPX29jo/ezyeAcd84403rnjM3t5eNTY2atKkSQGfFwCAYAR1JylJa9euVV5entLS0uTxeLRjxw41NzeroKBA0sVfcba2tqqiokKStG3bNk2ZMkXTpk2TdPG5yZ///Od69NFHnWOuXr1ad999t372s59p6dKlevXVV/Xmm2/q0KFDzprHH39c2dnZmjJlijo6OrRp0yb5fD7l5+df1QUAAMAm6Ejm5uaqq6tLGzduVFtbm5KSklRdXa2EhARJUltbm98zk/39/SoqKtKZM2c0evRoTZ06VZs3b9YPfvADZ01GRoZ27typkpISrV+/XlOnTlVVVZXS09OdNR9++KFWrFihzs5ORUdHa/bs2Tpy5IhzXgAAhlrQz0neyHhOEgBwzZ6TBADgZkIkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYDGoSJaXlysxMVHh4eFKTU1VbW2tde2hQ4f0jW98Q7feeqsiIiI0bdo0/eIXvxiwbteuXZoxY4ZcLpdmzJihV1555arOCwDA1Qo6klVVVVqzZo2Ki4vV0NCguXPnKisrS83NzZddHxkZqcLCQh08eFCNjY0qKSlRSUmJduzY4aw5fPiwcnNzlZeXp7/85S/Ky8vTsmXLdPTo0UGfFwCAqxVijDHBvCE9PV0pKSnavn27s2/69OnKyclRWVlZQMe47777FBkZqZdeekmSlJubK5/Pp9dee81Zs3jxYt1yyy2qrKwcsvP6fD653W55vV5FRUUF9B4AwD+WYFoQ1J1kX1+f6uvrlZmZ6bc/MzNTdXV1AR2joaFBdXV1mjdvnrPv8OHDA465aNEi55iDPW9vb698Pp/fBgBAoIKKZGdnpy5cuKCYmBi//TExMWpvb7/ieydPniyXy6W0tDStWrVKDz/8sPNae3v7FY852POWlZXJ7XY7W3x8fECfEwAAaZBf3AkJCfH72RgzYN+lamtr9ec//1nPPvusnnnmGefXqMEcM9jzFhUVyev1OltLS8sVZwQA4P8bHcziCRMmKDQ0dMDdW0dHx4C7vEslJiZKkpKTk3X27FmVlpZqxYoVkqTY2NgrHnOw53W5XHK5XIF9OAAALhHUnWRYWJhSU1NVU1Pjt7+mpkYZGRkBH8cYo97eXudnj8cz4JhvvPGGc8yhOi8AAMEI6k5SktauXau8vDylpaXJ4/Fox44dam5uVkFBgaSLv+JsbW1VRUWFJGnbtm2aMmWKpk2bJunic5M///nP9eijjzrHXL16te6++2797Gc/09KlS/Xqq6/qzTff1KFDhwI+LwAAQy3oSObm5qqrq0sbN25UW1ubkpKSVF1drYSEBElSW1ub37OL/f39Kioq0pkzZzR69GhNnTpVmzdv1g9+8ANnTUZGhnbu3KmSkhKtX79eU6dOVVVVldLT0wM+LwAAQy3o5yRvZDwnCQC4Zs9JAgBwMyGSAABYEEkAACyIJAAAFkQSAAALIgkAgAWRBADAgkgCAGBBJAEAsCCSAABYEEkAACyIJAAAFkQSAAALIgkAgAWRBADAgkgCAGBBJAEAsCCSAABYEEkAACyIJAAAFkQSAAALIgkAgAWRBADAgkgCAGBBJAEAsCCSAABYEEkAACyIJAAAFkQSAAALIgkAgAWRBADAgkgCAGBBJAEAsCCSAABYEEkAACyIJAAAFkQSAAALIgkAgAWRBADAgkgCAGBBJAEAsCCSAABYEEkAACyIJAAAFkQSAAALIgkAgAWRBADAgkgCAGBBJAEAsBhUJMvLy5WYmKjw8HClpqaqtrbWunb37t1auHChoqOjFRUVJY/Ho3379vmt+fzzz7Vx40ZNnTpV4eHhmjVrll5//XW/NaWlpQoJCfHbYmNjBzM+AAABCTqSVVVVWrNmjYqLi9XQ0KC5c+cqKytLzc3Nl11/8OBBLVy4UNXV1aqvr9f8+fOVnZ2thoYGZ01JSYmee+45/fKXv9SpU6dUUFCg7373u35rJGnmzJlqa2tztpMnTwY7PgAAAQsxxphg3pCenq6UlBRt377d2Td9+nTl5OSorKwsoGPMnDlTubm5evLJJyVJcXFxKi4u1qpVq5w1OTk5Gjt2rH7zm99IungnuWfPHp04cSKYcf34fD653W55vV5FRUUN+jgAgBtXMC0I6k6yr69P9fX1yszM9NufmZmpurq6gI7R39+v7u5ujR8/3tnX29ur8PBwv3URERE6dOiQ376mpibFxcUpMTFRy5cv1+nTp694rt7eXvl8Pr8NAIBABRXJzs5OXbhwQTExMX77Y2Ji1N7eHtAxnn76afX09GjZsmXOvkWLFmnLli1qampSf3+/ampq9Oqrr6qtrc1Zk56eroqKCu3bt0/PP/+82tvblZGRoa6uLuu5ysrK5Ha7nS0+Pj6YjwsAuMkN6os7ISEhfj8bYwbsu5zKykqVlpaqqqpKEydOdPZv3bpVt99+u6ZNm6awsDAVFhbqe9/7nkJDQ501WVlZuv/++5WcnKwFCxZo7969kqQXX3zRer6ioiJ5vV5na2lpCfajAgBuYkFFcsKECQoNDR1w19jR0THg7vJSVVVVeuihh/S73/1OCxYs8HstOjpae/bsUU9Pjz744AO99957Gjt2rBITE63Hi4yMVHJyspqamqxrXC6XoqKi/DYAAAIVVCTDwsKUmpqqmpoav/01NTXKyMiwvq+yslIrV67Uyy+/rCVLlljXhYeH67bbbtP58+e1a9cuLV261Lq2t7dXjY2NmjRpUjAfAQCAgI0O9g1r165VXl6e0tLS5PF4tGPHDjU3N6ugoEDSxV9xtra2qqKiQtLFQD744IPaunWrZs+e7dyFRkREyO12S5KOHj2q1tZWfe1rX1Nra6tKS0vV39+vJ554wjnv448/ruzsbE2ZMkUdHR3atGmTfD6f8vPzr/oiAABwOUFHMjc3V11dXdq4caPa2tqUlJSk6upqJSQkSJLa2tr8npl87rnndP78ea1atcrvEY/8/Hz9+te/liSdO3dOJSUlOn36tMaOHavvfOc7eumll/RP//RPzvoPP/xQK1asUGdnp6KjozV79mwdOXLEOS8AAEMt6Ockb2Q8JwkAuGbPSQIAcDMhkgAAWBBJAAAsiCQAABZEEgAACyIJAIAFkQQAwIJIAgBgQSQBALAgkgAAWBBJAAAsiCQAABZEEgAACyIJAIAFkQQAwIJIAgBgQSQBALAgkgAAWBBJAAAsiCQAABZEEgAACyIJAIAFkQQAwIJIAgBgQSQBALAgkgAAWBBJAAAsiCQAABajh3uAG039B5/o730XhnsMALhpTb4lQl+ZEHldzkUkg7Ru19tq6vjbcI8BADetH35zqp5YPO26nItIBilxQqRCR4UM9xgAcNOKHue6bucikkHa8WDacI8AALhO+OIOAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWg4pkeXm5EhMTFR4ertTUVNXW1lrX7t69WwsXLlR0dLSioqLk8Xi0b98+vzWff/65Nm7cqKlTpyo8PFyzZs3S66+/flXnBQDgagUdyaqqKq1Zs0bFxcVqaGjQ3LlzlZWVpebm5suuP3jwoBYuXKjq6mrV19dr/vz5ys7OVkNDg7OmpKREzz33nH75y1/q1KlTKigo0He/+12/NcGeFwCAqxVijDHBvCE9PV0pKSnavn27s2/69OnKyclRWVlZQMeYOXOmcnNz9eSTT0qS4uLiVFxcrFWrVjlrcnJyNHbsWP3mN78ZsvP6fD653W55vV5FRUUF9B4AwD+WYFoQ1J1kX1+f6uvrlZmZ6bc/MzNTdXV1AR2jv79f3d3dGj9+vLOvt7dX4eHhfusiIiJ06NChqzpvb2+vfD6f3wYAQKCCimRnZ6cuXLigmJgYv/0xMTFqb28P6BhPP/20enp6tGzZMmffokWLtGXLFjU1Nam/v181NTV69dVX1dbWdlXnLSsrk9vtdrb4+PhAPyoAAIP74k5ISIjfz8aYAfsup7KyUqWlpaqqqtLEiROd/Vu3btXtt9+uadOmKSwsTIWFhfre976n0NDQqzpvUVGRvF6vs7W0tATy8QAAkBRkJCdMmKDQ0NABd28dHR0D7vIuVVVVpYceeki/+93vtGDBAr/XoqOjtWfPHvX09OiDDz7Qe++9p7FjxyoxMfGqzutyuRQVFeW3AQAQqKAiGRYWptTUVNXU1Pjtr6mpUUZGhvV9lZWVWrlypV5++WUtWbLEui48PFy33Xabzp8/r127dmnp0qVXdV4AAK7G6GDfsHbtWuXl5SktLU0ej0c7duxQc3OzCgoKJF38FWdra6sqKiokXQzkgw8+qK1bt2r27NnO3WBERITcbrck6ejRo2ptbdXXvvY1tba2qrS0VP39/XriiScCPi8AAEPODMK2bdtMQkKCCQsLMykpKebAgQPOa/n5+WbevHnOz/PmzTOSBmz5+fnOmv3795vp06cbl8tlbr31VpOXl2daW1uDOm8gvF6vkWS8Xm/QnxkA8I8hmBYE/ZzkjYznJAEA1+w5SQAAbiZEEgAACyIJAIAFkQQAwIJIAgBgQSQBALAgkgAAWBBJAAAsiCQAABZEEgAACyIJAIAFkQQAwIJIAgBgQSQBALAgkgAAWBBJAAAsiCQAABZEEgAACyIJAIAFkQQAwIJIAgBgQSQBALAgkgAAWBBJAAAsiCQAABZEEgAACyIJAIAFkQQAwIJIAgBgQSQBALAgkgAAWBBJAAAsiCQAABZEEgAACyIJAIAFkQQAwIJIAgBgQSQBALAgkgAAWBBJAAAsiCQAABZEEgAACyIJAIAFkQQAwIJIAgBgQSQBALAgkgAAWBBJAAAsiCQAABaDimR5ebkSExMVHh6u1NRU1dbWWtfu3r1bCxcuVHR0tKKiouTxeLRv374B65555hndeeedioiIUHx8vH70ox/p3LlzzuulpaUKCQnx22JjYwczPgAAAQk6klVVVVqzZo2Ki4vV0NCguXPnKisrS83NzZddf/DgQS1cuFDV1dWqr6/X/PnzlZ2drYaGBmfNb3/7W61bt04bNmxQY2OjXnjhBVVVVamoqMjvWDNnzlRbW5uznTx5MtjxAQAIWIgxxgTzhvT0dKWkpGj79u3OvunTpysnJ0dlZWUBHWPmzJnKzc3Vk08+KUkqLCxUY2Oj/vSnPzlrHnvsMR07dsy5Sy0tLdWePXt04sSJYMb14/P55Ha75fV6FRUVNejjAABuXMG0IKg7yb6+PtXX1yszM9Nvf2Zmpurq6gI6Rn9/v7q7uzV+/Hhn35w5c1RfX69jx45Jkk6fPq3q6motWbLE771NTU2Ki4tTYmKili9frtOnT1/xXL29vfL5fH4bAACBGh3M4s7OTl24cEExMTF++2NiYtTe3h7QMZ5++mn19PRo2bJlzr7ly5fr448/1pw5c2SM0fnz5/XII49o3bp1zpr09HRVVFTojjvu0NmzZ7Vp0yZlZGTo3Xff1a233nrZc5WVleknP/lJMB8RAADHoL64ExIS4vezMWbAvsuprKxUaWmpqqqqNHHiRGf//v379dRTT6m8vFzHjx/X7t279cc//lE//elPnTVZWVm6//77lZycrAULFmjv3r2SpBdffNF6vqKiInm9XmdraWkJ9qMCAG5iQd1JTpgwQaGhoQPuGjs6OgbcXV6qqqpKDz30kH7/+99rwYIFfq+tX79eeXl5evjhhyVJycnJ6unp0fe//30VFxdr1KiBLY+MjFRycrKampqs53S5XHK5XIF+PAAA/AR1JxkWFqbU1FTV1NT47a+pqVFGRob1fZWVlVq5cqVefvnlAX/OKEmfffbZgBCGhobKGCPb94p6e3vV2NioSZMmBfMRAAAIWFB3kpK0du1a5eXlKS0tTR6PRzt27FBzc7MKCgokXfwVZ2trqyoqKiRdDOSDDz6orVu3avbs2c5daEREhNxutyQpOztbW7Zs0de//nWlp6frr3/9q9avX697771XoaGhkqTHH39c2dnZmjJlijo6OrRp0yb5fD7l5+cPyYUAAOBSQUcyNzdXXV1d2rhxo9ra2pSUlKTq6molJCRIktra2vyemXzuued0/vx5rVq1SqtWrXL25+fn69e//rUkqaSkRCEhISopKVFra6uio6OVnZ2tp556yln/4YcfasWKFers7FR0dLRmz56tI0eOOOcFAGCoBf2c5I2M5yQBANfsOUkAAG4mRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWg4pkeXm5EhMTFR4ertTUVNXW1lrX7t69WwsXLlR0dLSioqLk8Xi0b9++AeueeeYZ3XnnnYqIiFB8fLx+9KMf6dy5c4M+LwAAVyvoSFZVVWnNmjUqLi5WQ0OD5s6dq6ysLDU3N192/cGDB7Vw4UJVV1ervr5e8+fPV3Z2thoaGpw1v/3tb7Vu3Tpt2LBBjY2NeuGFF1RVVaWioqJBnxcAgKsVYowxwbwhPT1dKSkp2r59u7Nv+vTpysnJUVlZWUDHmDlzpnJzc/Xkk09KkgoLC9XY2Kg//elPzprHHntMx44dc+4Wh+K8Pp9PbrdbXq9XUVFRAb0HAPCPJZgWBHUn2dfXp/r6emVmZvrtz8zMVF1dXUDH6O/vV3d3t8aPH+/smzNnjurr63Xs2DFJ0unTp1VdXa0lS5Zc1Xl7e3vl8/n8NgAAAjU6mMWdnZ26cOGCYmJi/PbHxMSovb09oGM8/fTT6unp0bJly5x9y5cv18cff6w5c+bIGKPz58/rkUce0bp1667qvGVlZfrJT34S6McDAMDPoL64ExIS4vezMWbAvsuprKxUaWmpqqqqNHHiRGf//v379dRTT6m8vFzHjx/X7t279cc//lE//elPr+q8RUVF8nq9ztbS0hLIxwMAQFKQd5ITJkxQaGjogLu3jo6OAXd5l6qqqtJDDz2k3//+91qwYIHfa+vXr1deXp4efvhhSVJycrJ6enr0/e9/X8XFxYM+r8vlksvlCuYjAgDgCOpOMiwsTKmpqaqpqfHbX1NTo4yMDOv7KisrtXLlSr388svOnzP+f5999plGjfIfJTQ0VMYYGWMGfV4AAK5GUHeSkrR27Vrl5eUpLS1NHo9HO3bsUHNzswoKCiRd/BVna2urKioqJF0M5IMPPqitW7dq9uzZzt1gRESE3G63JCk7O1tbtmzR17/+daWnp+uvf/2r1q9fr3vvvVehoaEBnRcAgCFnBmHbtm0mISHBhIWFmZSUFHPgwAHntfz8fDNv3jzn53nz5hlJA7b8/Hxnzeeff25KS0vN1KlTTXh4uImPjzc//OEPzSeffBLweQPh9XqNJOP1egfzsQEA/wCCaUHQz0neyHhOEgBwzZ6TBADgZkIkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkAQCwIJIAAFgQSQAALIgkAAAWRBIAAAsiCQCABZEEAMCCSAIAYDF6uAe4nowxkiSfzzfMkwAAhssXDfiiCVdyU0Wyu7tbkhQfHz/MkwAAhlt3d7fcbvcV14SYQFL6D6K/v18fffSRxo0bp5CQkEEdw+fzKT4+Xi0tLYqKihriCYce815bzHttMe+1d6PNPBTzGmPU3d2tuLg4jRp15T91vKnuJEeNGqXJkycPybGioqJuiH+hvsC81xbzXlvMe+3daDNf7bxfdgf5Bb64AwCABZEEAMCCSAbJ5XJpw4YNcrlcwz1KQJj32mLea4t5r70bbebrPe9N9cUdAACCwZ0kAAAWRBIAAAsiCQCABZEEAMCCSAIAYEEkg1ReXq7ExESFh4crNTVVtbW1132G0tJShYSE+G2xsbHO68YYlZaWKi4uThEREfrmN7+pd9991+8Yvb29evTRRzVhwgRFRkbq3nvv1Ycffjgk8x08eFDZ2dmKi4tTSEiI9uzZ4/f6UM33ySefKC8vT263W263W3l5efr000+HfN6VK1cOuN6zZ88etnnLysr0L//yLxo3bpwmTpyonJwcvf/++35rRtI1DmTekXSNt2/frrvuusv5f3TxeDx67bXXnNdH0rUNZN6RdG0vp6ysTCEhIVqzZo2zb0RdY4OA7dy504wZM8Y8//zz5tSpU2b16tUmMjLSfPDBB9d1jg0bNpiZM2eatrY2Z+vo6HBe37x5sxk3bpzZtWuXOXnypMnNzTWTJk0yPp/PWVNQUGBuu+02U1NTY44fP27mz59vZs2aZc6fP3/V81VXV5vi4mKza9cuI8m88sorfq8P1XyLFy82SUlJpq6uztTV1ZmkpCRzzz33DPm8+fn5ZvHixX7Xu6ury2/N9Zx30aJF5le/+pV55513zIkTJ8ySJUvMlClTzN/+9jdnzUi6xoHMO5Ku8R/+8Aezd+9e8/7775v333/f/PjHPzZjxowx77zzzoi7toHMO5Ku7aWOHTtmvvKVr5i77rrLrF692tk/kq4xkQzCv/7rv5qCggK/fdOmTTPr1q27rnNs2LDBzJo167Kv9ff3m9jYWLN582Zn37lz54zb7TbPPvusMcaYTz/91IwZM8bs3LnTWdPa2mpGjRplXn/99SGd9dLoDNV8p06dMpLMkSNHnDWHDx82ksx77703ZPMac/G/ZJYuXWp9z3DOa4wxHR0dRpI5cOCAMWbkX+NL5zVm5F/jW265xfzXf/3XiL+2l85rzMi9tt3d3eb22283NTU1Zt68eU4kR9o15tetAerr61N9fb0yMzP99mdmZqquru66z9PU1KS4uDglJiZq+fLlOn36tCTpzJkzam9v95vT5XJp3rx5zpz19fX6/PPP/dbExcUpKSnpmn+WoZrv8OHDcrvdSk9Pd9bMnj1bbrf7mnyG/fv3a+LEibrjjjv0H//xH+ro6HBeG+55vV6vJGn8+PGSRv41vnTeL4zEa3zhwgXt3LlTPT098ng8I/7aXjrvF0bitV21apWWLFmiBQsW+O0fadf4pvpbQK5GZ2enLly4oJiYGL/9MTExam9vv66zpKenq6KiQnfccYfOnj2rTZs2KSMjQ++++64zy+Xm/OCDDyRJ7e3tCgsL0y233DJgzbX+LEM1X3t7uyZOnDjg+BMnThzyz5CVlaV/+7d/U0JCgs6cOaP169frW9/6lurr6+VyuYZ1XmOM1q5dqzlz5igpKck51xfnv3Se4b7Gl5tXGnnX+OTJk/J4PDp37pzGjh2rV155RTNmzHD+y3WkXVvbvNLIu7aStHPnTh0/flz/8z//M+C1kfbvL5EM0qV/D6UxZtB/N+VgZWVlOf+cnJwsj8ejqVOn6sUXX3T+QH4wc17PzzIU811u/bX4DLm5uc4/JyUlKS0tTQkJCdq7d6/uu+++YZ23sLBQb7/9tg4dOjTgtZF4jW3zjrRrfOedd+rEiRP69NNPtWvXLuXn5+vAgQPW8wz3tbXNO2PGjBF3bVtaWrR69Wq98cYbCg8Pt64bKdeYX7cGaMKECQoNDR3wv0A6OjoG/C+e6y0yMlLJyclqampyvuV6pTljY2PV19enTz75xLrmWhmq+WJjY3X27NkBx//444+v+WeYNGmSEhIS1NTUNKzzPvroo/rDH/6gt956y+/vSR2p19g27+UM9zUOCwvTV7/6VaWlpamsrEyzZs3S1q1bR+y1tc17OcN9bevr69XR0aHU1FSNHj1ao0eP1oEDB/Sf//mfGj16tHO8kXKNiWSAwsLClJqaqpqaGr/9NTU1ysjIGKapLurt7VVjY6MmTZqkxMRExcbG+s3Z19enAwcOOHOmpqZqzJgxfmva2tr0zjvvXPPPMlTzeTweeb1eHTt2zFlz9OhReb3ea/4Zurq61NLSokmTJg3LvMYYFRYWavfu3frv//5vJSYm+r0+0q7xl817OcN9jS/3GXp7e0fctf2yeS9nuK/tt7/9bZ08eVInTpxwtrS0ND3wwAM6ceKE/vmf/3lkXeOAv+ID5xGQF154wZw6dcqsWbPGREZGmv/93/+9rnM89thjZv/+/eb06dPmyJEj5p577jHjxo1z5ti8ebNxu91m9+7d5uTJk2bFihWX/fr05MmTzZtvvmmOHz9uvvWtbw3ZIyDd3d2moaHBNDQ0GElmy5YtpqGhwXlUZqjmW7x4sbnrrrvM4cOHzeHDh01ycvKgvpJ+pXm7u7vNY489Zurq6syZM2fMW2+9ZTwej7ntttuGbd5HHnnEuN1us3//fr+v9X/22WfOmpF0jb9s3pF2jYuKiszBgwfNmTNnzNtvv21+/OMfm1GjRpk33nhjxF3bL5t3pF1bm///7VZjRtY1JpJB2rZtm0lISDBhYWEmJSXF72vs18sXzwyNGTPGxMXFmfvuu8+8++67zuv9/f1mw4YNJjY21rhcLnP33XebkydP+h3j73//uyksLDTjx483ERER5p577jHNzc1DMt9bb71lJA3Y8vPzh3S+rq4u88ADD5hx48aZcePGmQceeMB88sknQzrvZ599ZjIzM010dLQZM2aMmTJlisnPzx8wy/Wc93KzSjK/+tWvnDUj6Rp/2bwj7Rr/+7//u/Of8ejoaPPtb3/bCaQxI+vaftm8I+3a2lwayZF0jfn7JAEAsODPJAEAsCCSAABYEEkAACyIJAAAFkQSAAALIgkAgAWRBADAgkgCAGBBJAEAsCCSAABYEEkAACz+Dx9uphDqkQ4+AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 500x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "plt.figure(figsize = (5, 5))\n",
    "plt.plot(big_ones_reduce)\n",
    "plt.draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Distributed Clusters\n",
    "\n",
    "Once we are ready to make a bigger calculation with dask, we can use a Dask [Distributed](http://distributed.dask.org/en/stable/) cluster.\n",
    "\n",
    "```{warning}\n",
    "A common mistake is to move to distributed mode too soon.\n",
    "For smaller data, distributed will actually be _much slower_ than the default multi-threaded scheduler or **not using Dask at all**.\n",
    "You should only use distributed when your data is much larger than what your computer can handle in memory.\n",
    "```\n",
    "\n",
    "### Local Cluster\n",
    "\n",
    "A local cluster uses all the CPU cores of the machine it is running on.\n",
    "For our cloud-based Jupyterlab environments, that is just 2 cores--not very much.\n",
    "However, it's good to know about."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "    <div style=\"width: 24px; height: 24px; background-color: #e1e1e1; border: 3px solid #9D9D9D; border-radius: 5px; position: absolute;\"> </div>\n",
       "    <div style=\"margin-left: 48px;\">\n",
       "        <h3 style=\"margin-bottom: 0px;\">Client</h3>\n",
       "        <p style=\"color: #9D9D9D; margin-bottom: 0px;\">Client-e89d4a18-87e3-11ef-8108-0242ac110002</p>\n",
       "        <table style=\"width: 100%; text-align: left;\">\n",
       "\n",
       "        <tr>\n",
       "        \n",
       "            <td style=\"text-align: left;\"><strong>Connection method:</strong> Cluster object</td>\n",
       "            <td style=\"text-align: left;\"><strong>Cluster type:</strong> distributed.LocalCluster</td>\n",
       "        \n",
       "        </tr>\n",
       "\n",
       "        \n",
       "            <tr>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Dashboard: </strong> <a href=\"http://127.0.0.1:8787/status\" target=\"_blank\">http://127.0.0.1:8787/status</a>\n",
       "                </td>\n",
       "                <td style=\"text-align: left;\"></td>\n",
       "            </tr>\n",
       "        \n",
       "\n",
       "        </table>\n",
       "\n",
       "        \n",
       "            <button style=\"margin-bottom: 12px;\" data-commandlinker-command=\"dask:populate-and-launch-layout\" data-commandlinker-args='{\"url\": \"http://127.0.0.1:8787/status\" }'>\n",
       "                Launch dashboard in JupyterLab\n",
       "            </button>\n",
       "        \n",
       "\n",
       "        \n",
       "            <details>\n",
       "            <summary style=\"margin-bottom: 20px;\"><h3 style=\"display: inline;\">Cluster Info</h3></summary>\n",
       "            <div class=\"jp-RenderedHTMLCommon jp-RenderedHTML jp-mod-trusted jp-OutputArea-output\">\n",
       "    <div style=\"width: 24px; height: 24px; background-color: #e1e1e1; border: 3px solid #9D9D9D; border-radius: 5px; position: absolute;\">\n",
       "    </div>\n",
       "    <div style=\"margin-left: 48px;\">\n",
       "        <h3 style=\"margin-bottom: 0px; margin-top: 0px;\">LocalCluster</h3>\n",
       "        <p style=\"color: #9D9D9D; margin-bottom: 0px;\">a1af8bb3</p>\n",
       "        <table style=\"width: 100%; text-align: left;\">\n",
       "            <tr>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Dashboard:</strong> <a href=\"http://127.0.0.1:8787/status\" target=\"_blank\">http://127.0.0.1:8787/status</a>\n",
       "                </td>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Workers:</strong> 4\n",
       "                </td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Total threads:</strong> 8\n",
       "                </td>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Total memory:</strong> 23.20 GiB\n",
       "                </td>\n",
       "            </tr>\n",
       "            \n",
       "            <tr>\n",
       "    <td style=\"text-align: left;\"><strong>Status:</strong> running</td>\n",
       "    <td style=\"text-align: left;\"><strong>Using processes:</strong> True</td>\n",
       "</tr>\n",
       "\n",
       "            \n",
       "        </table>\n",
       "\n",
       "        <details>\n",
       "            <summary style=\"margin-bottom: 20px;\">\n",
       "                <h3 style=\"display: inline;\">Scheduler Info</h3>\n",
       "            </summary>\n",
       "\n",
       "            <div style=\"\">\n",
       "    <div>\n",
       "        <div style=\"width: 24px; height: 24px; background-color: #FFF7E5; border: 3px solid #FF6132; border-radius: 5px; position: absolute;\"> </div>\n",
       "        <div style=\"margin-left: 48px;\">\n",
       "            <h3 style=\"margin-bottom: 0px;\">Scheduler</h3>\n",
       "            <p style=\"color: #9D9D9D; margin-bottom: 0px;\">Scheduler-6252adc7-8def-425e-a059-040adac6557c</p>\n",
       "            <table style=\"width: 100%; text-align: left;\">\n",
       "                <tr>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Comm:</strong> tcp://127.0.0.1:40865\n",
       "                    </td>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Workers:</strong> 4\n",
       "                    </td>\n",
       "                </tr>\n",
       "                <tr>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Dashboard:</strong> <a href=\"http://127.0.0.1:8787/status\" target=\"_blank\">http://127.0.0.1:8787/status</a>\n",
       "                    </td>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Total threads:</strong> 8\n",
       "                    </td>\n",
       "                </tr>\n",
       "                <tr>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Started:</strong> Just now\n",
       "                    </td>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Total memory:</strong> 23.20 GiB\n",
       "                    </td>\n",
       "                </tr>\n",
       "            </table>\n",
       "        </div>\n",
       "    </div>\n",
       "\n",
       "    <details style=\"margin-left: 48px;\">\n",
       "        <summary style=\"margin-bottom: 20px;\">\n",
       "            <h3 style=\"display: inline;\">Workers</h3>\n",
       "        </summary>\n",
       "\n",
       "        \n",
       "        <div style=\"margin-bottom: 20px;\">\n",
       "            <div style=\"width: 24px; height: 24px; background-color: #DBF5FF; border: 3px solid #4CC9FF; border-radius: 5px; position: absolute;\"> </div>\n",
       "            <div style=\"margin-left: 48px;\">\n",
       "            <details>\n",
       "                <summary>\n",
       "                    <h4 style=\"margin-bottom: 0px; display: inline;\">Worker: 0</h4>\n",
       "                </summary>\n",
       "                <table style=\"width: 100%; text-align: left;\">\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Comm: </strong> tcp://127.0.0.1:40481\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Total threads: </strong> 2\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Dashboard: </strong> <a href=\"http://127.0.0.1:45757/status\" target=\"_blank\">http://127.0.0.1:45757/status</a>\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Memory: </strong> 5.80 GiB\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Nanny: </strong> tcp://127.0.0.1:33941\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\"></td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td colspan=\"2\" style=\"text-align: left;\">\n",
       "                            <strong>Local directory: </strong> /tmp/dask-scratch-space/worker-74hm7c4j\n",
       "                        </td>\n",
       "                    </tr>\n",
       "\n",
       "                    \n",
       "\n",
       "                    \n",
       "\n",
       "                </table>\n",
       "            </details>\n",
       "            </div>\n",
       "        </div>\n",
       "        \n",
       "        <div style=\"margin-bottom: 20px;\">\n",
       "            <div style=\"width: 24px; height: 24px; background-color: #DBF5FF; border: 3px solid #4CC9FF; border-radius: 5px; position: absolute;\"> </div>\n",
       "            <div style=\"margin-left: 48px;\">\n",
       "            <details>\n",
       "                <summary>\n",
       "                    <h4 style=\"margin-bottom: 0px; display: inline;\">Worker: 1</h4>\n",
       "                </summary>\n",
       "                <table style=\"width: 100%; text-align: left;\">\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Comm: </strong> tcp://127.0.0.1:35465\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Total threads: </strong> 2\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Dashboard: </strong> <a href=\"http://127.0.0.1:45121/status\" target=\"_blank\">http://127.0.0.1:45121/status</a>\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Memory: </strong> 5.80 GiB\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Nanny: </strong> tcp://127.0.0.1:33029\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\"></td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td colspan=\"2\" style=\"text-align: left;\">\n",
       "                            <strong>Local directory: </strong> /tmp/dask-scratch-space/worker-26klett3\n",
       "                        </td>\n",
       "                    </tr>\n",
       "\n",
       "                    \n",
       "\n",
       "                    \n",
       "\n",
       "                </table>\n",
       "            </details>\n",
       "            </div>\n",
       "        </div>\n",
       "        \n",
       "        <div style=\"margin-bottom: 20px;\">\n",
       "            <div style=\"width: 24px; height: 24px; background-color: #DBF5FF; border: 3px solid #4CC9FF; border-radius: 5px; position: absolute;\"> </div>\n",
       "            <div style=\"margin-left: 48px;\">\n",
       "            <details>\n",
       "                <summary>\n",
       "                    <h4 style=\"margin-bottom: 0px; display: inline;\">Worker: 2</h4>\n",
       "                </summary>\n",
       "                <table style=\"width: 100%; text-align: left;\">\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Comm: </strong> tcp://127.0.0.1:37889\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Total threads: </strong> 2\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Dashboard: </strong> <a href=\"http://127.0.0.1:37131/status\" target=\"_blank\">http://127.0.0.1:37131/status</a>\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Memory: </strong> 5.80 GiB\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Nanny: </strong> tcp://127.0.0.1:40107\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\"></td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td colspan=\"2\" style=\"text-align: left;\">\n",
       "                            <strong>Local directory: </strong> /tmp/dask-scratch-space/worker-msv884v0\n",
       "                        </td>\n",
       "                    </tr>\n",
       "\n",
       "                    \n",
       "\n",
       "                    \n",
       "\n",
       "                </table>\n",
       "            </details>\n",
       "            </div>\n",
       "        </div>\n",
       "        \n",
       "        <div style=\"margin-bottom: 20px;\">\n",
       "            <div style=\"width: 24px; height: 24px; background-color: #DBF5FF; border: 3px solid #4CC9FF; border-radius: 5px; position: absolute;\"> </div>\n",
       "            <div style=\"margin-left: 48px;\">\n",
       "            <details>\n",
       "                <summary>\n",
       "                    <h4 style=\"margin-bottom: 0px; display: inline;\">Worker: 3</h4>\n",
       "                </summary>\n",
       "                <table style=\"width: 100%; text-align: left;\">\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Comm: </strong> tcp://127.0.0.1:45103\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Total threads: </strong> 2\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Dashboard: </strong> <a href=\"http://127.0.0.1:40561/status\" target=\"_blank\">http://127.0.0.1:40561/status</a>\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Memory: </strong> 5.80 GiB\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Nanny: </strong> tcp://127.0.0.1:41359\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\"></td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td colspan=\"2\" style=\"text-align: left;\">\n",
       "                            <strong>Local directory: </strong> /tmp/dask-scratch-space/worker-zrgl_4cb\n",
       "                        </td>\n",
       "                    </tr>\n",
       "\n",
       "                    \n",
       "\n",
       "                    \n",
       "\n",
       "                </table>\n",
       "            </details>\n",
       "            </div>\n",
       "        </div>\n",
       "        \n",
       "\n",
       "    </details>\n",
       "</div>\n",
       "\n",
       "        </details>\n",
       "    </div>\n",
       "</div>\n",
       "            </details>\n",
       "        \n",
       "\n",
       "    </div>\n",
       "</div>"
      ],
      "text/plain": [
       "<Client: 'tcp://127.0.0.1:40865' processes=4 threads=8, memory=23.20 GiB>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from dask.distributed import Client, LocalCluster\n",
    "cluster = LocalCluster()\n",
    "client = Client(cluster)\n",
    "client"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the \"Dashboard\" link will open a new page where you can monitor a computation's progress."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "big_calc.compute()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is another bigger calculation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <td>\n",
       "            <table style=\"border-collapse: collapse;\">\n",
       "                <thead>\n",
       "                    <tr>\n",
       "                        <td> </td>\n",
       "                        <th> Array </th>\n",
       "                        <th> Chunk </th>\n",
       "                    </tr>\n",
       "                </thead>\n",
       "                <tbody>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Bytes </th>\n",
       "                        <td> 800 B </td>\n",
       "                        <td> 800 B </td>\n",
       "                    </tr>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Shape </th>\n",
       "                        <td> (100,) </td>\n",
       "                        <td> (100,) </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <th> Dask graph </th>\n",
       "                        <td colspan=\"2\"> 1 chunks in 9 graph layers </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <th> Data type </th>\n",
       "                        <td colspan=\"2\"> int64 numpy.ndarray </td>\n",
       "                    </tr>\n",
       "                </tbody>\n",
       "            </table>\n",
       "        </td>\n",
       "        <td>\n",
       "        <svg width=\"170\" height=\"75\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"0\" y1=\"25\" x2=\"120\" y2=\"25\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"25\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"25\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,25.412616514582485 0.0,25.412616514582485\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Text -->\n",
       "  <text x=\"60.000000\" y=\"45.412617\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >100</text>\n",
       "  <text x=\"140.000000\" y=\"12.706308\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,140.000000,12.706308)\">1</text>\n",
       "</svg>\n",
       "        </td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "dask.array<sum-aggregate, shape=(100,), dtype=int64, chunksize=(100,), chunktype=numpy.ndarray>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "random_values = da.random.normal(size=(2e9,), chunks=(1e6,))\n",
    "hist, bins = da.histogram(random_values, bins=100, range=[-5, 5]) \n",
    "hist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Trigger the histogram computation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "hist_c = hist.compute()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the histogram:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhYAAAGsCAYAAACB/u5dAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAb9ElEQVR4nO3df5DUdf3A8dd6jKvi3Zok6unxI9IU8EeCQ/grKcMIHbHJUccUf+QMhSbdmIL9UEy/h2MWTQ4k/UCNBMcStPFXWgE1SgMoaVomCkKCP7DaRZpZ8tjvH41XJxywx3tvb4/HY2b/2M991s+LFb3nvPezn0+mVCqVAgAggT2qPQAA0HMICwAgGWEBACQjLACAZIQFAJCMsAAAkhEWAEAywgIASEZYAADJCAsAIJmqhcXixYvjzDPPjMbGxshkMrFgwYKyXn/DDTdEJpPZ6tG7d+/KDAwA7FDVwmLTpk1xzDHHxO23396p11999dWxfv36do/BgwfHOeeck3hSAGBnVS0sxowZEzfddFN89rOf3ebPN2/eHNdcc00ccsgh0bt37xgxYkQsXLiw7ef77rtvHHTQQW2PN954I1544YW47LLLuuhPAAC8X69qD9CRSy65JFavXh3z5s2LxsbGmD9/fnz605+O5557Lg477LCt9v/Rj34Uhx9+eJx88slVmBYAiOimJ2++/PLLMXfu3Ljvvvvi5JNPjkGDBsXVV18dJ510UsyePXur/YvFYvzsZz+zWgEAVdYtVyyefvrpKJVKcfjhh7fbXiwWo0+fPlvtf//998fGjRvjoosu6qoRAYBt6JZhsWXLlqirq4vly5dHXV1du5/tu+++W+3/ox/9KM4444w46KCDumpEAGAbumVYfPSjH43W1tZ48803d3jOxKpVq+K3v/1tPPjgg100HQDQkaqFxTvvvBMrV65se75q1apYsWJF7L///nH44YfHBRdcEBdddFHcdttt8dGPfjQ2bNgQv/nNb+Koo46Kz3zmM22v+8lPfhIHH3xwjBkzphp/DADgf2RKpVKpGgdeuHBhjBo1aqvt48ePjzvvvDP+/e9/x0033RR33313vPbaa9GnT58YOXJkTJ06NY466qiI+M9HJv3794+LLroobr755q7+IwAA71O1sAAAep5u+XVTAKA2CQsAIJkuP3lzy5YtsW7duqivr49MJtPVhwcAOqFUKsXGjRujsbEx9tij43WJLg+LdevWRVNTU1cfFgBIYO3atXHooYd2+PMuD4v6+vqI+M9gDQ0NXX14AKATCoVCNDU1tf0e70iXh8V7H380NDQICwCoMTs6jcHJmwBAMsICAEhGWAAAyQgLACAZYQEAJCMsAIBkhAUAkIywAACSERYAQDJlhcW7774bX//612PgwIGx9957x4c+9KG48cYbY8uWLZWaDwCoIWVd0vuWW26JH/zgB3HXXXfFkCFDYtmyZXHJJZdELpeLq666qlIzAgA1oqyweOqpp+Kss86KsWPHRkTEgAEDYu7cubFs2bKKDAcA1JayPgo56aST4te//nX89a9/jYiIP/7xj/H73/8+PvOZz3T4mmKxGIVCod0DAOiZylqxuPbaayOfz8cRRxwRdXV10draGjfffHOcf/75Hb6mpaUlpk6dusuDAgDdX1lhce+998acOXPinnvuiSFDhsSKFSti0qRJ0djYGOPHj9/ma6ZMmRLNzc1tz9+7nztQ+wZMfmiH+6yeNrYLJgG6i7LC4qtf/WpMnjw5zjvvvIiIOOqoo+LVV1+NlpaWDsMim81GNpvd9UkBgG6vrHMs/vWvf8Uee7R/SV1dna+bAgARUeaKxZlnnhk333xz9OvXL4YMGRLPPPNMfOc734lLL720UvMBADUkUyqVSju788aNG+Mb3/hGzJ8/P958881obGyM888/P775zW/GnnvuuVP/jEKhELlcLvL5fDQ0NHR6cKDr7cw5FTvDeRdQe3b293dZKxb19fUxffr0mD59+q7OBwD0QO4VAgAkIywAgGSEBQCQjLAAAJIRFgBAMmV9KwTYfaT6aunO/LN9/RR6DisWAEAywgIASEZYAADJCAsAIBlhAQAkIywAgGSEBQCQjLAAAJIRFgBAMq68CUREZa+02Zljuxon1CYrFgBAMsICAEhGWAAAyQgLACAZYQEAJCMsAIBkhAUAkIywAACSERYAQDLCAgBIRlgAAMkICwAgGTchg91QNW84trPeP6ObkkFtsGIBACQjLACAZIQFAJCMsAAAkikrLAYMGBCZTGarx8SJEys1HwBQQ8r6VsjSpUujtbW17fmf/vSn+NSnPhXnnHNO8sEAgNpTVlgccMAB7Z5PmzYtBg0aFB//+MeTDgUA1KZOX8di8+bNMWfOnGhubo5MJtPhfsViMYrFYtvzQqHQ2UMCAN1cp0/eXLBgQfzzn/+Miy++eLv7tbS0RC6Xa3s0NTV19pAAQDfX6bD48Y9/HGPGjInGxsbt7jdlypTI5/Ntj7Vr13b2kABAN9epj0JeffXVeOKJJ+L+++/f4b7ZbDay2WxnDgMA1JhOrVjMnj07+vbtG2PHunY/APBfZYfFli1bYvbs2TF+/Pjo1cs9zACA/yo7LJ544olYs2ZNXHrppZWYBwCoYZlSqVTqygMWCoXI5XKRz+ejoaGhKw8Nu61auE16Z7iVOnSdnf397V4hAEAywgIASEZYAADJCAsAIBlhAQAkIywAgGSEBQCQjLAAAJIRFgBAMsICAEhGWAAAyQgLACAZYQEAJCMsAIBkhAUAkIywAACSERYAQDK9qj0AkNaAyQ9Ve4Qu8/4/6+ppY6s0CfAeKxYAQDLCAgBIRlgAAMkICwAgGWEBACQjLACAZIQFAJCMsAAAkhEWAEAywgIASEZYAADJCAsAIBlhAQAkIywAgGSEBQCQTNlh8dprr8XnP//56NOnT+yzzz5x7LHHxvLlyysxGwBQY3qVs/M//vGPOPHEE2PUqFHxyCOPRN++fePll1+O/fbbr0LjAQC1pKywuOWWW6KpqSlmz57dtm3AgAGpZwIAalSmVCqVdnbnwYMHx+mnnx5/+9vfYtGiRXHIIYfEl770pbj88ss7fE2xWIxisdj2vFAoRFNTU+Tz+WhoaNi16YEYMPmhao/Qra2eNrbaI0CPUCgUIpfL7fD3d1nnWLzyyisxc+bMOOyww+Kxxx6LCRMmxJe//OW4++67O3xNS0tL5HK5tkdTU1M5hwQAakhZKxZ77rlnDB8+PJ588sm2bV/+8pdj6dKl8dRTT23zNVYsoLKsWGyfFQtIoyIrFgcffHAMHjy43bYjjzwy1qxZ0+FrstlsNDQ0tHsAAD1TWWFx4oknxosvvthu21//+tfo379/0qEAgNpUVlh85StfiSVLlsT//d//xcqVK+Oee+6JWbNmxcSJEys1HwBQQ8oKi+OPPz7mz58fc+fOjaFDh8a3vvWtmD59elxwwQWVmg8AqCFlXcciIuKMM86IM844oxKzAAA1zr1CAIBkhAUAkIywAACSERYAQDLCAgBIRlgAAMkICwAgGWEBACQjLACAZIQFAJCMsAAAkhEWAEAywgIASKbsu5sC1TNg8kPVHqHmvP89Wz1tbJUmgd2DFQsAIBlhAQAkIywAgGSEBQCQjLAAAJIRFgBAMsICAEhGWAAAyQgLACAZYQEAJCMsAIBkhAUAkIywAACSERYAQDLCAgBIRlgAAMkICwAgGWEBACQjLACAZMoKixtuuCEymUy7x0EHHVSp2QCAGtOr3BcMGTIknnjiibbndXV1SQcC/mvA5IeqPUKPs633dPW0sVWYBHqmssOiV69eVikAgG0q+xyLl156KRobG2PgwIFx3nnnxSuvvLLd/YvFYhQKhXYPAKBnKissRowYEXfffXc89thj8cMf/jBef/31OOGEE+Ltt9/u8DUtLS2Ry+XaHk1NTbs8NADQPWVKpVKpsy/etGlTDBo0KK655ppobm7e5j7FYjGKxWLb80KhEE1NTZHP56OhoaGzh4bdgnMsuoZzLGDHCoVC5HK5Hf7+Lvsci//Vu3fvOOqoo+Kll17qcJ9sNhvZbHZXDgMA1Ihduo5FsViMP//5z3HwwQenmgcAqGFlhcXVV18dixYtilWrVsUf/vCH+NznPheFQiHGjx9fqfkAgBpS1kchf/vb3+L888+PDRs2xAEHHBAf+9jHYsmSJdG/f/9KzQcA1JCywmLevHmVmgMA6AHcKwQASEZYAADJCAsAIBlhAQAkIywAgGSEBQCQjLAAAJIRFgBAMsICAEhGWAAAyQgLACCZsu4VAlTOgMkPVXuE3db73/vV08ZWaRKofVYsAIBkhAUAkIywAACSERYAQDLCAgBIRlgAAMkICwAgGWEBACQjLACAZIQFAJCMsAAAkhEWAEAywgIASEZYAADJCAsAIBlhAQAkIywAgGSEBQCQjLAAAJIRFgBAMr2qPQDsjgZMfqjaI7Ad2/r3s3ra2CpMArXHigUAkMwuhUVLS0tkMpmYNGlSonEAgFrW6bBYunRpzJo1K44++uiU8wAANaxTYfHOO+/EBRdcED/84Q/jAx/4QOqZAIAa1amwmDhxYowdOzZOO+20He5bLBajUCi0ewAAPVPZ3wqZN29ePP3007F06dKd2r+lpSWmTp1a9mAAQO0pa8Vi7dq1cdVVV8WcOXNir7322qnXTJkyJfL5fNtj7dq1nRoUAOj+ylqxWL58ebz55psxbNiwtm2tra2xePHiuP3226NYLEZdXV2712Sz2chms2mmBQC6tbLC4pOf/GQ899xz7bZdcsklccQRR8S11167VVQAALuXssKivr4+hg4d2m5b7969o0+fPlttBwB2P668CQAks8v3Clm4cGGCMQCAnsCKBQCQjLAAAJIRFgBAMsICAEhGWAAAyQgLACAZYQEAJCMsAIBkdvkCWcCODZj8ULVHYBe9/9/h6mljqzQJdG9WLACAZIQFAJCMsAAAkhEWAEAywgIASEZYAADJCAsAIBlhAQAkIywAgGSEBQCQjLAAAJIRFgBAMsICAEhGWAAAyQgLACAZYQEAJCMsAIBkhAUAkIywAACS6VXtAaCnGTD5oWqPQBfY1r/n1dPGVmES6F6sWAAAyQgLACAZYQEAJCMsAIBkygqLmTNnxtFHHx0NDQ3R0NAQI0eOjEceeaRSswEANaassDj00ENj2rRpsWzZsli2bFl84hOfiLPOOiuef/75Ss0HANSQsr5ueuaZZ7Z7fvPNN8fMmTNjyZIlMWTIkKSDAQC1p9PXsWhtbY377rsvNm3aFCNHjuxwv2KxGMVise15oVDo7CEBgG6u7JM3n3vuudh3330jm83GhAkTYv78+TF48OAO929paYlcLtf2aGpq2qWBAYDuq+yw+MhHPhIrVqyIJUuWxBe/+MUYP358vPDCCx3uP2XKlMjn822PtWvX7tLAAED3VfZHIXvuuWd8+MMfjoiI4cOHx9KlS+N73/te3HHHHdvcP5vNRjab3bUpAYCasMvXsSiVSu3OoQAAdl9lrVhcd911MWbMmGhqaoqNGzfGvHnzYuHChfHoo49Waj4AoIaUFRZvvPFGXHjhhbF+/frI5XJx9NFHx6OPPhqf+tSnKjUfAFBDygqLH//4x5WaAwDoAdwrBABIRlgAAMl0+sqbwH8MmPxQtUegm3j/34XV08ZWaRKoHisWAEAywgIASEZYAADJCAsAIBlhAQAkIywAgGSEBQCQjLAAAJIRFgBAMsICAEhGWAAAyQgLACAZYQEAJCMsAIBkhAUAkIywAACS6VXtAaCWDJj8ULVHoIZs6+/L6mljqzAJdB0rFgBAMsICAEhGWAAAyQgLACAZYQEAJCMsAIBkhAUAkIywAACSERYAQDLCAgBIRlgAAMkICwAgGWEBACRTVli0tLTE8ccfH/X19dG3b98YN25cvPjii5WaDQCoMWWFxaJFi2LixImxZMmSePzxx+Pdd9+N0aNHx6ZNmyo1HwBQQzKlUqnU2Re/9dZb0bdv31i0aFGccsopO/WaQqEQuVwu8vl8NDQ0dPbQ0CUGTH6o2iPQw62eNrbaI8BO2dnf37125SD5fD4iIvbff/8O9ykWi1EsFtsNBgD0TJ0+ebNUKkVzc3OcdNJJMXTo0A73a2lpiVwu1/Zoamrq7CEBgG6u02FxxRVXxLPPPhtz587d7n5TpkyJfD7f9li7dm1nDwkAdHOd+ijkyiuvjAcffDAWL14chx566Hb3zWazkc1mOzUcAFBbygqLUqkUV155ZcyfPz8WLlwYAwcOrNRcAEANKissJk6cGPfcc0888MADUV9fH6+//npERORyudh7770rMiAAUDvKOsdi5syZkc/n49RTT42DDz647XHvvfdWaj4AoIaU/VEIAEBH3CsEAEhGWAAAyQgLACAZYQEAJCMsAIBkdukmZNCTuJMp1bCtv3fueEots2IBACQjLACAZIQFAJCMsAAAkhEWAEAywgIASEZYAADJCAsAIBlhAQAkIywAgGSEBQCQjLAAAJJxEzJ2W246Rnf1/r+bbkpGLbFiAQAkIywAgGSEBQCQjLAAAJIRFgBAMsICAEhGWAAAyQgLACAZYQEAJCMsAIBkXNKb3YLLd1PLtvX312W+6a6sWAAAyQgLACAZYQEAJFN2WCxevDjOPPPMaGxsjEwmEwsWLKjAWABALSo7LDZt2hTHHHNM3H777ZWYBwCoYWV/K2TMmDExZsyYSswCANS4in/dtFgsRrFYbHteKBQqfUgAoEoqfvJmS0tL5HK5tkdTU1OlDwkAVEmmVCqVOv3iTCbmz58f48aN63Cfba1YNDU1RT6fj4aGhs4eGrbLBbHY3bhgFpVWKBQil8vt8Pd3xT8KyWazkc1mK30YAKAbcB0LACCZslcs3nnnnVi5cmXb81WrVsWKFSti//33j379+iUdDgCoLWWHxbJly2LUqFFtz5ubmyMiYvz48XHnnXcmGwwAqD1lh8Wpp54au3C+JwDQgznHAgBIRlgAAMlU/OumUGmuWQHb/u/AtS2oBisWAEAywgIASEZYAADJCAsAIBlhAQAkIywAgGR83ZSa4+ulsHPe/9+Kr5/SFaxYAADJCAsAIBlhAQAkIywAgGSEBQCQjG+F0K35Bgik40ZldAUrFgBAMsICAEhGWAAAyQgLACAZJ2/SrThZE7qWy36TmhULACAZYQEAJCMsAIBknGNB1TifArofF9FiV1mxAACSERYAQDI+CqHL+OgDapOvpFIOKxYAQDJWLKgIqxPQcznBk+2xYgEAJGPFgiSsUMDuzXkYvMeKBQCQTKdWLGbMmBG33nprrF+/PoYMGRLTp0+Pk08+OfVsdFNWJ4AdcR7G7qvssLj33ntj0qRJMWPGjDjxxBPjjjvuiDFjxsQLL7wQ/fr1q8SMVJmQAFLwccnuIVMqlUrlvGDEiBFx3HHHxcyZM9u2HXnkkTFu3LhoaWnZ4esLhULkcrnI5/PR0NBQ/sQkJRqA7kx8dB87+/u7rBWLzZs3x/Lly2Py5Mntto8ePTqefPLJbb6mWCxGsVhse57P59sGpLKGXv9YtUcA2CX9vnLfDvf509TTu2AS3vu9vaP1iLLCYsOGDdHa2hoHHnhgu+0HHnhgvP7669t8TUtLS0ydOnWr7U1NTeUcGgC2KTe92hPsXjZu3Bi5XK7Dn3fq5M1MJtPuealU2mrbe6ZMmRLNzc1tz7ds2RJ///vfo0+fPh2+ZndRKBSiqakp1q5d62OhCvI+dx3vddfwPncN73N7pVIpNm7cGI2Njdvdr6yw+OAHPxh1dXVbrU68+eabW61ivCebzUY2m223bb/99ivnsD1eQ0ODv7RdwPvcdbzXXcP73DW8z/+1vZWK95R1HYs999wzhg0bFo8//ni77Y8//niccMIJ5U0HAPQ4ZX8U0tzcHBdeeGEMHz48Ro4cGbNmzYo1a9bEhAkTKjEfAFBDyg6Lc889N95+++248cYbY/369TF06NB4+OGHo3///pWYr0fLZrNx/fXXb/VREWl5n7uO97preJ+7hve5c8q+jgUAQEfcKwQASEZYAADJCAsAIBlhAQAkIyy6mWKxGMcee2xkMplYsWJFtcfpcVavXh2XXXZZDBw4MPbee+8YNGhQXH/99bF58+Zqj1bzZsyYEQMHDoy99torhg0bFr/73e+qPVKP0tLSEscff3zU19dH3759Y9y4cfHiiy9We6wer6WlJTKZTEyaNKnao9QMYdHNXHPNNTu8XCqd95e//CW2bNkSd9xxRzz//PPx3e9+N37wgx/EddddV+3Ratq9994bkyZNiq997WvxzDPPxMknnxxjxoyJNWvWVHu0HmPRokUxceLEWLJkSTz++OPx7rvvxujRo2PTpk3VHq3HWrp0acyaNSuOPvroao9SU3zdtBt55JFHorm5OX7xi1/EkCFD4plnnoljjz222mP1eLfeemvMnDkzXnnllWqPUrNGjBgRxx13XMycObNt25FHHhnjxo2LlpaWKk7Wc7311lvRt2/fWLRoUZxyyinVHqfHeeedd+K4446LGTNmxE033RTHHntsTJ8+vdpj1QQrFt3EG2+8EZdffnn89Kc/jX322afa4+xW8vl87L///tUeo2Zt3rw5li9fHqNHj263ffTo0fHkk09WaaqeL5/PR0T4u1shEydOjLFjx8Zpp51W7VFqTqfubkpapVIpLr744pgwYUIMHz48Vq9eXe2Rdhsvv/xyfP/734/bbrut2qPUrA0bNkRra+tWNyI88MADt7phIWmUSqVobm6Ok046KYYOHVrtcXqcefPmxdNPPx1Lly6t9ig1yYpFBd1www2RyWS2+1i2bFl8//vfj0KhEFOmTKn2yDVrZ9/r/7Vu3br49Kc/Heecc0584QtfqNLkPUcmk2n3vFQqbbWNNK644op49tlnY+7cudUepcdZu3ZtXHXVVTFnzpzYa6+9qj1OTXKORQVt2LAhNmzYsN19BgwYEOedd1788pe/bPc/4dbW1qirq4sLLrgg7rrrrkqPWvN29r1+738U69ati1GjRsWIESPizjvvjD320NidtXnz5thnn33ivvvui7PPPrtt+1VXXRUrVqyIRYsWVXG6nufKK6+MBQsWxOLFi2PgwIHVHqfHWbBgQZx99tlRV1fXtq21tTUymUzsscceUSwW2/2MrQmLbmDNmjVRKBTanq9bty5OP/30+PnPfx4jRoyIQw89tIrT9TyvvfZajBo1KoYNGxZz5szxP4kERowYEcOGDYsZM2a0bRs8eHCcddZZTt5MpFQqxZVXXhnz58+PhQsXxmGHHVbtkXqkjRs3xquvvtpu2yWXXBJHHHFEXHvttT562gnOsegG+vXr1+75vvvuGxERgwYNEhWJrVu3Lk499dTo169ffPvb34633nqr7WcHHXRQFSerbc3NzXHhhRfG8OHDY+TIkTFr1qxYs2ZNTJgwodqj9RgTJ06Me+65Jx544IGor69vO38ll8vF3nvvXeXpeo76+vqt4qF3797Rp08fUbGThAW7lV/96lexcuXKWLly5VbRZvGu884999x4++2348Ybb4z169fH0KFD4+GHH47+/ftXe7Qe472v8p566qntts+ePTsuvvjirh8IOuCjEAAgGWesAQDJCAsAIBlhAQAkIywAgGSEBQCQjLAAAJIRFgBAMsICAEhGWAAAyQgLACAZYQEAJCMsAIBk/h8pitI0F4dghAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = 0.5 * (bins[1:] + bins[:-1])\n",
    "width = np.diff(bins)\n",
    "plt.bar(x, hist_c, width);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}