Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add notebook that shows how to use skorch with optuna #724

Closed
wants to merge 8 commits into from
Closed

Add notebook that shows how to use skorch with optuna #724

wants to merge 8 commits into from

Conversation

BenjaminBossan
Copy link
Collaborator

@BenjaminBossan BenjaminBossan commented Nov 21, 2020 •
edited
Loading

Fixes #718

This notebook shows how to use skorch in conjunction with optuna, which can be much more efficient than GridSearchCV or RandomizedSearchCV. I don't have a deep knowledge of optuna, so there may be room for improvement. The whole example is inspired by this one.

Right now, the implementation makes use of the skorch-internal train/valid split. This allows us to use optuna with very few lines of extra code.

Right now, this notebook contains an ad hoc implementation of an OptunaCallback, which is responsible for logging and pruning trials. Possibly, this could be extended for more functionality. If so, it could find it's way into the skorch code base.

Eventually, we could consider adding a section to the docs about hyper-parameter optimization using different libraries. WDYT?

@BenjaminBossan BenjaminBossan self-assigned this Nov 21, 2020
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

Use optuna's SkorchPruningCallback in notebook
ee3efde 
Optuna already provides a skorch callback, which is essentially the
same that I also implemented.
@BenjaminBossan
Copy link
Collaborator Author

Update: optuna already provides a skorch callback, which is almost exactly the same that I used. I changed the notebook to use this instead.

@BenjaminBossan BenjaminBossan mentioned this pull request Nov 22, 2020
Closed
thomasjpfan
thomasjpfan reviewed Nov 25, 2020
View reviewed changes
Copy link
Member

@thomasjpfan thomasjpfan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's nice to see how easy it is to integrate with optuna.

notebooks/optuna-example.ipynb Outdated
Comment on lines 174 to 182
" \"\"\"Model definition and hyper-parameter setting\n",
"\n",
" We set the number of units, the optimizer, the learning rate, and the number of epochs.\n",
" \n",
" To make this work efficiently with optuna, add the\n",
" `SkorchPruningCallback <https://optuna.readthedocs.io/en/stable/reference/generated/optuna.integration.OptunaSearchCV.html?highlight=optuna.integration.OptunaSearchCV>`\n",
" from the optuna library.\n",
"\n",
" \"\"\"\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given this is a notebook, can we pull this out into an markdown block above this code block?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

notebooks/optuna-example.ipynb Outdated
"outputs": [],
"source": [
"def objective(trial):\n",
" \"\"\"Train the model and return the value to be maximized\"\"\"\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A markdown description before defining this function?

Next, we define an objective function that trains a NeuralNetClassifier and returns the value to be maximized.

Copy link
Collaborator Author

@BenjaminBossan BenjaminBossan Nov 28, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

notebooks/optuna-example.ipynb
" model = define_model(trial)\n",
" X, y = get_data()\n",
" model.fit(X, y)\n",
" accuracy = model.history[-1, METRIC_NAME]\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be cleaner to directly place the metric name here?

Suggested change
" accuracy = model.history[-1, METRIC_NAME]\n",
" accuracy = model.history[-1, 'valid_acc']\n",

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would leave it as is. The idea is to set all the constants at the beginning of the notebook. This way, if I want to change the them (say, optimizing valid_loss instead), I don't need to chase through the notebook to replace all instances.

Add improvements to notebook, reviewer feedback
2127c77 
- add more markdown descriptions
- set number of trials as constant
- increase image size and change order
@BenjaminBossan
Copy link
Collaborator Author

@thomasjpfan I improved the notebook by adding more descriptions, as you correctly pointed out. Now it should be more accessible for users who don't already know optuna. Furthermore, I increased the image size, which was quite small originally.

thomasjpfan
thomasjpfan approved these changes Dec 4, 2020
View reviewed changes
Copy link
Member

@thomasjpfan thomasjpfan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise LGTM

notebooks/optuna-example.ipynb Outdated Show resolved Hide resolved
notebooks/optuna-example.ipynb Outdated Show resolved Hide resolved
notebooks/optuna-example.ipynb Outdated
Comment on lines 493 to 526
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAAGHCAYAAACOFvWHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdeVxU9f4/8NeZGbZhAJkZBFncUHHJXMLlKioIbplK30yvLd9Mr1lZWZk3t1wyS01LM/Wm8tVuy/1a/TIrywXzImpu9cU1FUxTVllkZxiYc35/EHMdWRyWWXk9Hw8eeT5zPufzPjMf6M3hfT5HkCRJAhERERERNSuZrQMgIiIiInJGTLSJiIiIiCyAiTYRERERkQUw0SYiIiIisgAm2kREREREFsBEm4iIiIjIAphoExHdJTIyEn/729/s5jj2Mk5D7NixAwqFwtZhNLupU6ciJibG1mEQkYNgok1EDiUrKwsvvvgi2rdvD1dXV/j5+WHixIlISkpq8LHeeusttG/fvkb7119/jffee6/JsTbXcapZOt57uX79OgRBwJEjR2q8tnTpUnTq1Mm4PXnyZKSlpZl97JiYGEydOrU5wmy0f//73xAEwfil0WgQFRWFxMTEJh23U6dOWLp0afMESUQOhYk2ETmMmzdvIjw8HMeOHcPmzZuRkpKCPXv2wMXFBQMHDsTevXubZRy1Wg1vb2+7OY69jNMQHh4e8Pf3t/q4kiShoqKiScf49ddfkZGRgYMHD8LDwwNjxozB9evXmydAImpRmGgTkcOYNWsWKioqcOjQIYwZMwZt27ZF//798a9//QvDhw/H1KlTUVZWBuA/V1g///xzdOzYEe7u7oiJicG1a9cAVJU2vPHGG/jjjz+MVzCrrzreXYoRGRmJ6dOnY9GiRWjdujVatWqFhQsXQhRFvPnmm/D394efnx8WLlxoEu+dx7n7amn1V/UVakmSMGPGDISGhsLDwwMdO3bEggULUF5e3uB4KyoqMG/ePAQFBcHV1RXdu3fH559/bhKbIAjYtGkTnnzySXh5eSEkJASrV69ung8KNUtHCgsL8fTTTyMgIABubm4ICQnBq6++CqCqHOPgwYP4+OOPjef273//GwBw+fJljB07FiqVCiqVCuPGjUNKSkqNcQ4dOoQ+ffrAzc0Nmzdvhkwmw7Fjx0xiSkhIgEwmw++//15v7H5+fggICEDv3r2xdetWlJSUYN++fbXuK0kS1qxZg44dO8LV1RWhoaFYt26d8fXIyEhcvXoVy5YtM54bk3ailoOJNhE5hNu3b2PPnj144YUXar16O3/+fGRlZeHAgQPGtoyMDGzatAk7d+5EYmIiioqKEBsbC0mSMHnyZLz++usIDg5GRkYGMjIy8Nprr9U5/ldffYWKigocOXIE7733Ht5++2089NBDKC4uRmJiItasWYO3334bP/74Y639Bw0aZBwnIyMDFy5cQGBgIKKiogBUJWz+/v74/PPP8dtvv2HdunXYvn073n77bQBoULwLFizA1q1bsW7dOpw/fx5PPPEEnnjiCRw8eNBkv2XLlmHo0KFISkrC3Llz8frrr+PQoUP1fxCNtGjRIvz666/YvXs3kpOTsXPnTnTr1g0AsH79egwZMgSTJk0yntugQYNQVlaGkSNHQqfTISEhAQkJCSguLsbo0aOh1+uNxxZFEX//+9+xdu1aXLp0CVOmTMGIESOwdetWkxi2bduG6OhodOzY0ey4PTw8AKDOq+SbNm3CG2+8gXnz5uHChQuYO3cu5s2bh7i4OABVZT3t27fHnDlzjOcWEhLSoPeOiByYRETkAE6cOCEBkL7++utaX8/NzZUASKtXr5YkSZKWLFkiAZCSk5ON+1y+fFkCIB04cECSJElavny51K5duxrHGjZsmDR9+nST7V69epns0717d+m+++4zabv//vulOXPm1Hmcanq9XoqMjJQiIiIknU5X5zm/9957UqdOnYzb5sRbUlIiubq6Shs3bjTZJzY2VoqKijJuA5BefPFFk33CwsKkefPm1RnPtWvXJACSh4eH5OnpafLl4uIihYaGGvfdvn27JJfLjdvjx4+XnnrqqTqPHR0dXeP1bdu2SR4eHlJ2draxLTMzU3J3d5c+/vhj4zgApMOHD5v0/X//7/9JSqVSys/PlyRJkm7fvi15eHhIX3zxRZ0xHDp0SAIg3bx5U5IkSSosLJT+9re/SQqFQjp37pwkSZL01FNPSdHR0cY+wcHB0ty5c02O8/LLL0sdOnQwboeGhkpLliypc1wicl68ok1EDkGSpHpfFwShRpufn5/JDXpdunSBVqvFxYsXGzx+r169TLYDAgJw//3312i7devWPY/13HPP4ebNm9i1axfc3NyM7Vu3bsWAAQPg7+8PlUqF+fPn448//mhQnCkpKdDr9Rg6dKhJ+7Bhw3DhwgWTtt69e5tsBwUFISsr655jbN++HUlJSSZfzz77bL19nn/+eXz11Ve47777MHv2bPz4448QRbHePhcuXED37t2h1WqNbf7+/ggLC6txLv369TPZHj9+PHx8fIwlM59++ilUKhUmTJhwz/MLCwuDSqWCj48P9u3bh3/+85+47777auxXWFiI1NTUWt/r69evo7S09J5jEZFzY6JNRA6hc+fOkMlkOH/+fK2vV7eHhYXVe5x7Jex1cXFxMdkWBKHWtnslj6tXr8bXX3+NPXv2mCSQX375JWbNmoXJkyfjhx9+wP/93/9h8eLFjb6x7+5fPCRJqtHm6ura4PiBqoS8U6dOJl9qtbrePqNGjcKNGzewcOFC6HQ6PPHEExg+fDgMBkODzqO2c5HL5XB3dzfZR6FQYPr06cbykW3btmHq1Kk1zrk2+/btw5kzZ5CTk4MbN25gypQpDYqxsXOMiJwPE20icghqtRpjxozBxo0bUVhYWOP1t99+G/7+/hgxYoSxLTs7G1evXjVuX7lyBbm5ucbaYFdX13smes3pm2++weLFi/H111/X+IXg8OHD6NOnD1599VU88MAD6Ny5c42b5syJt1OnTnBzc0NCQkKN4/fo0aNZzqOx1Go1pkyZgo8++gh79uxBQkKC8a8LtZ1bjx49cOHCBeTk5BjbsrKycOXKFbPOZcaMGThz5gz+8Y9/4MyZM2avNd6+fXuEhobe85cHb29vBAcH1/ped+jQAUqlss5zI6KWgYk2ETmMjRs3Qi6XY/jw4di7dy9u3ryJU6dO4bHHHsOhQ4ewY8cO481rAKBUKvH000/jl19+wenTp/HUU0+hZ8+exgeOdOjQAZmZmfj555+Rk5Nj0T/1X7hwAU888QSWLl2Krl27IjMzE5mZmcjOzgZQdSX+3Llz2L17N65evYr169fj66+/NjmGOfEqlUq89NJLeOONN/Dll18iOTkZb7/9Nnbv3o0FCxZY7PzuZeHChfj6669x+fJlJCcn47PPPoNKpULbtm0BVJ3bL7/8gqtXryInJwcVFRV47LHH4Ofnh8mTJ+PXX3/FL7/8gr/+9a8ICgrC5MmT7zlm27ZtMXr0aMyePRuRkZHo0qVLs5/X/PnzsWHDBmzduhXJycn46KOPsHnzZpP3ukOHDjh69Chu3LiBnJwcs/5qQETOgYk2ETmMdu3a4fTp0xgwYABmzpyJ0NBQjBkzBuXl5fj5558xevRok/3btGmDZ555Bo888ggGDx4MDw8P7Nq1y/in/tjYWDz66KMYO3Ys/Pz8mnV5u7udOnUKJSUlmD9/Ptq0aWP8qq4tnjlzJp588kk8/fTT6NOnD06cOFHjISfmxrtixQrMmDEDL7/8Mnr06IFPP/0Un376KaKjoy12fvfi7u6OxYsX44EHHkB4eDjOnj2LH3/8ET4+PgCAOXPmQKvVolevXvDz88PRo0fh4eGB/fv3w83NDUOHDsWwYcPg6emJvXv3mlUCAgDPPPMM9Ho9nnnmGYuc13PPPYc333wTb7/9Nrp3745Vq1Zh5cqVmD59unGfZcuWoaCgAGFhYfDz88ONGzcsEgsR2R9BYjEZETmhpUuX4tNPPzVZc5lank2bNmHx4sVIS0szufGUiMgaFPfehYiIyLEUFxcjJSUFa9aswQsvvMAkm4hsgqUjRETkdF544QX0798f3bp1w+uvv27rcIiohWLpCBERERGRBfCKNhERERGRBTDRJiIiIiKyACbaREREREQW4NSrjqSnp1t9TK1Wa/IUM2qZOA+Ic4AAzgPiHGgJAgMD63yNV7SJiIiIiCyAiTYRERERkQUw0SYiIiIisgCnrtG+myRJ0Ol0EEURgiBYZIysrCyUl5db5NgtnSRJkMlkcHd3t9jnR0RERNRcrJZoJyUlYfv27RBFEdHR0YiNjTV5vbi4GJs3b0ZWVhZcXFzw3HPPoW3btgCAWbNmwd3dHTKZDHK5HCtXrmxUDDqdDi4uLlAoLHfaCoUCcrncYsdv6SorK6HT6eDh4WHrUIiIiIjqZZVEWxRFxMXFYdGiRdBoNJg/fz7Cw8MRHBxs3GfXrl1o37495s6di7S0NMTFxWHx4sXG15csWQJvb+8mx2HJJJssT6FQ8C8GRERE5BCsUqOdkpKCgIAA+Pv7Q6FQYNCgQTh16pTJPqmpqejZsycAICgoCNnZ2cjPz2/WOFhu4Bz4ORIREZEjsEqinZeXB41GY9zWaDTIy8sz2addu3Y4ceIEgKrEPDs722SfFStW4PXXX0d8fLw1QraYkJAQjBgxAjExMRg1alSNXzjMtXXrVpSVldVoX7t2Ld555x2TtvPnz2PYsGH1Hm/ixIk4c+ZMo2IhIiIiopqsUkchSVKNtruvSsbGxmLHjh2YO3cu2rZtiw4dOkAmq/o9YPny5VCr1SgoKMBbb72FwMBAdO/evcYx4+PjjYn4ypUrodVqTV7PyspqUOlIRcpVVBw+DDErCzJ/f7gMHQqXTqH37FffGO7u7jh06BAA4NChQ1i1ahW++eYbs2OqFhcXh0mTJsHLy8uk/ZFHHsGUKVPwxhtvGNu+++47PPLII/XGJQgC5HK5Q5TWuLm51fhs7Y1CobD7GMmyOAcI4DwgzoGWzipZlUajQW5urnE7NzcXvr6+JvsolUo8//zzAKoS8xdeeAGtW7cGAKjVagCAj48P+vXrh5SUlFoT7ZiYGMTExBi3734SU3l5udk3KlZevQr9F19CUKkAjQaGggJU/utfMEx6FIrQupNthUKBysrK+o/95+v5+fnw9vY2bm/evBnfffcd9Ho9Ro8ejddeew2lpaWYOXMmMjIyIIoiZs+ejZycHGRmZuK//uu/4Ovri6+++sp47Pbt28Pb2xsnT55E3759AQDffvstPvvsM1RWVmLevHk4c+YMdDodxo4di9deew1A1XtuMBhQWVmJzp07Izk5GQDw/fffIz4+HuvWrUNubi7mzZuHtLQ0AMCyZcvQr18/s97P5lReXm73T9nik8CIc4AAzgPiHGgJ6nsypFUS7dDQUGRkZODWrVtQq9U4duwYXnrpJZN9SkpK4ObmBoVCgYMHD6Jbt25QKpXQ6XSQJAkeHh7Q6XQ4e/YsJk6c2OSYKk6ehHRX+YrJ60eOQtLpIJSUGNsknQ7l23dAjBhcax9BrYZi0KB6x9XpdBgxYgTKy8tx69YtfPHFFwCAhIQEXLt2DXv27IEkSZg6dSqOHz+O3NxcBAQE4JNPPgEAFBYWwtvbG1u2bMGXX35p/CXkTrGxsdi9ezf69u2LX375Bb6+vujYsSMA4PXXX4evry8MBgMmT56Mixcv1vpLS20WL16MGTNmoH///khLS8Njjz2GhIQEs/oSERERtTRWSbTlcjmmTZuGFStWQBRFREVFISQkBPv37wcAjBw5Emlpafjwww8hk8kQHByMZ599FgBQUFCANWvWAAAMBgMiIiLQu3dvi8csFRYCd5VlwM2tqr0J3N3dceDAAQDA6dOnMXv2bPz0009ISEhAQkICRo4cCQAoLS3FtWvX0L9/fyxfvhwrVqxATEwMBgwYcM8xxo8fjwkTJmDJkiXYvXs3JkyYYHztu+++w2effQaDwYCsrCwkJyebnWgnJibiypUrxu3i4mIUFxdDpVI15C0gIiIiahGsVpDbt29fYylDteqkEgC6dOmCDz74oEY/f39/vPvuu80ej0v//vW+LmZmQSoqgnBHsi0VFUHo3Bmuo0c3Swzh4eHIy8tDbm6usVzmySefrLHfjz/+iJ9++gnvvPMOhg0bhldeeaXe4wYFBSEkJAQ///wzfvjhB3z77bcAgBs3buCjjz7Cnj170KpVK7z88svQ6XQ1+t9ZP3/nUnqiKOLbb7/lGtZEREREZuAj2OugGBIBqbgYUlERJFGs+m9xMRRDIpptjJSUFBgMBvj6+iIyMhI7d+5EyZ+lKhkZGcZabA8PDzzyyCN49tlnce7cOQCASqVCcXFxnceeMGECli5divbt2xtrh4qKiuDh4QFvb29kZ2cbb8q8m5+fH5KTkyGKIvbu3WtsHzZsGHbs2GHcPn/+fFPfAiIiIiKnZf9LTNiIIjQUmPQoKhOP/GfVkQfH1HsjpDmqa7SBqhsQ161bB7lcjmHDhiE5ORnjx48HUHVz6IYNG3D9+nW89dZbEAQBLi4uxqX7Hn/8cTzxxBNo3bq1yc2Q1caNG4clS5Zg+fLlxrYePXrgvvvuQ1RUFNq2bVvnjYzz58/HU089hcDAQISFhRmT/+XLl2PBggWIiYlBZWUlBgwYgFWrVjXp/SAiIiJyVoJU29p7TiI9Pd1ku7S0FEql0qJjmrPqCDWNNT7HpuJd5sQ5QADnAXEOtAT1rTrC0hEiIiIiIgtg6QgREdUp48wlnPlqH1xzs6HX+KHXxFFo06trvX0qr141KbtTDIloctmdLTnT+VjrXJxtHGtpzPk423vQGPb8HsiXLl261NZBWEpRUZHJdkVFBVxcXCw6pkwmgyiKFh2jpbPG59hUSqUSpaWltg6DbMgZ5kDGmUtI2rADWUUVyJApUV5QhOKTv0DVLhheAbU/6a76YV8AILRqBRQXw/B/SRAC20BWy7r/9q6p52NP88Ban42zjdNU5s6BxpyPo7wHlmQP78HdT+m+E2u0mxlrtC2PNdrkCJxhDux9Yz1y0rKhLi+CW6UeAOBqqICXtxLdxgyttU/FqVOQysshuLkZ26q3XWzwJNmmaur5eKo8UVJccs/9rMFan42zjdNU5s6BxpyPo7wHlnTneyAL8IdMrTEuz+w+9SmrxGDzJ0MSEZHjcc/OhH9JAVylShS4qiAJACR3KCvLIQtsU3sngwGCRm2yHj9UnkBBYd197FkTz8fF2xuyJj7orNlY67NxtnGayOw50JjzcZD3wKLueA8EF9eqNk9PiFlZto3rT0y0iYioBrGwEK0qSlBWqcNl3xCUuFQ9qMpTXwa3QDVcBg+utZ8hOaX2h321bVdnH3vW1PPx1GpRZid/2bDWZ+Ns4zSVuXOgMefjKO+BJdX2HqCkBDJ/f9sFdQeuOmJl6enpePrppzF48GAMGjQIixcvhl5f9SfZnTt3YuHChbX2q15fu6H27t1r8tj0d999F4cPH27Usart3LkTzz//vElbXl4eevbsafIkybu9/PLL+P7775s0NhFZnpiXh4of9yK4RyeUePkCEiBIEjz1ZfAXytFr4qg6+1rjYV/W5EznY61zcbZxrKUx5+Ns70Fj2Pt7wETbiiRJwowZMzB69GgcPXoUiYmJKCkpMeuhL9WPUW+ouxPtuXPnYujQ2msrzfXggw/i8OHDKCsrM7Z9//33GDlyJNzuqBMjIscjZmejYt8+QCZAPe2/0eOVGQgIVKOrrAQBgWr0fnFqvauOKEJD4TrpUQheXpCysyF4ecF10qN2swJAQznT+VjrXJxtHGtpzPk423vQGPb+HrB0pB7pBeXYcjwDOcUV0Kpc8MzANgj0aXwieeTIEbi5uWHy5MkAALlcjqVLl2LgwIF47bXXqsZMT8fjjz+OGzdu4OGHH8arr74KAOjcuTOSk5MBAJs3b8Z3330HvV6P0aNHG/t++eWX+OijjwAA3bp1w3//93/jwIEDOH78ONavX4+tW7di3bp1iImJgVKpxM6dO437Hzt2DB999BE+/vhjJCQkYM2aNdDr9WjXrh3ef/99eHp6Gs/Dy8sLAwcOxP79+zFhwgQAVb8IvPTSSwCA999/HwcOHIBOp0N4eDhWrVplWj8GYMCAAfjxxx+hVqtx5swZLF++HF999RVKS0uxaNEiXLp0CZWVlZgzZw5Gjar76hkRNR9DWjoqDh2CoPSA68iREFQqtGnV6p7L+d1NERpqN/+Taw7OdD7WOhdnG8daGnM+zvYeNIY9vwdMtOuQXlCO2btSkFaoN7ZdyCjB+oc7NTrZvnLlCnr27GnS5uXlhaCgIFy7dg0AkJSUhIMHD8LDwwNjx45FdHQ0evXqZdw/ISEB165dw549eyBJEqZOnYrjx4/D19cXH3zwAXbv3g21Wo3bt2/D19cXI0aMQExMDB566CGTcYcOHYrXX3/duILHt99+i/HjxyMvLw/r16/Hzp07oVQqsXHjRmzZsgWvvPKKSf8JEybgm2++wYQJE5CZmYnff/8dg/+sB5s6dapx/xdffBEHDhzAyJEjzXqP1q9fj8GDB+O9995DQUEBxo4diyFDhtj9KiNEjs7wxx+oOHwYgrcPXEeOgODhYeuQiFqcxlzga+6LgtS8mGjXYcvxDJMkGwDSCvXYcjwDS0e1b9QxJUmqcWX37vYhQ4ZA/ee6j2PGjMHJkydrJNoJCQnGxLW0tBTXrl3DxYsXMXbsWGNfX1/femNRKBSIiorCgQMHMHbsWBw8eBCLFi3Czz//jCtXrhivVFdUVOCBBx6o0T8mJgYLFixAUVERvvvuO4wdOxZyuRxA1dXxzZs3o6ysDPn5+QgLCzM70T58+DAOHDiAf/zjHwCA8vJypKWloXPnzmb1J6KGMyQno+LYz5D5aeESEwPB1dXWIRG1OI25wGeJi4LUvJho1yGnuKL29pLa283RpUsX/PDDDyZtRUVFSE9PR/v27XH27Nkaifjd25Ik4YUXXsCTTz5p0h4XF1drEl+fcePG4eOPP0arVq3Qu3dvqFQqSJKEoUOHYtOmTfX29fDwQGRkJH788Ufs3r0b1c890ul0WLBgAX744QcEBQVh7dq1td4gqVAojA/2ufN1SZKwZcsWdOrUqUHnQkSNU3nhAipPnYYsKBAukZEQ7PxhUETOqjEX+CxxUZCaF2+GrINWVfv/bLSejf+f0JAhQ1BWVoYvv6x6gpHBYMCbb76JSZMmwePPP9MmJibi9u3bKCsrw759+9DvrgXnIyMjsXPnTpSUVC1+n5GRgZycHEREROC7775DXl4eAOD27dsAAJVKZdz3boMGDcK5c+fw2WefYdy4cQCABx54AKdOnTKWspSVleHq1au19o+NjcWWLVuQk5NjvOpdnTSr1WqUlJRgz549tfYNDg7G2bNnAcBkn2HDhmH79u2ofo7S+fPna38ziahJJElC5a//h8pTpyFv3x4uw4czySayocZc4LPERUFqXryiXYdnBrbBhYwSk98Ug7xd8czAxi8ALwgCtm3bhgULFmDdunWQJAnDhw/HvHnzjPv069cPL730Eq5fv46HH37YWDZSfbV62LBhSE5ONi73p1QqsWHDBoSFheGll17CxIkTIZPJcN9992HdunWYMGEC5s6di7i4OGzZssUkHrlcjpiYGHzxxRdYv349AECj0eD999/HrFmzjMsO/v3vf0doLTcZDBs2DC+//DKmTJlijM/HxwePPfYYYmJiEBwcbFL2cqdXX30Vc+bMwYYNG9CnTx9j+8svv4wlS5YgJiYGkiQhODgY//znPxv1ft8p48wlnPlqH1xzs6HX+KHXxFENvsHLkVVevYrKxCMQs7Ig8/eHYkiE3d44Yk8a875V97lVUAC9j0+D+lj687lzHKmiAlAq4TpgABR/GQhBxusudWENLFlDYy7wWeKiIDUvPoK9vv7VP1xLKqD1NO+HqyUewZ6Xl4fRo0fj5MmTzXpcR9XQzzHjzCUkbdiBLMkNpS7uUFbo4C+U33OZsqawp8dvV169Cv0XX0JQqQBPT6CkBFJxsV0tf2SPGvO+3dlH6eeH0uzsBvWx5OdTPQ48PSHm5UG6dQuCUgm3Z5+FSyfOg7rUVgMb5O1qdg2sPf0sINswdw40Zq41dX5S86jvEexMtJtZcyfamZmZmDhxIqZNm4Zp06Y123EdWUM+R0mScHDeauhSM6CQDJBJVXXhbgY9vL1V6D5+uEViVKk8UVxce8mOtVX8/DMknQ6Cu7uxrXrb5S9/sWFk9q0x79udfVxdXaHX6xvUx9xxmnQ+ri5AuR6ywDYQPJQQvLzgPvWpZhvH2Szddx37L9+u0T4yzNesGlgm2tSQOdCYC3yN6UPNq75Em6Ujdi4gIABHjhyxdRgORRJFSLeyYbh5A+KNGwi6dhE5ogtKXdxRIav6c1qFoIC3rgzCHeuDNyeZyguCvfwKW1YGwdvb9GZZFxegsNBi5+8UGvO+3dFHcHWF0MA+Zo/TGHfGFhAAma8akihCzMpqvjGcEGtgyZoCfdwafBNjY/qQ9TDRJodTmleAw+9sNam3DujRCWJ6OsQbNyCmpkLSlQNyGWRt2iAnqCNuFOiR7+ZlPIanvgzyQDX6RFvmiraXVotyO7mKJd68CamoCILXf85fKiqC0LEjXC10/s6gMe+bePMminLz8Us+UF4MuCk88UArwOsefazx+dQ2DkpKIPP3b7YxnBFrYImoKVrU3S9OXCXTYuhLy3D7Ugoy0/OQUukO3bUbuPnOWuS+vwEVPx2C4cZNyAKD4DJsGNwmT4ZrTAw6TZ0MtcwAT30ZBEmCp74M/kI5ek1sGU+cVAyJgFRcDKmoqOpqf1ERpOJiKIZE2Do0u9aY9+12nwE4/VsGMtPzkHa7FJnpeTj9WwZu9xnQrONY63yo6sb4IG/TdcWbemM8EbUc8qXVCyA7oaKiIpNtg8EAQRAgs+Dd9TKZzLg+NDW//IxsZJy5BNnlywgsyYHSUA6DKKGgEmj79BQoBgyAvEN7yHxbQfjzATpeAVqo2gWj5MLybLEAACAASURBVGY6/HX58PDXovfTEy266ohSqURpaanFjt8QMrUaQmAbSJlZkG7dgkythsuDY3gj5D005n1be6YYh0o8EFCah9a6fOS7e+H7kH647hOIyE6tmm0ca50PAV7uCkR08EG+rhI+Hgr0bOOJRSPamV0Da08/C8g2OAecn9edfym8S4sqHXF3d4dOp0N5eXmDH+5iLjc3t1of0EJNI1VUQCosROH+Q9DtS4Sr3AW3lL4odPWETu6KrlIJZPXcjNCmV9cWtZzf3RShoUyoGqGh71tOcQWu+wTiuo/pXFTfo57XWp8P50HjsAaWiBqrRSXagiAYHwxjKbzDvPlIkgQxPQOGy5cg3kwFAOScPocsD19keaqN+3nqy6AP9LNVmERGrOclqsK1x4mqtKhEmxyDVF4OQ0oKDJcvQyosguDuDkXP+yDv0gXBvfsjZ8MOeOrLTNbEbin11mTfLPGgKyJHU9vazhcySri2M7VITLTJbog5OTBcvgzD79cAgwEy/9ZQ9O4NWbt2xnrrNr26Ai9O/c9THgNb3lMeyX4F+rhh/cOdsOV4Bgr0gI8reCWPWpwtxzNMkmwASCvUY8vxDJbgUIvDRJtsSqqshHj9OgyXLkPMyQFcFJB3CoU8LAwytbrWPi293prsW3U9L8vIqKXi2uNE/8FEm2xCLCyE4fIViCnJkMr1EFr5VK0YEtoRgqvrvQ9A5ERYz0rOxJr3KvB7h+wdE22yGkkUIaamVd3cmJYOyATI27aDvGsYBH9/i60EQ2TPWM9KzsZa9yrwe4ccARNtsjiprAyG5GQYLl+BVFICQekBRZ/ekHfuDEGptHV4RDbFelZyNnfeq5BTUgGtp2WuNPN7hxwBE22yCEmSIN26VXVz4x9/AAYRssA2UPTvB1lwsPHmRqKWjvWs5IyssfY4v3fIEVgt0U5KSsL27dshiiKio6MRGxtr8npxcTE2b96MrKwsuLi44LnnnkPbtm3N6kv2Q6qogHjtGiovXYaUlwe4ukDepQvkXbtC5uNj6/CIGsQa9Z+sZyVqHK5bT9Xs+WebVRJtURQRFxeHRYsWQaPRYP78+QgPD0dwcLBxn127dqF9+/aYO3cu0tLSEBcXh8WLF5vVl2xPzM+vunp99Sqgr4CgVsNl0F8g69ABggt/6JHjsVb9J+tZiRqH69YTYP8/22TWGCQlJQUBAQHw9/eHQqHAoEGDcOrUKZN9UlNT0bNnTwBAUFAQsrOzkZ+fb1Zfsg3JYIDh+nXo9+2D/pvdMFy5AnlwCFwfHAPXcQ9B3qULk2xyWPXVfzan6nrWkWG+6BuswsgwX4v8D8Ja50NkLdb63iH7Zu8/26xyRTsvLw8ajca4rdFokJycbLJPu3btcOLECXTt2hUpKSnIzs5GXl6eWX2rxcfHIz4+HgCwcuVKaLVaC5xN/RQKhU3GtSaxuBjlv/2G8osXIZaUws3LC27Do+DWtStkvLkRQMuYB86uoPx67e16mPXZNmQOaLXAh6FBDQmvwZp6PtQ4/FlgWdb43mkqzgHLsvefbVZJtCVJqtF291JusbGx2LFjB+bOnYu2bduiQ4cOkMlkZvWtFhMTg5iYGOO2LR4W4SwPqcg4c+k/T1/U+KHXIyPh798KlZcvQ7xxA5AAWVAg5PffD1lQECplMpSUlgKlpbYO3S44yzxoyeq6KObjat7PFnubA009n4aw53pJa7O3ecDPxvrsbQ44G2v+bKtLYGBgna9ZJdHWaDTIzc01bufm5sLX19dkH6VSieeffx5AVWL+wgsvoHXr1tDr9ffsS80r48wlJG3YgSzJDeUKD7S59gcyV6yGywM94NU+GPLu3aEIC4Pg5WXrUIksxtnqP1kLTvxsyBnZ+89qq9Roh4aGIiMjA7du3UJlZSWOHTuG8PBwk31KSkpQWVkJADh48CC6desGpVJpVl9qXme+2ocsyQ2elTp0vf0HfMuLUCC44mJWKVwffRQu4eFMssnpOVv9J2vBiZ8NOSN7/1ltlSvacrkc06ZNw4oVKyCKIqKiohASEoL9+/cDAEaOHIm0tDR8+OGHkMlkCA4OxrPPPltvX7Ic19xslCs80LEwHcUuSmR6qqGTu6JreQnXv6YWxRprAVsT1zZu2fjZkLOy55/VVltHu2/fvujbt69J28iRI43/7tKlCz744AOz+5Ll6DV+8Lt+EwKALKUvyhRu8NSXQR/oZ+vQqJFYl0nWwrWN7Rc/GyLrs0rpCDmWXhNHIaiyCDJJhE7hCk99GfyFcvSaOMrWoVEjVNdl7r98G7+mFWP/5duYvSsF6QXltg6NnNAzA9sgyNvVpM2e6iVbMn42RNbHR7BTDQE9u8ClXw/8npyGMKEU+kA/9Jo4Cm16dbV1aNQI9dVl2uuf2shxVddLbjmegZySCmg9+RcUe8HPhsj6mGhTDVJuLlRqHzww9yHIO3a0dTjURKzLJGuz53rJlo6fDZF1sXSEahBvpgKCAFmQfT8EgMzDukwiIiLbYKJNNRjSUiFr7QfBjX9OdAasyyQiIrINlo6QCam0FFJuHuR9+9g6FGomrMskIiKyDSbaZEJMTQUAyIKDbRwJNSfWZRIREVkfS0fIhCEtDYKnEgIfc09ERETUJEy0yUgyGCCmp0MWHAxBEGwdDhEREZFDY6JNRlJWFlBRCVkQy0aIiIiImoqJNhkZUlMBuRyyNgG2DoWIiIjI4THRJiMxNQ2yAH8ILlxfmYiIiKipmGgTAEAsKIBUWAhZcIitQyEiIiJyCky0CQAgpqUBAOTBfBokERERUXNgok0AqspGBB8fCF5etg6FiIiIyCkw0SZIFRUQszL5kBoiIiKiZsREmyBmZAAGkWUjRERERM2IiTZBvJkKuLpAaN3a1qEQEREROQ0m2i2cJEkQ01IhCwyEIJfbOhwiIiIip8FEu4WT8vIglZZBzvpsIiIiombFRLuFE1OrlvWTBbE+m4iIiKg5MdFu4cS0VMi0GggeHrYOhYiIiMipMNFuwSSdDmJ2Dpf1IyIiIrIAJtotmJiWBkgSy0aIiI
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated note: In general, I have found parallel coordinate plots more useful when they are interactive. (Something like https://facebookresearch.github.io/hiplot/)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, maybe worth suggesting to the optuna maintainers.

Copy link

@Abdelgha-4 Abdelgha-4 Sep 19, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optuna now have the interactive parallel coordinate plot.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, wish come true, thanks for letting us know.

thomasjpfan
thomasjpfan reviewed Oct 6, 2021
View reviewed changes
notebooks/optuna-example.ipynb Outdated Show resolved Hide resolved
notebooks/optuna-example.ipynb Outdated
Comment on lines 493 to 526
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAAGHCAYAAACOFvWHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdeVxU9f4/8NeZGbZhAJkZBFncUHHJXMLlKioIbplK30yvLd9Mr1lZWZk3t1wyS01LM/Wm8tVuy/1a/TIrywXzImpu9cU1FUxTVllkZxiYc35/EHMdWRyWWXk9Hw8eeT5zPufzPjMf6M3hfT5HkCRJAhERERERNSuZrQMgIiIiInJGTLSJiIiIiCyAiTYRERERkQUw0SYiIiIisgAm2kREREREFsBEm4iIiIjIAphoExHdJTIyEn/729/s5jj2Mk5D7NixAwqFwtZhNLupU6ciJibG1mEQkYNgok1EDiUrKwsvvvgi2rdvD1dXV/j5+WHixIlISkpq8LHeeusttG/fvkb7119/jffee6/JsTbXcapZOt57uX79OgRBwJEjR2q8tnTpUnTq1Mm4PXnyZKSlpZl97JiYGEydOrU5wmy0f//73xAEwfil0WgQFRWFxMTEJh23U6dOWLp0afMESUQOhYk2ETmMmzdvIjw8HMeOHcPmzZuRkpKCPXv2wMXFBQMHDsTevXubZRy1Wg1vb2+7OY69jNMQHh4e8Pf3t/q4kiShoqKiScf49ddfkZGRgYMHD8LDwwNjxozB9evXmydAImpRmGgTkcOYNWsWKioqcOjQIYwZMwZt27ZF//798a9//QvDhw/H1KlTUVZWBuA/V1g///xzdOzYEe7u7oiJicG1a9cAVJU2vPHGG/jjjz+MVzCrrzreXYoRGRmJ6dOnY9GiRWjdujVatWqFhQsXQhRFvPnmm/D394efnx8WLlxoEu+dx7n7amn1V/UVakmSMGPGDISGhsLDwwMdO3bEggULUF5e3uB4KyoqMG/ePAQFBcHV1RXdu3fH559/bhKbIAjYtGkTnnzySXh5eSEkJASrV69ung8KNUtHCgsL8fTTTyMgIABubm4ICQnBq6++CqCqHOPgwYP4+OOPjef273//GwBw+fJljB07FiqVCiqVCuPGjUNKSkqNcQ4dOoQ+ffrAzc0Nmzdvhkwmw7Fjx0xiSkhIgEwmw++//15v7H5+fggICEDv3r2xdetWlJSUYN++fbXuK0kS1qxZg44dO8LV1RWhoaFYt26d8fXIyEhcvXoVy5YtM54bk3ailoOJNhE5hNu3b2PPnj144YUXar16O3/+fGRlZeHAgQPGtoyMDGzatAk7d+5EYmIiioqKEBsbC0mSMHnyZLz++usIDg5GRkYGMjIy8Nprr9U5/ldffYWKigocOXIE7733Ht5++2089NBDKC4uRmJiItasWYO3334bP/74Y639Bw0aZBwnIyMDFy5cQGBgIKKiogBUJWz+/v74/PPP8dtvv2HdunXYvn073n77bQBoULwLFizA1q1bsW7dOpw/fx5PPPEEnnjiCRw8eNBkv2XLlmHo0KFISkrC3Llz8frrr+PQoUP1fxCNtGjRIvz666/YvXs3kpOTsXPnTnTr1g0AsH79egwZMgSTJk0yntugQYNQVlaGkSNHQqfTISEhAQkJCSguLsbo0aOh1+uNxxZFEX//+9+xdu1aXLp0CVOmTMGIESOwdetWkxi2bduG6OhodOzY0ey4PTw8AKDOq+SbNm3CG2+8gXnz5uHChQuYO3cu5s2bh7i4OABVZT3t27fHnDlzjOcWEhLSoPeOiByYRETkAE6cOCEBkL7++utaX8/NzZUASKtXr5YkSZKWLFkiAZCSk5ON+1y+fFkCIB04cECSJElavny51K5duxrHGjZsmDR9+nST7V69epns0717d+m+++4zabv//vulOXPm1Hmcanq9XoqMjJQiIiIknU5X5zm/9957UqdOnYzb5sRbUlIiubq6Shs3bjTZJzY2VoqKijJuA5BefPFFk33CwsKkefPm1RnPtWvXJACSh4eH5OnpafLl4uIihYaGGvfdvn27JJfLjdvjx4+XnnrqqTqPHR0dXeP1bdu2SR4eHlJ2draxLTMzU3J3d5c+/vhj4zgApMOHD5v0/X//7/9JSqVSys/PlyRJkm7fvi15eHhIX3zxRZ0xHDp0SAIg3bx5U5IkSSosLJT+9re/SQqFQjp37pwkSZL01FNPSdHR0cY+wcHB0ty5c02O8/LLL0sdOnQwboeGhkpLliypc1wicl68ok1EDkGSpHpfFwShRpufn5/JDXpdunSBVqvFxYsXGzx+r169TLYDAgJw//3312i7devWPY/13HPP4ebNm9i1axfc3NyM7Vu3bsWAAQPg7+8PlUqF+fPn448//mhQnCkpKdDr9Rg6dKhJ+7Bhw3DhwgWTtt69e5tsBwUFISsr655jbN++HUlJSSZfzz77bL19nn/+eXz11Ve47777MHv2bPz4448QRbHePhcuXED37t2h1WqNbf7+/ggLC6txLv369TPZHj9+PHx8fIwlM59++ilUKhUmTJhwz/MLCwuDSqWCj48P9u3bh3/+85+47777auxXWFiI1NTUWt/r69evo7S09J5jEZFzY6JNRA6hc+fOkMlkOH/+fK2vV7eHhYXVe5x7Jex1cXFxMdkWBKHWtnslj6tXr8bXX3+NPXv2mCSQX375JWbNmoXJkyfjhx9+wP/93/9h8eLFjb6x7+5fPCRJqtHm6ura4PiBqoS8U6dOJl9qtbrePqNGjcKNGzewcOFC6HQ6PPHEExg+fDgMBkODzqO2c5HL5XB3dzfZR6FQYPr06cbykW3btmHq1Kk1zrk2+/btw5kzZ5CTk4MbN25gypQpDYqxsXOMiJwPE20icghqtRpjxozBxo0bUVhYWOP1t99+G/7+/hgxYoSxLTs7G1evXjVuX7lyBbm5ucbaYFdX13smes3pm2++weLFi/H111/X+IXg8OHD6NOnD1599VU88MAD6Ny5c42b5syJt1OnTnBzc0NCQkKN4/fo0aNZzqOx1Go1pkyZgo8++gh79uxBQkKC8a8LtZ1bjx49cOHCBeTk5BjbsrKycOXKFbPOZcaMGThz5gz+8Y9/4MyZM2avNd6+fXuEhobe85cHb29vBAcH1/ped+jQAUqlss5zI6KWgYk2ETmMjRs3Qi6XY/jw4di7dy9u3ryJU6dO4bHHHsOhQ4ewY8cO481rAKBUKvH000/jl19+wenTp/HUU0+hZ8+exgeOdOjQAZmZmfj555+Rk5Nj0T/1X7hwAU888QSWLl2Krl27IjMzE5mZmcjOzgZQdSX+3Llz2L17N65evYr169fj66+/NjmGOfEqlUq89NJLeOONN/Dll18iOTkZb7/9Nnbv3o0FCxZY7PzuZeHChfj6669x+fJlJCcn47PPPoNKpULbtm0BVJ3bL7/8gqtXryInJwcVFRV47LHH4Ofnh8mTJ+PXX3/FL7/8gr/+9a8ICgrC5MmT7zlm27ZtMXr0aMyePRuRkZHo0qVLs5/X/PnzsWHDBmzduhXJycn46KOPsHnzZpP3ukOHDjh69Chu3LiBnJwcs/5qQETOgYk2ETmMdu3a4fTp0xgwYABmzpyJ0NBQjBkzBuXl5fj5558xevRok/3btGmDZ555Bo888ggGDx4MDw8P7Nq1y/in/tjYWDz66KMYO3Ys/Pz8mnV5u7udOnUKJSUlmD9/Ptq0aWP8qq4tnjlzJp588kk8/fTT6NOnD06cOFHjISfmxrtixQrMmDEDL7/8Mnr06IFPP/0Un376KaKjoy12fvfi7u6OxYsX44EHHkB4eDjOnj2LH3/8ET4+PgCAOXPmQKvVolevXvDz88PRo0fh4eGB/fv3w83NDUOHDsWwYcPg6emJvXv3mlUCAgDPPPMM9Ho9nnnmGYuc13PPPYc333wTb7/9Nrp3745Vq1Zh5cqVmD59unGfZcuWoaCgAGFhYfDz88ONGzcsEgsR2R9BYjEZETmhpUuX4tNPPzVZc5lank2bNmHx4sVIS0szufGUiMgaFPfehYiIyLEUFxcjJSUFa9aswQsvvMAkm4hsgqUjRETkdF544QX0798f3bp1w+uvv27rcIiohWLpCBERERGRBfCKNhERERGRBTDRJiIiIiKyACbaREREREQW4NSrjqSnp1t9TK1Wa/IUM2qZOA+Ic4AAzgPiHGgJAgMD63yNV7SJiIiIiCyAiTYRERERkQUw0SYiIiIisgCnrtG+myRJ0Ol0EEURgiBYZIysrCyUl5db5NgtnSRJkMlkcHd3t9jnR0RERNRcrJZoJyUlYfv27RBFEdHR0YiNjTV5vbi4GJs3b0ZWVhZcXFzw3HPPoW3btgCAWbNmwd3dHTKZDHK5HCtXrmxUDDqdDi4uLlAoLHfaCoUCcrncYsdv6SorK6HT6eDh4WHrUIiIiIjqZZVEWxRFxMXFYdGiRdBoNJg/fz7Cw8MRHBxs3GfXrl1o37495s6di7S0NMTFxWHx4sXG15csWQJvb+8mx2HJJJssT6FQ8C8GRERE5BCsUqOdkpKCgIAA+Pv7Q6FQYNCgQTh16pTJPqmpqejZsycAICgoCNnZ2cjPz2/WOFhu4Bz4ORIREZEjsEqinZeXB41GY9zWaDTIy8sz2addu3Y4ceIEgKrEPDs722SfFStW4PXXX0d8fLw1QraYkJAQjBgxAjExMRg1alSNXzjMtXXrVpSVldVoX7t2Ld555x2TtvPnz2PYsGH1Hm/ixIk4c+ZMo2IhIiIiopqsUkchSVKNtruvSsbGxmLHjh2YO3cu2rZtiw4dOkAmq/o9YPny5VCr1SgoKMBbb72FwMBAdO/evcYx4+PjjYn4ypUrodVqTV7PyspqUOlIRcpVVBw+DDErCzJ/f7gMHQqXTqH37FffGO7u7jh06BAA4NChQ1i1ahW++eYbs2OqFhcXh0mTJsHLy8uk/ZFHHsGUKVPwxhtvGNu+++47PPLII/XGJQgC5HK5Q5TWuLm51fhs7Y1CobD7GMmyOAcI4DwgzoGWzipZlUajQW5urnE7NzcXvr6+JvsolUo8//zzAKoS8xdeeAGtW7cGAKjVagCAj48P+vXrh5SUlFoT7ZiYGMTExBi3734SU3l5udk3KlZevQr9F19CUKkAjQaGggJU/utfMEx6FIrQupNthUKBysrK+o/95+v5+fnw9vY2bm/evBnfffcd9Ho9Ro8ejddeew2lpaWYOXMmMjIyIIoiZs+ejZycHGRmZuK//uu/4Ovri6+++sp47Pbt28Pb2xsnT55E3759AQDffvstPvvsM1RWVmLevHk4c+YMdDodxo4di9deew1A1XtuMBhQWVmJzp07Izk5GQDw/fffIz4+HuvWrUNubi7mzZuHtLQ0AMCyZcvQr18/s97P5lReXm73T9nik8CIc4AAzgPiHGgJ6nsypFUS7dDQUGRkZODWrVtQq9U4duwYXnrpJZN9SkpK4ObmBoVCgYMHD6Jbt25QKpXQ6XSQJAkeHh7Q6XQ4e/YsJk6c2OSYKk6ehHRX+YrJ60eOQtLpIJSUGNsknQ7l23dAjBhcax9BrYZi0KB6x9XpdBgxYgTKy8tx69YtfPHFFwCAhIQEXLt2DXv27IEkSZg6dSqOHz+O3NxcBAQE4JNPPgEAFBYWwtvbG1u2bMGXX35p/CXkTrGxsdi9ezf69u2LX375Bb6+vujYsSMA4PXXX4evry8MBgMmT56Mixcv1vpLS20WL16MGTNmoH///khLS8Njjz2GhIQEs/oSERERtTRWSbTlcjmmTZuGFStWQBRFREVFISQkBPv37wcAjBw5Emlpafjwww8hk8kQHByMZ599FgBQUFCANWvWAAAMBgMiIiLQu3dvi8csFRYCd5VlwM2tqr0J3N3dceDAAQDA6dOnMXv2bPz0009ISEhAQkICRo4cCQAoLS3FtWvX0L9/fyxfvhwrVqxATEwMBgwYcM8xxo8fjwkTJmDJkiXYvXs3JkyYYHztu+++w2effQaDwYCsrCwkJyebnWgnJibiypUrxu3i4mIUFxdDpVI15C0gIiIiahGsVpDbt29fYylDteqkEgC6dOmCDz74oEY/f39/vPvuu80ej0v//vW+LmZmQSoqgnBHsi0VFUHo3Bmuo0c3Swzh4eHIy8tDbm6usVzmySefrLHfjz/+iJ9++gnvvPMOhg0bhldeeaXe4wYFBSEkJAQ///wzfvjhB3z77bcAgBs3buCjjz7Cnj170KpVK7z88svQ6XQ1+t9ZP3/nUnqiKOLbb7/lGtZEREREZuAj2OugGBIBqbgYUlERJFGs+m9xMRRDIpptjJSUFBgMBvj6+iIyMhI7d+5EyZ+lKhkZGcZabA8PDzzyyCN49tlnce7cOQCASqVCcXFxnceeMGECli5divbt2xtrh4qKiuDh4QFvb29kZ2cbb8q8m5+fH5KTkyGKIvbu3WtsHzZsGHbs2GHcPn/+fFPfAiIiIiKnZf9LTNiIIjQUmPQoKhOP/GfVkQfH1HsjpDmqa7SBqhsQ161bB7lcjmHDhiE5ORnjx48HUHVz6IYNG3D9+nW89dZbEAQBLi4uxqX7Hn/8cTzxxBNo3bq1yc2Q1caNG4clS5Zg+fLlxrYePXrgvvvuQ1RUFNq2bVvnjYzz58/HU089hcDAQISFhRmT/+XLl2PBggWIiYlBZWUlBgwYgFWrVjXp/SAiIiJyVoJU29p7TiI9Pd1ku7S0FEql0qJjmrPqCDWNNT7HpuJd5sQ5QADnAXEOtAT1rTrC0hEiIiIiIgtg6QgREdUp48wlnPlqH1xzs6HX+KHXxFFo06trvX0qr141KbtTDIloctmdLTnT+VjrXJxtHGtpzPk423vQGPb8HsiXLl261NZBWEpRUZHJdkVFBVxcXCw6pkwmgyiKFh2jpbPG59hUSqUSpaWltg6DbMgZ5kDGmUtI2rADWUUVyJApUV5QhOKTv0DVLhheAbU/6a76YV8AILRqBRQXw/B/SRAC20BWy7r/9q6p52NP88Ban42zjdNU5s6BxpyPo7wHlmQP78HdT+m+E2u0mxlrtC2PNdrkCJxhDux9Yz1y0rKhLi+CW6UeAOBqqICXtxLdxgyttU/FqVOQysshuLkZ26q3XWzwJNmmaur5eKo8UVJccs/9rMFan42zjdNU5s6BxpyPo7wHlnTneyAL8IdMrTEuz+w+9SmrxGDzJ0MSEZHjcc/OhH9JAVylShS4qiAJACR3KCvLIQtsU3sngwGCRm2yHj9UnkBBYd197FkTz8fF2xuyJj7orNlY67NxtnGayOw50JjzcZD3wKLueA8EF9eqNk9PiFlZto3rT0y0iYioBrGwEK0qSlBWqcNl3xCUuFQ9qMpTXwa3QDVcBg+utZ8hOaX2h321bVdnH3vW1PPx1GpRZid/2bDWZ+Ns4zSVuXOgMefjKO+BJdX2HqCkBDJ/f9sFdQeuOmJl6enpePrppzF48GAMGjQIixcvhl5f9SfZnTt3YuHChbX2q15fu6H27t1r8tj0d999F4cPH27Usart3LkTzz//vElbXl4eevbsafIkybu9/PLL+P7775s0NhFZnpiXh4of9yK4RyeUePkCEiBIEjz1ZfAXytFr4qg6+1rjYV/W5EznY61zcbZxrKUx5+Ns70Fj2Pt7wETbiiRJwowZMzB69GgcPXoUiYmJKCkpMeuhL9WPUW+ouxPtuXPnYujQ2msrzfXggw/i8OHDKCsrM7Z9//33GDlyJNzuqBMjIscjZmejYt8+QCZAPe2/0eOVGQgIVKOrrAQBgWr0fnFqvauOKEJD4TrpUQheXpCysyF4ecF10qN2swJAQznT+VjrXJxtHGtpzPk423vQGPb+HrB0pB7pBeXYcjwDOcUV0Kpc8MzANgj0aXwieeTIEbi5uWHy5MkAALlcjqVLl2LgwIF47bXXqsZMT8fjjz+OGzdu4OGHH8arr74KAOjcuTOSk5MBAJs3b8Z3330HvV6P0aNHG/t++eWX+OijjwAA3bp1w3//93/jwIEDOH78ONavX4+tW7di3bp1iImJgVKpxM6dO437Hzt2DB999BE+/vhjJCQkYM2aNdDr9WjXrh3ef/99eHp6Gs/Dy8sLAwcOxP79+zFhwgQAVb8IvPTSSwCA999/HwcOHIBOp0N4eDhWrVplWj8GYMCAAfjxxx+hVqtx5swZLF++HF999RVKS0uxaNEiXLp0CZWVlZgzZw5Gjar76hkRNR9DWjoqDh2CoPSA68iREFQqtGnV6p7L+d1NERpqN/+Taw7OdD7WOhdnG8daGnM+zvYeNIY9vwdMtOuQXlCO2btSkFaoN7ZdyCjB+oc7NTrZvnLlCnr27GnS5uXlhaCgIFy7dg0AkJSUhIMHD8LDwwNjx45FdHQ0evXqZdw/ISEB165dw549eyBJEqZOnYrjx4/D19cXH3zwAXbv3g21Wo3bt2/D19cXI0aMQExMDB566CGTcYcOHYrXX3/duILHt99+i/HjxyMvLw/r16/Hzp07oVQqsXHjRmzZsgWvvPKKSf8JEybgm2++wYQJE5CZmYnff/8dg/+sB5s6dapx/xdffBEHDhzAyJEjzXqP1q9fj8GDB+O9995DQUEBxo4diyFDhtj9KiNEjs7wxx+oOHwYgrcPXEeOgODhYeuQiFqcxlzga+6LgtS8mGjXYcvxDJMkGwDSCvXYcjwDS0e1b9QxJUmqcWX37vYhQ4ZA/ee6j2PGjMHJkydrJNoJCQnGxLW0tBTXrl3DxYsXMXbsWGNfX1/femNRKBSIiorCgQMHMHbsWBw8eBCLFi3Czz//jCtXrhivVFdUVOCBBx6o0T8mJgYLFixAUVERvvvuO4wdOxZyuRxA1dXxzZs3o6ysDPn5+QgLCzM70T58+DAOHDiAf/zjHwCA8vJypKWloXPnzmb1J6KGMyQno+LYz5D5aeESEwPB1dXWIRG1OI25wGeJi4LUvJho1yGnuKL29pLa283RpUsX/PDDDyZtRUVFSE9PR/v27XH27Nkaifjd25Ik4YUXXsCTTz5p0h4XF1drEl+fcePG4eOPP0arVq3Qu3dvqFQqSJKEoUOHYtOmTfX29fDwQGRkJH788Ufs3r0b1c890ul0WLBgAX744QcEBQVh7dq1td4gqVAojA/2ufN1SZKwZcsWdOrUqUHnQkSNU3nhAipPnYYsKBAukZEQ7PxhUETOqjEX+CxxUZCaF2+GrINWVfv/bLSejf+f0JAhQ1BWVoYvv6x6gpHBYMCbb76JSZMmwePPP9MmJibi9u3bKCsrw759+9DvrgXnIyMjsXPnTpSUVC1+n5GRgZycHEREROC7775DXl4eAOD27dsAAJVKZdz3boMGDcK5c+fw2WefYdy4cQCABx54AKdOnTKWspSVleHq1au19o+NjcWWLVuQk5NjvOpdnTSr1WqUlJRgz549tfYNDg7G2bNnAcBkn2HDhmH79u2ofo7S+fPna38ziahJJElC5a//h8pTpyFv3x4uw4czySayocZc4LPERUFqXryiXYdnBrbBhYwSk98Ug7xd8czAxi8ALwgCtm3bhgULFmDdunWQJAnDhw/HvHnzjPv069cPL730Eq5fv46HH37YWDZSfbV62LBhSE5ONi73p1QqsWHDBoSFheGll17CxIkTIZPJcN9992HdunWYMGEC5s6di7i4OGzZssUkHrlcjpiYGHzxxRdYv349AECj0eD999/HrFmzjMsO/v3vf0doLTcZDBs2DC+//DKmTJlijM/HxwePPfYYYmJiEBwcbFL2cqdXX30Vc+bMwYYNG9CnTx9j+8svv4wlS5YgJiYGkiQhODgY//znPxv1ft8p48wlnPlqH1xzs6HX+KHXxFENvsHLkVVevYrKxCMQs7Ig8/eHYkiE3d44Yk8a875V97lVUAC9j0+D+lj687lzHKmiAlAq4TpgABR/GQhBxusudWENLFlDYy7wWeKiIDUvPoK9vv7VP1xLKqD1NO+HqyUewZ6Xl4fRo0fj5MmTzXpcR9XQzzHjzCUkbdiBLMkNpS7uUFbo4C+U33OZsqawp8dvV169Cv0XX0JQqQBPT6CkBFJxsV0tf2SPGvO+3dlH6eeH0uzsBvWx5OdTPQ48PSHm5UG6dQuCUgm3Z5+FSyfOg7rUVgMb5O1qdg2sPf0sINswdw40Zq41dX5S86jvEexMtJtZcyfamZmZmDhxIqZNm4Zp06Y123EdWUM+R0mScHDeauhSM6CQDJBJVXXhbgY9vL1V6D5+uEViVKk8UVxce8mOtVX8/DMknQ6Cu7uxrXrb5S9/sWFk9q0x79udfVxdXaHX6xvUx9xxmnQ+ri5AuR6ywDYQPJQQvLzgPvWpZhvH2Szddx37L9+u0T4yzNesGlgm2tSQOdCYC3yN6UPNq75Em6Ujdi4gIABHjhyxdRgORRJFSLeyYbh5A+KNGwi6dhE5ogtKXdxRIav6c1qFoIC3rgzCHeuDNyeZyguCvfwKW1YGwdvb9GZZFxegsNBi5+8UGvO+3dFHcHWF0MA+Zo/TGHfGFhAAma8akihCzMpqvjGcEGtgyZoCfdwafBNjY/qQ9TDRJodTmleAw+9sNam3DujRCWJ6OsQbNyCmpkLSlQNyGWRt2iAnqCNuFOiR7+ZlPIanvgzyQDX6RFvmiraXVotyO7mKJd68CamoCILXf85fKiqC0LEjXC10/s6gMe+bePMminLz8Us+UF4MuCk88UArwOsefazx+dQ2DkpKIPP3b7YxnBFrYImoKVrU3S9OXCXTYuhLy3D7Ugoy0/OQUukO3bUbuPnOWuS+vwEVPx2C4cZNyAKD4DJsGNwmT4ZrTAw6TZ0MtcwAT30ZBEmCp74M/kI5ek1sGU+cVAyJgFRcDKmoqOpqf1ERpOJiKIZE2Do0u9aY9+12nwE4/VsGMtPzkHa7FJnpeTj9WwZu9xnQrONY63yo6sb4IG/TdcWbemM8EbUc8qXVCyA7oaKiIpNtg8EAQRAgs+Dd9TKZzLg+NDW//IxsZJy5BNnlywgsyYHSUA6DKKGgEmj79BQoBgyAvEN7yHxbQfjzATpeAVqo2gWj5MLybLEAACAASURBVGY6/HX58PDXovfTEy266ohSqURpaanFjt8QMrUaQmAbSJlZkG7dgkythsuDY3gj5D005n1be6YYh0o8EFCah9a6fOS7e+H7kH647hOIyE6tmm0ca50PAV7uCkR08EG+rhI+Hgr0bOOJRSPamV0Da08/C8g2OAecn9edfym8S4sqHXF3d4dOp0N5eXmDH+5iLjc3t1of0EJNI1VUQCosROH+Q9DtS4Sr3AW3lL4odPWETu6KrlIJZPXcjNCmV9cWtZzf3RShoUyoGqGh71tOcQWu+wTiuo/pXFTfo57XWp8P50HjsAaWiBqrRSXagiAYHwxjKbzDvPlIkgQxPQOGy5cg3kwFAOScPocsD19keaqN+3nqy6AP9LNVmERGrOclqsK1x4mqtKhEmxyDVF4OQ0oKDJcvQyosguDuDkXP+yDv0gXBvfsjZ8MOeOrLTNbEbin11mTfLPGgKyJHU9vazhcySri2M7VITLTJbog5OTBcvgzD79cAgwEy/9ZQ9O4NWbt2xnrrNr26Ai9O/c9THgNb3lMeyX4F+rhh/cOdsOV4Bgr0gI8reCWPWpwtxzNMkmwASCvUY8vxDJbgUIvDRJtsSqqshHj9OgyXLkPMyQFcFJB3CoU8LAwytbrWPi293prsW3U9L8vIqKXi2uNE/8FEm2xCLCyE4fIViCnJkMr1EFr5VK0YEtoRgqvrvQ9A5ERYz0rOxJr3KvB7h+wdE22yGkkUIaamVd3cmJYOyATI27aDvGsYBH9/i60EQ2TPWM9KzsZa9yrwe4ccARNtsjiprAyG5GQYLl+BVFICQekBRZ/ekHfuDEGptHV4RDbFelZyNnfeq5BTUgGtp2WuNPN7hxwBE22yCEmSIN26VXVz4x9/AAYRssA2UPTvB1lwsPHmRqKWjvWs5IyssfY4v3fIEVgt0U5KSsL27dshiiKio6MRGxtr8npxcTE2b96MrKwsuLi44LnnnkPbtm3N6kv2Q6qogHjtGiovXYaUlwe4ukDepQvkXbtC5uNj6/CIGsQa9Z+sZyVqHK5bT9Xs+WebVRJtURQRFxeHRYsWQaPRYP78+QgPD0dwcLBxn127dqF9+/aYO3cu0tLSEBcXh8WLF5vVl2xPzM+vunp99Sqgr4CgVsNl0F8g69ABggt/6JHjsVb9J+tZiRqH69YTYP8/22TWGCQlJQUBAQHw9/eHQqHAoEGDcOrUKZN9UlNT0bNnTwBAUFAQsrOzkZ+fb1Zfsg3JYIDh+nXo9+2D/pvdMFy5AnlwCFwfHAPXcQ9B3qULk2xyWPXVfzan6nrWkWG+6BuswsgwX4v8D8Ja50NkLdb63iH7Zu8/26xyRTsvLw8ajca4rdFokJycbLJPu3btcOLECXTt2hUpKSnIzs5GXl6eWX2rxcfHIz4+HgCwcuVKaLVaC5xN/RQKhU3GtSaxuBjlv/2G8osXIZaUws3LC27Do+DWtStkvLkRQMuYB86uoPx67e16mPXZNmQOaLXAh6FBDQmvwZp6PtQ4/FlgWdb43mkqzgHLsvefbVZJtCVJqtF291JusbGx2LFjB+bOnYu2bduiQ4cOkMlkZvWtFhMTg5iYGOO2LR4W4SwPqcg4c+k/T1/U+KHXIyPh798KlZcvQ7xxA5AAWVAg5PffD1lQECplMpSUlgKlpbYO3S44yzxoyeq6KObjat7PFnubA009n4aw53pJa7O3ecDPxvrsbQ44G2v+bKtLYGBgna9ZJdHWaDTIzc01bufm5sLX19dkH6VSieeffx5AVWL+wgsvoHXr1tDr9ffsS80r48wlJG3YgSzJDeUKD7S59gcyV6yGywM94NU+GPLu3aEIC4Pg5WXrUIksxtnqP1kLTvxsyBnZ+89qq9Roh4aGIiMjA7du3UJlZSWOHTuG8PBwk31KSkpQWVkJADh48CC6desGpVJpVl9qXme+2ocsyQ2elTp0vf0HfMuLUCC44mJWKVwffRQu4eFMssnpOVv9J2vBiZ8NOSN7/1ltlSvacrkc06ZNw4oVKyCKIqKiohASEoL9+/cDAEaOHIm0tDR8+OGHkMlkCA4OxrPPPltvX7Ic19xslCs80LEwHcUuSmR6qqGTu6JreQnXv6YWxRprAVsT1zZu2fjZkLOy55/VVltHu2/fvujbt69J28iRI43/7tKlCz744AOz+5Ll6DV+8Lt+EwKALKUvyhRu8NSXQR/oZ+vQqJFYl0nWwrWN7Rc/GyLrs0rpCDmWXhNHIaiyCDJJhE7hCk99GfyFcvSaOMrWoVEjVNdl7r98G7+mFWP/5duYvSsF6QXltg6NnNAzA9sgyNvVpM2e6iVbMn42RNbHR7BTDQE9u8ClXw/8npyGMKEU+kA/9Jo4Cm16dbV1aNQI9dVl2uuf2shxVddLbjmegZySCmg9+RcUe8HPhsj6mGhTDVJuLlRqHzww9yHIO3a0dTjURKzLJGuz53rJlo6fDZF1sXSEahBvpgKCAFmQfT8EgMzDukwiIiLbYKJNNRjSUiFr7QfBjX9OdAasyyQiIrINlo6QCam0FFJuHuR9+9g6FGomrMskIiKyDSbaZEJMTQUAyIKDbRwJNSfWZRIREVkfS0fIhCEtDYKnEgIfc09ERETUJEy0yUgyGCCmp0MWHAxBEGwdDhEREZFDY6JNRlJWFlBRCVkQy0aIiIiImoqJNhkZUlMBuRyyNgG2DoWIiIjI4THRJiMxNQ2yAH8ILlxfmYiIiKipmGgTAEAsKIBUWAhZcIitQyEiIiJyCky0CQAgpqUBAOTBfBokERERUXNgok0AqspGBB8fCF5etg6FiIiIyCkw0SZIFRUQszL5kBoiIiKiZsREmyBmZAAGkWUjRERERM2IiTZBvJkKuLpAaN3a1qEQEREROQ0m2i2cJEkQ01IhCwyEIJfbOhwiIiIip8FEu4WT8vIglZZBzvpsIiIiombFRLuFE1OrlvWTBbE+m4iIiKg5MdFu4cS0VMi0GggeHrYOhYiIiMipMNFuwSSdDmJ2Dpf1IyIiIrIAJtotmJiWBkgSy0aIiIiILICJdgsmpqZC8PCAoNXaOhQiIiIip8NEu4WSR
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am getting an error running with optuna=2.10.0 and matplotlib=3.4.3

    optuna.visualization.matplotlib.plot_parallel_coordinate(
        study, params=['num_units', 'lr', 'max_epochs', 'optimizer'])
ValueError: Data has no positive values, and therefore can not be log-scaled.

Are you getting this error too?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not get this error when I ran the notebook. Then I upgraded optuna from 2.3.0 to 2.10.0 and now I get the same error. Seems like it's a regression in optuna?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels like a regression. The function works when "lr" is removed.

@BenjaminBossan
Copy link
Collaborator Author

Closing this as it's super old :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Abdelgha-4 Abdelgha-4 Abdelgha-4 left review comments

@thomasjpfan thomasjpfan thomasjpfan approved these changes

@ottonemo ottonemo Awaiting requested review from ottonemo

Assignees

@BenjaminBossan BenjaminBossan

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add an optuna example
3 participants
@BenjaminBossan @thomasjpfan @Abdelgha-4