Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dask integration #2

Open
wants to merge 51 commits into
base: dask_integration
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
6364df6
adding reqs and new notebook
blue442 Jul 18, 2020
938a815
fix file name
blue442 Jul 18, 2020
0da0680
fix file name
blue442 Jul 18, 2020
c4d3256
wright edited
wrightstephen Jul 22, 2020
4a48efd
adding files for pilot simulations to generate figures
blue442 Jul 22, 2020
086b7e4
added reqs
blue442 Jul 22, 2020
e319779
Merge branch 'master' of https://github.com/AFIDSI/group-testing
blue442 Jul 22, 2020
be8fcb2
updated nominal yaml for pilot simulations
blue442 Jul 22, 2020
2ade4db
specify multiple params in one config file
blue442 Jul 22, 2020
94b2efe
update uw sensitivity configure, plot uw sample_path_severity_distrib…
chenyd Jul 22, 2020
f7b9d45
update uw fig 1 configuration
chenyd Jul 22, 2020
d9252b3
updated plotting for std y axis and changed order
blue442 Jul 22, 2020
72a8093
updated graphs
blue442 Jul 23, 2020
6f43ba8
added uw_pilot.sh
blue442 Jul 27, 2020
a32673a
implementing multiprocessing.Pool
blue442 Jul 28, 2020
ff786ae
removed pickling of lambda functions
blue442 Jul 30, 2020
84479a9
running simulation for dorm room-level modeling and plotting output i…
blue442 Aug 5, 2020
3159203
updated axis label
blue442 Aug 5, 2020
985d972
more graph manipulations
blue442 Aug 5, 2020
fa573f6
updates to room sims
blue442 Aug 5, 2020
819a7d3
switched from multiprocessing.Pool to dask implementation
blue442 Aug 7, 2020
e72c990
killing req branch
blue442 Aug 11, 2020
62ea289
Merge branch 'requirements' into multiprocessing_pool
blue442 Aug 11, 2020
e4ebbf2
implemented distributed computing w/ dash-chtc (need to try yet)
blue442 Aug 11, 2020
44af306
add csv generating notebook
masseyc1 Aug 14, 2020
8c70ed5
piece together gateway + semester, interpolating
Aug 14, 2020
5e63c56
added daily FPR for phase 3
Aug 14, 2020
9226eb9
added daily FPR for phase 3
Aug 14, 2020
043fcaf
fixed import omission, updated requirements.txt
blue442 Aug 17, 2020
4ca567d
updated req's
blue442 Aug 17, 2020
7b42363
exploring moving dfs definition for CHTC
blue442 Aug 17, 2020
eea2318
updated run_sensitivity.py output path for CHTC
blue442 Aug 17, 2020
6162e10
Merge branch 'multiprocessing_pool' of https://github.com/AFIDSI/grou…
blue442 Aug 17, 2020
d6dd846
reconfiguring dask submission
blue442 Aug 17, 2020
22e8465
Merge branch 'multiprocessing_pool' of https://github.com/AFIDSI/grou…
blue442 Aug 17, 2020
2a39be3
update configure_lockdown logic in multigroup simulation & update loc…
masseyc1 Aug 18, 2020
e49aebb
Merge pull request #1 from peter-i-frazier/master
blue442 Aug 18, 2020
6b60914
adding materials for uw multigroup simulations
blue442 Aug 19, 2020
4cdf979
updated uw group modeling
blue442 Aug 21, 2020
3e6eda8
forgot a few things...
blue442 Aug 21, 2020
e720cd6
added age_severity_config.yaml (and a few others) to source control
blue442 Aug 23, 2020
91e013e
Merge branch 'multiprocessing_pool' of https://github.com/AFIDSI/grou…
blue442 Sep 4, 2020
2915a46
stuffing stochastic simulation class definition into run sensitivity.…
blue442 Sep 4, 2020
8a9517b
Merge branch 'multiprocessing_pool' of https://github.com/AFIDSI/grou…
blue442 Sep 4, 2020
0170d30
added logging
blue442 Sep 4, 2020
c97a69a
Merge branch 'multiprocessing_pool' of https://github.com/AFIDSI/grou…
blue442 Sep 4, 2020
b0de676
refactor
blue442 Sep 4, 2020
db9250d
Merge branch 'multiprocessing_pool' of https://github.com/AFIDSI/grou…
blue442 Sep 4, 2020
d8c571b
moving sampling functions internal to StochasticSimulation class
blue442 Sep 4, 2020
2e4b65c
add log directory to CHTCCluster
blue442 Sep 4, 2020
154a066
Merge branch 'multiprocessing_pool' of https://github.com/AFIDSI/grou…
blue442 Sep 4, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
768 changes: 768 additions & 0 deletions notebooks/UW-8-group-simulations.ipynb

Large diffs are not rendered by default.

543 changes: 543 additions & 0 deletions notebooks/UW_multigroup_simulation_interaction_matrix.ipynb

Large diffs are not rendered by default.

2,653 changes: 2,653 additions & 0 deletions notebooks/aug_11_generate_csvs.ipynb

Large diffs are not rendered by default.

4,965 changes: 4,965 additions & 0 deletions notebooks/aug_17_evaluate_lockdown_protocols_multiprocessing.ipynb

Large diffs are not rendered by default.

5,381 changes: 5,381 additions & 0 deletions notebooks/aug_18_evaluate_second_deriv_lockdown_protocols_multiprocessing.ipynb

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions notebooks/aug_5_mandatory_quarantine_analysis.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "IPython (Python 3)",
"language": "python",
"name": "python3"
},
Expand All @@ -259,7 +259,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.6"
"version": "3.6.2"
}
},
"nbformat": 4,
Expand Down

Large diffs are not rendered by default.

108 changes: 65 additions & 43 deletions notebooks/aug_6_evaluate_lockdown_protocols.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,7 @@
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"metadata": {},
"outputs": [],
"source": [
"import sys\n",
Expand Down Expand Up @@ -34,10 +32,19 @@
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": true
},
"outputs": [],
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/stevewangen/projects/COVID/uw_reopening/src/simulations_v2/load_params.py:132: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.\n",
" params = yaml.load(f)\n",
"/Users/stevewangen/projects/COVID/uw_reopening/src/simulations_v2/load_params.py:46: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.\n",
" age_sev_params = yaml.load(f)\n"
]
}
],
"source": [
"ug_dorm_params = load_params('../src/simulations_v2/params/baseline_testing/august_5/ug_dorm.yaml')[1]\n",
"ug_off_campus_params = load_params('../src/simulations_v2/params/baseline_testing/august_5/ug_off_campus.yaml')[1]\n",
Expand All @@ -52,9 +59,7 @@
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true
},
"metadata": {},
"outputs": [],
"source": [
"interaction_matrix = np.array([[12.5,4,0.1,0.1,1,0.05,0.05,0.1],\n",
Expand All @@ -71,7 +76,7 @@
},
{
"cell_type": "code",
"execution_count": 32,
"execution_count": 4,
"metadata": {},
"outputs": [
{
Expand All @@ -80,7 +85,7 @@
"[1.23]"
]
},
"execution_count": 32,
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -91,10 +96,8 @@
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"collapsed": true
},
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"lockdown_matrix_str = \\\n",
Expand All @@ -112,10 +115,8 @@
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": true
},
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"def get_cum_hosp(df):\n",
Expand Down Expand Up @@ -154,10 +155,8 @@
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": true
},
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"def run_multigroup_sim(sim, T):\n",
Expand Down Expand Up @@ -188,10 +187,8 @@
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"collapsed": true
},
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"def evaluate_testing_policy(params_list, interaction_matrix, group_names, test_frac, T, n,\n",
Expand All @@ -215,10 +212,8 @@
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": true
},
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"def get_params_list_copy():\n",
Expand All @@ -234,10 +229,8 @@
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"collapsed": true
},
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"def simulate_lockdown(contact_multiplier, infections_thresh, infections_window, nreps=10):\n",
Expand All @@ -258,14 +251,43 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"elapsed time 72.88\n"
"ename": "TypeError",
"evalue": "configure_lockdown() missing 5 required positional arguments: 'self', 'post_lockdown_interaction_matrix', 'new_case_sims_list', 'new_cases_threshold', and 'new_cases_time_window'",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mTypeError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-17-894c8e60d180>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdir\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mMultiGroupSimulation\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconfigure_lockdown\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mTypeError\u001b[0m: configure_lockdown() missing 5 required positional arguments: 'self', 'post_lockdown_interaction_matrix', 'new_case_sims_list', 'new_cases_threshold', and 'new_cases_time_window'"
]
}
],
"source": [
"dir(MultiGroupSimulation.configure_lockdown())"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"scrolled": true
},
"outputs": [
{
"ename": "TypeError",
"evalue": "configure_lockdown() missing 1 required positional argument: 'new_cases_time_window'",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mTypeError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-11-1be1c1e80f6c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0mstart\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mlockdowns\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfinal_infections\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msimulate_lockdown\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1000\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m7\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 4\u001b[0m \u001b[0mend\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"elapsed time {:.2f}\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mend\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0mstart\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m<ipython-input-10-9d85281ee980>\u001b[0m in \u001b[0;36msimulate_lockdown\u001b[0;34m(contact_multiplier, infections_thresh, infections_window, nreps)\u001b[0m\n\u001b[1;32m 12\u001b[0m \u001b[0mlockdown_matrix\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 13\u001b[0m \u001b[0mnew_cases_thresh\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 14\u001b[0;31m infections_window)\n\u001b[0m",
"\u001b[0;32m<ipython-input-8-9395c849a930>\u001b[0m in \u001b[0;36mevaluate_testing_policy\u001b[0;34m(params_list, interaction_matrix, group_names, test_frac, T, n, post_lockdown_matrix, new_cases_threshold, new_cases_time_window)\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0msim\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mMultiGroupSimulation\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mparams_list\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minteraction_matrix\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgroup_names\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m \u001b[0msim\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconfigure_lockdown\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpost_lockdown_matrix\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnew_cases_threshold\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnew_cases_time_window\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[0minf_matrix\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhosp_matrix\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlockdown_trajectories\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minfection_trajectories\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrun_multiple_trajectories\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msim\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mT\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mTypeError\u001b[0m: configure_lockdown() missing 1 required positional argument: 'new_cases_time_window'"
]
}
],
Expand Down Expand Up @@ -548,7 +570,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "IPython (Python 3)",
"language": "python",
"name": "python3"
},
Expand All @@ -562,7 +584,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.3"
"version": "3.6.2"
}
},
"nbformat": 4,
Expand Down
Loading