Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SIGMOD Revision Experiments #610

Merged
merged 53 commits into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
d6470de
make experiments runnable again
MaxiBoether Sep 2, 2024
cf97e8c
lay out experiments
MaxiBoether Sep 2, 2024
85d5965
small fix
MaxiBoether Sep 2, 2024
85aadd8
Merge remote-tracking branch 'origin/main' into feature/MaxiBoether/s…
MaxiBoether Sep 2, 2024
f183eb6
set more params; wait for #595
MaxiBoether Sep 2, 2024
7dc4141
Merge remote-tracking branch 'origin/main' into feature/MaxiBoether/s…
MaxiBoether Sep 2, 2024
d6bdcbf
test
MaxiBoether Sep 2, 2024
58d9565
slight updates
MaxiBoether Sep 2, 2024
0032362
they see me rollin
MaxiBoether Sep 2, 2024
957d2dd
further reduce
MaxiBoether Sep 2, 2024
898fb82
initial arxiv test
MaxiBoether Sep 3, 2024
5ee0cab
start ts
MaxiBoether Sep 3, 2024
f08e7d1
fix stupid mistake
MaxiBoether Sep 3, 2024
a552da4
fix
MaxiBoether Sep 3, 2024
72553c2
Merge branch 'fix/MaxiBoether/dyn-perf-fcast' into feature/MaxiBoethe…
MaxiBoether Sep 3, 2024
1959fbd
Merge remote-tracking branch 'origin/main' into feature/MaxiBoether/s…
MaxiBoether Sep 4, 2024
a7a6efa
full arxiv experiments; fix yearbook
MaxiBoether Sep 4, 2024
82f13bf
start timestamp for warmup triggers
MaxiBoether Sep 4, 2024
7be88a0
cnsistent eval intervals across performance / drift triggers
MaxiBoether Sep 4, 2024
12a38b7
Merge remote-tracking branch 'origin/main' into feature/MaxiBoether/s…
MaxiBoether Sep 8, 2024
d4142ad
Merge remote-tracking branch 'origin/main' into feature/MaxiBoether/s…
MaxiBoether Sep 10, 2024
4da52f0
fix
MaxiBoether Sep 10, 2024
4ca1b21
Merge remote-tracking branch 'origin/main' into feature/MaxiBoether/s…
MaxiBoether Sep 10, 2024
5952d8a
Merge remote-tracking branch 'origin/main' into feature/MaxiBoether/s…
MaxiBoether Sep 10, 2024
3bda225
Merge remote-tracking branch 'origin/main' into feature/MaxiBoether/s…
MaxiBoether Sep 10, 2024
d7c0710
Update yb triggering
MaxiBoether Sep 10, 2024
2f56a95
Yearbook fix and arXiv update
MaxiBoether Sep 10, 2024
01004de
Updates for arXiv
MaxiBoether Sep 11, 2024
5395598
Merge remote-tracking branch 'origin/main' into feature/MaxiBoether/s…
MaxiBoether Sep 12, 2024
dd497a2
Update to quantiles
MaxiBoether Sep 12, 2024
2815295
first retry version
MaxiBoether Sep 20, 2024
2fee48b
improve check at evaluator
MaxiBoether Sep 20, 2024
4e3564d
retry on exception at evaluator
MaxiBoether Sep 20, 2024
b5d28cd
minor
MaxiBoether Sep 20, 2024
9a2495a
Merge branch 'main' into feature/MaxiBoether/emptyevals
MaxiBoether Sep 23, 2024
ba26601
Merge branch 'feature/MaxiBoether/emptyevals' into feature/MaxiBoethe…
MaxiBoether Sep 23, 2024
fe537a8
find invalid runs notebook
MaxiBoether Sep 23, 2024
5cd0131
find invalid runs notebook
MaxiBoether Sep 23, 2024
5f23bc8
Some fixes
MaxiBoether Sep 23, 2024
48cc329
Merge branch 'feature/MaxiBoether/emptyevals' into feature/MaxiBoethe…
MaxiBoether Sep 23, 2024
e356273
fix some evaluator tets to new valid meaning
MaxiBoether Sep 23, 2024
9f5178f
only pop if in there to avoid exception
MaxiBoether Sep 24, 2024
c9cefce
Merge branch 'feature/MaxiBoether/emptyevals' into feature/MaxiBoethe…
MaxiBoether Sep 24, 2024
6cd49e1
fix more tests
MaxiBoether Sep 24, 2024
7d4c548
fix pylint
MaxiBoether Sep 24, 2024
8efde93
Merge branch 'feature/MaxiBoether/emptyevals' into feature/MaxiBoethe…
MaxiBoether Sep 24, 2024
b255971
fix pylint
MaxiBoether Sep 24, 2024
df6e0cc
Plots for revision
MaxiBoether Oct 7, 2024
90f4058
heatmap
MaxiBoether Oct 14, 2024
0b099ae
more notebooks
MaxiBoether Oct 14, 2024
0295235
port from main
MaxiBoether Oct 14, 2024
fa54ff3
Merge remote-tracking branch 'origin/main' into feature/MaxiBoether/s…
MaxiBoether Oct 14, 2024
bc7ed02
Fix mypy
MaxiBoether Oct 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# Images
*.svg
*.png

# Logging files
*.log

Expand Down
8 changes: 4 additions & 4 deletions analytics/app/pages/plots/cost_vs_eval_metric_agg.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from dataclasses import dataclass
from typing import get_args
from typing import Any, get_args

import pandas as pd
import plotly.express as px
Expand Down Expand Up @@ -32,9 +32,9 @@ class _PageState:

def gen_fig_scatter_num_triggers(
page: str,
eval_handler: str,
dataset_id: str,
metric: str,
eval_handler: str | Any | None,
dataset_id: str | Any | None,
metric: str | Any | None,
agg_func_x: AGGREGATION_FUNCTION,
agg_func_y: EVAL_AGGREGATION_FUNCTION,
stages: list[str],
Expand Down
7 changes: 4 additions & 3 deletions analytics/app/pages/plots/eval_heatmap.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from dataclasses import dataclass
from typing import Any

import pandas as pd
from dash import Input, Output, callback, dcc, html
Expand Down Expand Up @@ -31,9 +32,9 @@ def gen_figure(
page: str,
multi_pipeline_mode: bool,
patch_yearbook: bool,
eval_handler: str,
dataset_id: str,
metric: str,
eval_handler: str | Any | None,
dataset_id: str | Any | None,
metric: str | Any | None,
) -> go.Figure:
"""Create the cost over time figure with barplot or histogram. Histogram
has nice binning while barplot is precise.
Expand Down
7 changes: 4 additions & 3 deletions analytics/app/pages/plots/eval_over_time.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from dataclasses import dataclass
from typing import Any

import pandas as pd
import plotly.express as px
Expand Down Expand Up @@ -29,9 +30,9 @@ def gen_figure(
page: str,
multi_pipeline_mode: bool,
patch_yearbook: bool,
eval_handler: str,
dataset_id: str,
metric: str,
eval_handler: str | Any | None,
dataset_id: str | Any | None,
metric: str | Any | None,
) -> go.Figure:
"""Create the evaluation over time figure with a line plot.

Expand Down
10 changes: 5 additions & 5 deletions analytics/app/pages/plots/num_samples.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,12 @@ class _PageState:
def gen_figure(
page: str,
multi_pipeline_mode: bool,
time_metric: str,
y_axis: YAxis,
use_scatter_size: bool,
time_metric: str | None,
y_axis: YAxis | None,
use_scatter_size: bool | None,
patch_yearbook: bool,
dataset_id: str,
eval_handler: str,
dataset_id: str | None,
eval_handler: str | None,
) -> go.Figure:
"""Create the cost over time figure with barplot or histogram. Histogram
has nice binning while barplot is precise.
Expand Down
6 changes: 3 additions & 3 deletions analytics/app/pages/plots/num_triggers_eval_metric.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ class _PageState:
def gen_fig_scatter_num_triggers(
page: str,
multi_pipeline_mode: bool,
eval_handler: str,
dataset_id: str,
metric: str,
eval_handler: str | None,
dataset_id: str | None,
metric: str | None,
aggregate_metric: bool = True,
time_weighted: bool = True,
only_active_periods: bool = True,
Expand Down
Loading
Loading