Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simplify progress bars, apply to all large operations #190

Open
robfitzgerald opened this issue Apr 19, 2024 · 0 comments
Open

simplify progress bars, apply to all large operations #190

robfitzgerald opened this issue Apr 19, 2024 · 0 comments
Labels
good first issue Good for newcomers rust Applies to the rust code

Comments

@robfitzgerald
Copy link
Collaborator

Compass runs are more informative with progress bars. For example, users get a heads up while the graph and geometries are loaded:

edge list: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 124346940/124346940 [02:47<00:00, 743093.62it/s]
vertex list: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 56306871/56306871 [01:13<00:00, 767702.94it/s]
geometry file: 100%|█████████████████████████████████████████████████████████████████████████████████████| 124379003/124379003 [04:54<00:00, 422744.47it/s]

the implementation is somewhat messy. the resource management of creating progress bars and printing a final newline is leaked to the user of read_utils methods via a callback:

pub fn from_csv<'a, T>(
    filepath: &dyn AsRef<Path>,
    has_headers: bool,
    row_callback: RowCallback<'a, T>,
) -> Result<Box<[T]>, csv::Error>
where
    T: serde::de::DeserializeOwned + 'a,
{ }

there's two spots in the edge rtree input plugin that don't give us good progress bar behavior:

  • loading the road class file doesn't have a progress bar
  • read_linestring_text_file doesn't call a final println!() to force a newline once the progress bar finishes; whatever is printed next overwrites the progress bar in stdout

a few things could be done here:

  1. move progress bar instantiation inside of read_utils operations, make optional via function args (pb_title: Option<String>)
    - without disrupting the availability of a row_callback: type RowCallback<'a, T> = Option<Box<dyn FnMut(&T) + 'a>>
    - updating all call sites in the repo so we are creating file loading progress bars
  2. create a newline at the end of geo_io_utils::read_linestring_text_file
  3. review: are there other big O(n) operations during instantiation of Compass, like creating an rtree, that are missing a progress bar?
@robfitzgerald robfitzgerald added the good first issue Good for newcomers label Apr 19, 2024
@nreinicke nreinicke added the rust Applies to the rust code label May 7, 2024
@kylecarow kylecarow assigned kylecarow and unassigned kylecarow May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers rust Applies to the rust code
Projects
None yet
Development

No branches or pull requests

3 participants