Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compile Times + Warp Types #5539

Open
PSeitz opened this issue Nov 4, 2024 · 3 comments
Open

Compile Times + Warp Types #5539

PSeitz opened this issue Nov 4, 2024 · 3 comments
Assignees

Comments

@PSeitz
Copy link
Contributor

PSeitz commented Nov 4, 2024

Quickwit compile times are pretty long, this could by some part be caused by overly complex warp types.
Below is an example of such a type. We probably should break these type chains by boxing them.

Super long type

83 (0.0%, 87.8%) 2 (0.0%, 65.4%) core::ptr::drop_in_place<warp::filter::or::Or<warp::filter::or::Or<warp::filter::or::Or<warp::filter::or::Or<warp::filter::or::Or<warp::filter::or::Or<warp::filter::or::Or<warp::filter::or::Or<warp::filter::or::Or<warp::filter::or::Or<warp::filter::or::Or<warp::filter::or::Or<warp::filter::or::Or<warp::filter::map::Map<warp::filter::and::And<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::get_index_metadata_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::get::{{closure}}::{{closure}}>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_proto::metastore::MetastoreServiceClient>::{{closure}}>>,quickwit_serve::index_api::rest_handler::get_index_metadata>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<quickwit_metastore::metastore::index_metadata::IndexMetadata,quickwit_proto::metastore::MetastoreError>>,warp::filter::map::Map<warp::filter::and::And<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::list_indexes_metadata_handler::__StaticPath>>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::get::{{closure}}::{{closure}}>>,warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::index_api::rest_handler::ListIndexesQueryParams>::{{closure}}>,serde_qs::warp::query<quickwit_serve::index_api::rest_handler::ListIndexesQueryParams>::{{closure}}>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_proto::metastore::MetastoreServiceClient>::{{closure}}>>,quickwit_serve::index_api::rest_handler::list_indexes_metadata>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<alloc::vec::Vec<quickwit_metastore::metastore::index_metadata::IndexMetadata>,quickwit_proto::metastore::MetastoreError>>>,warp::filter::map::Map<warp::filter::and::And<warp::filter::map::Map<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::create_index_handler::__StaticPath>>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::post::{{closure}}::{{closure}}>>,warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::index_api::rest_handler::CreateIndexQueryParams>::{{closure}}>,serde_qs::warp::query<quickwit_serve::index_api::rest_handler::CreateIndexQueryParams>::{{closure}}>>,warp::filter::and_then::AndThen<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::header::optionalmime::Mime::{{closure}},futures_util::future::ready::Ready<core::result::Result<core::option::Optionmime::Mime,warp::reject::Rejection>>>::{{closure}}>,quickwit_serve::format::extract_config_format::{{closure}}>>,warp::filter::untuple_one::UntupleOne<warp::filter::and_then::AndThen<warp::filter::map_err::MapErr<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::header::header2headers::common::content_length::ContentLength::{{closure}},futures_util::future::ready::Ready<core::result::Resultheaders::common::content_length::ContentLength,warp::reject::Rejection>>::{{closure}}>,warp::filters::body::content_length_limit::{{closure}}>,warp::filters::body::content_length_limit::{{closure}}>>>,warp::filter::and_then::AndThen<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::body::body::{{closure}},futures_util::future::ready::Ready<core::result::Resulthyper::body::body::Body,warp::reject::Rejection>>::{{closure}}>,warp::filters::body::bytes::{{closure}}>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_index_management::index::IndexService>::{{closure}}>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<alloc::sync::Arc<quickwit_config::node_config::NodeConfig>>::{{closure}}>>,quickwit_serve::index_api::rest_handler::create_index>,quickwit_serve::index_api::rest_handler::log_failure<quickwit_metastore::metastore::index_metadata::IndexMetadata,quickwit_index_management::index::IndexServiceError>::{{closure}}>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<quickwit_metastore::metastore::index_metadata::IndexMetadata,quickwit_index_management::index::IndexServiceError>>>,warp::filter::map::Map<warp::filter::and::And<warp::filter::map::Map<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::update_index_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::put::{{closure}}::{{closure}}>>,warp::filter::and_then::AndThen<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::header::optionalmime::Mime::{{closure}},futures_util::future::ready::Ready<core::result::Result<core::option::Optionmime::Mime,warp::reject::Rejection>>>::{{closure}}>,quickwit_serve::format::extract_config_format::{{closure}}>>,warp::filter::untuple_one::UntupleOne<warp::filter::and_then::AndThen<warp::filter::map_err::MapErr<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::header::header2headers::common::content_length::ContentLength::{{closure}},futures_util::future::ready::Ready<core::result::Resultheaders::common::content_length::ContentLength,warp::reject::Rejection>>::{{closure}}>,warp::filters::body::content_length_limit::{{closure}}>,warp::filters::body::content_length_limit::{{closure}}>>>,warp::filter::and_then::AndThen<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::body::body::{{closure}},futures_util::future::ready::Ready<core::result::Resulthyper::body::body::Body,warp::reject::Rejection>>::{{closure}}>,warp::filters::body::bytes::{{closure}}>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_proto::metastore::MetastoreServiceClient>::{{closure}}>>,quickwit_serve::index_api::rest_handler::update_index>,quickwit_serve::index_api::rest_handler::log_failure<quickwit_metastore::metastore::index_metadata::IndexMetadata,quickwit_index_management::index::IndexServiceError>::{{closure}}>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<quickwit_metastore::metastore::index_metadata::IndexMetadata,quickwit_index_management::index::IndexServiceError>>>,warp::filter::map::Map<warp::filter::and::And<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::clear_index_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::clear_index_handler::__StaticPath>>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::put::{{closure}}::{{closure}}>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_index_management::index::IndexService>::{{closure}}>>,quickwit_serve::index_api::rest_handler::clear_index>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<(),quickwit_index_management::index::IndexServiceError>>>,warp::filter::map::Map<warp::filter::and::And<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::delete_index_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::delete::{{closure}}::{{closure}}>>,warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::index_api::rest_handler::DeleteIndexQueryParam>::{{closure}}>,serde_qs::warp::query<quickwit_serve::index_api::rest_handler::DeleteIndexQueryParam>::{{closure}}>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_index_management::index::IndexService>::{{closure}}>>,quickwit_serve::index_api::rest_handler::delete_index>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<alloc::vec::Vec<quickwit_metastore::split_metadata::SplitInfo>,quickwit_index_management::index::IndexServiceError>>>,warp::filter::map::Map<warp::filter::and::And<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::list_splits_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::list_splits_handler::__StaticPath>>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::get::{{closure}}::{{closure}}>>,warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::index_api::rest_handler::ListSplitsQueryParams>::{{closure}}>,serde_qs::warp::query<quickwit_serve::index_api::rest_handler::ListSplitsQueryParams>::{{closure}}>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_proto::metastore::MetastoreServiceClient>::{{closure}}>>,quickwit_serve::index_api::rest_handler::list_splits>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<quickwit_serve::index_api::rest_handler::ListSplitsResponse,quickwit_proto::metastore::MetastoreError>>>,warp::filter::map::Map<warp::filter::and::And<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::describe_index_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::describe_index_handler::__StaticPath>>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::get::{{closure}}::{{closure}}>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_proto::metastore::MetastoreServiceClient>::{{closure}}>>,quickwit_serve::index_api::rest_handler::describe_index>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<quickwit_serve::index_api::rest_handler::IndexStats,quickwit_proto::metastore::MetastoreError>>>,warp::filter::map::Map<warp::filter::and::And<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::mark_splits_for_deletion_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::mark_splits_for_deletion_handler::__StaticPath>>>,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::mark_splits_for_deletion_handler::__StaticPath>>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::put::{{closure}}::{{closure}}>>,warp::filter::and::And<warp::filter::untuple_one::UntupleOne<warp::filter::and_then::AndThen<warp::filter::map_err::MapErr<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::header::header2headers::common::content_length::ContentLength::{{closure}},futures_util::future::ready::Ready<core::result::Resultheaders::common::content_length::ContentLength,warp::reject::Rejection>>::{{closure}}>,warp::filters::body::content_length_limit::{{closure}}>,warp::filters::body::content_length_limit::{{closure}}>>,warp::filter::and_then::AndThen<warp::filter::and::And<warp::filter::FilterFn<warp::filters::body::is_content_typewarp::filters::body::Json::{{closure}}>,warp::filter::and_then::AndThen<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::body::body::{{closure}},futures_util::future::ready::Ready<core::result::Resulthyper::body::body::Body,warp::reject::Rejection>>::{{closure}}>,warp::filters::body::bytes::{{closure}}>>,warp::filters::body::json<quickwit_serve::index_api::rest_handler::SplitsForDeletion>::{{closure}}>>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_proto::metastore::MetastoreServiceClient>::{{closure}}>>,quickwit_serve::index_api::rest_handler::mark_splits_for_deletion>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<(),quickwit_proto::metastore::MetastoreError>>>,warp::filter::map::Map<warp::filter::and::And<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::reset_source_checkpoint_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::reset_source_checkpoint_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::reset_source_checkpoint_handler::__StaticPath>>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::put::{{closure}}::{{closure}}>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_proto::metastore::MetastoreServiceClient>::{{closure}}>>,quickwit_serve::index_api::rest_handler::reset_source_checkpoint>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<(),quickwit_proto::metastore::MetastoreError>>>,warp::filter::map::Map<warp::filter::and::And<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::toggle_source_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::toggle_source_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::toggle_source_handler::__StaticPath>>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::put::{{closure}}::{{closure}}>>,warp::filter::and::And<warp::filter::untuple_one::UntupleOne<warp::filter::and_then::AndThen<warp::filter::map_err::MapErr<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::header::header2headers::common::content_length::ContentLength::{{closure}},futures_util::future::ready::Ready<core::result::Resultheaders::common::content_length::ContentLength,warp::reject::Rejection>>::{{closure}}>,warp::filters::body::content_length_limit::{{closure}}>,warp::filters::body::content_length_limit::{{closure}}>>,warp::filter::and_then::AndThen<warp::filter::and::And<warp::filter::FilterFn<warp::filters::body::is_content_typewarp::filters::body::Json::{{closure}}>,warp::filter::and_then::AndThen<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::body::body::{{closure}},futures_util::future::ready::Ready<core::result::Resulthyper::body::body::Body,warp::reject::Rejection>>::{{closure}}>,warp::filters::body::bytes::{{closure}}>>,warp::filters::body::json<quickwit_serve::index_api::rest_handler::ToggleSource>::{{closure}}>>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_proto::metastore::MetastoreServiceClient>::{{closure}}>>,quickwit_serve::index_api::rest_handler::toggle_source>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<(),quickwit_index_management::index::IndexServiceError>>>,warp::filter::map::Map<warp::filter::and::And<warp::filter::map::Map<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::create_source_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::create_source_handler::__StaticPath>>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::post::{{closure}}::{{closure}}>>,warp::filter::and_then::AndThen<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::header::optionalmime::Mime::{{closure}},futures_util::future::ready::Ready<core::result::Result<core::option::Optionmime::Mime,warp::reject::Rejection>>>::{{closure}}>,quickwit_serve::format::extract_config_format::{{closure}}>>,warp::filter::untuple_one::UntupleOne<warp::filter::and_then::AndThen<warp::filter::map_err::MapErr<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::header::header2headers::common::content_length::ContentLength::{{closure}},futures_util::future::ready::Ready<core::result::Resultheaders::common::content_length::ContentLength,warp::reject::Rejection>>::{{closure}}>,warp::filters::body::content_length_limit::{{closure}}>,warp::filters::body::content_length_limit::{{closure}}>>>,warp::filter::and_then::AndThen<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::body::body::{{closure}},futures_util::future::ready::Ready<core::result::Resulthyper::body::body::Body,warp::reject::Rejection>>::{{closure}}>,warp::filters::body::bytes::{{closure}}>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_index_management::index::IndexService>::{{closure}}>>,quickwit_serve::index_api::rest_handler::create_source>,quickwit_serve::index_api::rest_handler::log_failure<quickwit_config::source_config::SourceConfig,quickwit_index_management::index::IndexServiceError>::{{closure}}>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<quickwit_config::source_config::SourceConfig,quickwit_index_management::index::IndexServiceError>>>,warp::filter::map::Map<warp::filter::and::And<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::get_source_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::get_source_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::get::{{closure}}::{{closure}}>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_proto::metastore::MetastoreServiceClient>::{{closure}}>>,quickwit_serve::index_api::rest_handler::get_source>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<quickwit_config::source_config::SourceConfig,quickwit_proto::metastore::MetastoreError>>>,warp::filter::map::Map<warp::filter::and::And<warp::filter::then::Then<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<warp::filters::any::Any,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::delete_source_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filters::path::Exact<warp::filters::path::internal::Opaque<quickwit_serve::index_api::rest_handler::delete_source_handler::__StaticPath>>>,warp::filter::FilterFn<warp::filters::path::filter_segment<warp::filters::path::paramalloc::string::String::{{closure}},(alloc::string::String,)>::{{closure}}>>,warp::filter::FilterFnwarp::filters::path::end::{{closure}}>,warp::filter::FilterFn<warp::filters::method::method_iswarp::filters::method::delete::{{closure}}::{{closure}}>>,warp::filter::map::Map<warp::filters::any::Any,quickwit_serve::with_arg<quickwit_proto::metastore::MetastoreServiceClient>::{{closure}}>>,quickwit_serve::index_api::rest_handler::delete_source>,warp::filter::map::Map<warp::filter::and_then::AndThen<warp::filter::or_else::OrElse<warp::filter::FilterFn<warp::filter::filter_fn_one<warp::filters::query::raw::{{closure}},futures_util::future::ready::Ready<core::result::Resultalloc::string::String,warp::reject::Rejection>>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,serde_qs::warp::query<quickwit_serve::format::FormatQueryString>::{{closure}}>,quickwit_serve::format::extract_format_from_qs::{{closure}}>>,quickwit_serve::rest_api_response::into_rest_api_response<(),quickwit_index_management::index::IndexServiceError>>>>

@fulmicoton
Copy link
Contributor

Yeah boxing would probably help. Can you experiment a little?

PSeitz added a commit that referenced this issue Nov 5, 2024
box warp routes to avoid super heavy types.
Performance impact should be okay, since these heavy types
are not cache friendly.

This reduces type complexity significantly, but some really long type chains remain.

I've identified two, but they may be more
1. Boxed or chains
2. Response type chains

We can replace the boxed or chains with a Vec, since they all contain the same
box type now. Such a type doesn't seem to exist in warp yet.

`cargo install --path .`
Compile time before
4m49s
Compile time now
4m14s

Tool to list heavy functions:
`
CARGO_PROFILE_RELEASE_LTO=fat cargo llvm-lines --release --bin quickwit > llvm_lines
`
Size down from 32MB to 22MB

addresses #5539
@PSeitz PSeitz mentioned this issue Nov 5, 2024
PSeitz added a commit that referenced this issue Nov 5, 2024
box warp routes to avoid super heavy types.
Performance impact should be okay, since these heavy types
are not cache friendly.

This reduces type complexity significantly, but some really long type chains remain.

I've identified two, but they may be more
1. Boxed or chains
2. Response type chains

We can replace the boxed or chains with a Vec, since they all contain the same
box type now. Such a type doesn't seem to exist in warp yet.

`cargo install --path .`
Compile time before
4m49s
Compile time now
4m14s

Tool to list heavy functions:
`
CARGO_PROFILE_RELEASE_LTO=fat cargo llvm-lines --release --bin quickwit > llvm_lines
`
Size down from 32MB to 22MB

addresses #5539
PSeitz added a commit that referenced this issue Nov 5, 2024
box warp routes to avoid super heavy types.
Performance impact should be okay, since these heavy types
are not cache friendly.

This reduces type complexity significantly, but some really long type chains remain.

I've identified two, but they may be more
1. Boxed or chains
2. Response type chains

We can replace the boxed or chains with a Vec, since they all contain the same
box type now. Such a type doesn't seem to exist in warp yet.

`cargo install --path .`
Compile time before
4m49s
Compile time now
4m14s

Tool to list heavy functions:
`
CARGO_PROFILE_RELEASE_LTO=fat cargo llvm-lines --release --bin quickwit > llvm_lines
`
Size down from 32MB to 22MB

addresses #5539
PSeitz added a commit that referenced this issue Nov 5, 2024
box warp routes to avoid super heavy types.
Performance impact should be okay, since these heavy types
are not cache friendly.

This reduces type complexity significantly, but some really long type chains remain.

I've identified two, but they may be more
1. Boxed or chains
2. Response type chains

We can replace the boxed or chains with a Vec, since they all contain the same
box type now. Such a type doesn't seem to exist in warp yet.

`cargo install --path .`
Compile time before
4m49s
Compile time now
4m14s

Tool to list heavy functions:
`
CARGO_PROFILE_RELEASE_LTO=fat cargo llvm-lines --release --bin quickwit > llvm_lines
`
Size down from 32MB to 22MB

addresses #5539
PSeitz added a commit that referenced this issue Nov 5, 2024
box warp routes to avoid super heavy types.
Performance impact should be okay, since these heavy types
are not cache friendly.

This reduces type complexity significantly, but some really long type chains remain.

I've identified two, but they may be more
1. Boxed or chains
2. Response type chains

We can replace the boxed or chains with a Vec, since they all contain the same
box type now. Such a type doesn't seem to exist in warp yet.

`cargo install --path .`
Compile time before
4m49s
Compile time now
4m14s

Tool to list heavy functions:
`
CARGO_PROFILE_RELEASE_LTO=fat cargo llvm-lines --release --bin quickwit > llvm_lines
`
Size down from 32MB to 22MB

addresses #5539
@fulmicoton
Copy link
Contributor

@PSeitz can we close this?

@PSeitz
Copy link
Contributor Author

PSeitz commented Nov 15, 2024

It's improved a lot with #5540, but some really long type chains remain.

Boxed or chains
Response type chains

We can replace the boxed or chains with a Vec, since they all contain the same
box type now. Such a type doesn't seem to exist in warp yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants