Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Paging #630

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open

Paging #630

wants to merge 18 commits into from

Conversation

matt-ramotar
Copy link
Collaborator

@matt-ramotar matt-ramotar commented Mar 16, 2024

Context:

Description

Introducing a solution for paging in KMP projects. Motivations:

  1. Seamless integration with Store and Mutable Store
  2. Support for local mutations and streaming of child items within the list of paging items
  3. Support for custom reducer, middleware, and post-reducer effects

Type of Change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Test Plan

Unit tests

Checklist:

Before submitting your PR, please review and check all of the following:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my change is effective
  • New and existing unit tests pass locally with my changes

Additional Notes:

Paging Technical Design Doc

1. Overview

Modular and flexible architecture. Using builder, reducer, middleware, and post-reducer effect patterns. Unidirectional
data flow. The Pager is the main component. Actions are dispatched through the Pager. The PagerBuilder creates
the Pager. It allows configuration of the paging behavior. The PagingSource defines the data loading logic. The FetchingStrategy determines when to fetch the next page. The AggregatingStrategy combines loaded pages into a single list. The Reducer handles state changes based on actions. When an action is dispatched, it goes through the middleware pipeline. The middleware can modify the action. The reducer then updates the state based on the action. After the reducer, we invoke post-reducer effects associated with the action and new state. The updated state is sent back the Pager and emitted to the UI.

2. Key Components

  • Pager: The main entry point for the paging library. It coordinates the paging process and provides access to the paging state and data.
  • PagingState: Represents the current state of the paging data, including loaded pages, errors, and loading status.
  • PagingAction: Defines the actions that can be dispatched to modify the paging state.
  • Reducer: Responsible for taking the current paging state and a dispatched action, and producing a new paging state based on the action and the current state.
  • Middleware: Intercepts and modifies paging actions before they reach the reducer, allowing for pre-processing, logging, or any other custom logic.
  • Effect: Represents side effects or additional actions that need to be performed after the state has been reduced based on a dispatched action.
  • PagingSource: Represents a data source that provides paged data, emitting a stream of load results.
  • PagingBuffer: A custom data structure for efficiently storing and retrieving paging data.

3. Customizations

Providing many extension points and customization options to tailor behavior. Main customization points:

  • PagingConfig: Allows configuring the paging behavior, such as page size, prefetch distance, and insertion strategy.
  • FetchingStrategy: Determines whether to fetch more data based on the current state of the pager.
  • AggregatingStrategy: Defines how loaded pages of data should be combined and ordered to form a coherent list of paging items.
  • ErrorHandlingStrategy: Specifies different strategies for handling errors during the paging process.
  • UserCustomActionReducer: Allows defining custom reducers for handling user-defined actions.

4. Data Flow

Unidirectional data flow. Main steps:

  1. Pager is configured using PagerBuilder and provided an initial key, flow of anchor position, and paging config.
  2. Pager subscribes to the PagingSource to receive paging data updates.
  3. When a PagingAction is dispatched, it goes through the configured Middleware chain. This enables interception and modification of the action.
  4. The modified action reaches the Reducer, which reduces the current PagingState based on the action and returns a new PagingState.
  5. After reduction, any configured Effect instances are launched, enabling side effects to be performed based on the new PagingState.
  6. Pager updates StateManager with the new PagingState.
  7. FetchingStrategy determines when to fetch the next page of data based on the PagingConfig and current PagingState.
  8. When a new page needs to be fetched, QueueManager enqueues the page key, and the JobCoordinator coordinates the execution of the paging job.
  9. PagingSource loads the requested page and emits the loaded data through the PagingSourceStreamProvider.
  10. The loaded page is stored in the MutablePagingBuffer for efficient retrieval and aggregation.
  11. The AggregatingStrategy aggregates the loaded pages into a single list, which is then emitted through the Pager for consumption by the UI.

5. Sample Code

See https://github.com/MobileNativeFoundation/Store/tree/paging/paging

Signed-off-by: mramotar <[email protected]>
* Define PagingBuffer

Signed-off-by: mramotar <[email protected]>

* Define MutablePagingBuffer

Signed-off-by: mramotar <[email protected]>

---------

Signed-off-by: mramotar <[email protected]>
Signed-off-by: mramotar <[email protected]>
* Implement RealDispatcher and Related Classes

Signed-off-by: mramotar <[email protected]>

* Implement RealOptionalInjector

Signed-off-by: mramotar <[email protected]>

* Implement DefaultLogger

Signed-off-by: mramotar <[email protected]>

---------

Signed-off-by: mramotar <[email protected]>
Signed-off-by: mramotar <[email protected]>
Signed-off-by: mramotar <[email protected]>

sourceSets {
val commonMain by getting {
dependencies {
implementation(libs.kotlin.stdlib)
implementation(project(":store"))
implementation(project(":cache"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is cache not needed anymore?

@@ -57,27 +44,11 @@ kotlin {
}

android {
namespace = "org.mobilenativefoundation.store.paging5"
namespace = "org.mobilenativefoundation.paging.core"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can also do org.mobilenativefoundation.store5.paging.core

in case you want to ever have something like 5/6 at same time

* @param P The type of the parameters associated with each page of data.
* @param D The type of the data items.
*/
interface AggregatingStrategy<Id : Comparable<Id>, K : Any, P : Any, D : Any> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still prefer Key,Request,Response as the param names :-)

@OliverRhyme
Copy link

Any updates on this?

@matt-ramotar
Copy link
Collaborator Author

@OliverRhyme sorry to be slow. Update here: #671 (comment)

@tyvsmith
Copy link
Member

A few high level comments:

  1. Are there any pagination standards that should be implementing or following by default, ie IETF cursor based pagination?
  2. Do you need to supply any specific test utils or fakes for the end-user to idiomatically test their pagination impls?
  3. There is limited use of internal or private visibility, even in impl package. Should everything be visible to the consumer?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 👀 In review
Development

Successfully merging this pull request may close these issues.

Improve StoreKey API [BUG] State Flow From launchPagingStore Does Not Reflect Latest Writes in Mutable Store
4 participants