You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Following the logic used for efficient copying in the copy_node functionality in Studio, ricecooker should batch upload subsections of the overall node tree with full associated file and assessment item metadata.
This would mean that rather than waiting for an API call for every single node, batches of ~1000 could be sent at a time, and an asynchronous task then set in motion on Studio to do the writing to the database.
Ricecooker would poll Studio at a low frequency interval (~1 minute, but could be tuned) to see when the task completed, then follow up with subsequent updates. For parallelization, different child node subsections that are independent of each other could be POSTed in parallel, relying on Studio's mptt locking mechanisms to ensure correct ordering and tree value calculation. However, to ensure correct ordering, direct siblings should be POSTed together.
The 'longer range' suggestion of parsing SQLite DBs from ricecooker would rely on the restorechannel command on Studio being reliable and fast. It is neither, and is likely not to be either unless it uses the Kolibri import mechanisms, so it seems more sustainable to take the 'short term' approach that is scalable and easy to implement.
The text was updated successfully, but these errors were encountered:
Following the logic used for efficient copying in the copy_node functionality in Studio, ricecooker should batch upload subsections of the overall node tree with full associated file and assessment item metadata.
This would mean that rather than waiting for an API call for every single node, batches of ~1000 could be sent at a time, and an asynchronous task then set in motion on Studio to do the writing to the database.
Ricecooker would poll Studio at a low frequency interval (~1 minute, but could be tuned) to see when the task completed, then follow up with subsequent updates. For parallelization, different child node subsections that are independent of each other could be POSTed in parallel, relying on Studio's mptt locking mechanisms to ensure correct ordering and tree value calculation. However, to ensure correct ordering, direct siblings should be POSTed together.
Previous discussion of this is here: #231 (comment)
The 'longer range' suggestion of parsing SQLite DBs from ricecooker would rely on the
restorechannel
command on Studio being reliable and fast. It is neither, and is likely not to be either unless it uses the Kolibri import mechanisms, so it seems more sustainable to take the 'short term' approach that is scalable and easy to implement.The text was updated successfully, but these errors were encountered: