Improvements to thumbnails code for using remote media storage #1952
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR includes two improvements to the thumbnails code when using remote media storage (eg. S3).
1. Allow absolute path for THUMBNAILS_DIR_NAME
Allow the
THUMBNAILS_DIR_NAME
setting to specify an absolute path. In this case,THUMBNAILS_DIR_NAME
will be created relative to theMEDIA_ROOT
directory, rather than relative to each image's directory. Each image's path is recreated below the rootTHUMBNAILS_DIR_NAME
directory to avoid name collisions. Default behavior is not changed, so existing installations are not affected.Using this feature offers improvements to
listdir()
on eg. a Boto3 storage backend, which is constrained to pulling all objects under a particular directory (ie. recursively) due to the nature S3-like object storage systems. With many thumbnail images, this can be a major performance hit, for example when browsing the Media Library in admin. By keeping this directory at the root level, outside of the location of media library files (/uploads
), thumbnails do not show up in the Media Library at all, mitigating the problem.2. Use default_storage for thumbnail creation rather than os
Implementation of the fix described here: https://groups.google.com/forum/#!msg/mezzanine-users/WGVaNhD5vRs/7Ktw7-UkosgJ. Thumbnails are now created solely with
default_storage
rather thanos
functions, which of course does not work when using S3 for media storage.Note that this fix also appears to be implemented in a couple of other PRs: #1951 and #1922