Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements to thumbnails code for using remote media storage #1952

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

cjh79
Copy link

@cjh79 cjh79 commented Oct 28, 2019

This PR includes two improvements to the thumbnails code when using remote media storage (eg. S3).

1. Allow absolute path for THUMBNAILS_DIR_NAME

Allow the THUMBNAILS_DIR_NAME setting to specify an absolute path. In this case, THUMBNAILS_DIR_NAME will be created relative to the MEDIA_ROOT directory, rather than relative to each image's directory. Each image's path is recreated below the root THUMBNAILS_DIR_NAME directory to avoid name collisions. Default behavior is not changed, so existing installations are not affected.

THUMBNAILS_DIR_NAME = "/.thumbnails"

Using this feature offers improvements to listdir() on eg. a Boto3 storage backend, which is constrained to pulling all objects under a particular directory (ie. recursively) due to the nature S3-like object storage systems. With many thumbnail images, this can be a major performance hit, for example when browsing the Media Library in admin. By keeping this directory at the root level, outside of the location of media library files (/uploads), thumbnails do not show up in the Media Library at all, mitigating the problem.

2. Use default_storage for thumbnail creation rather than os

Implementation of the fix described here: https://groups.google.com/forum/#!msg/mezzanine-users/WGVaNhD5vRs/7Ktw7-UkosgJ. Thumbnails are now created solely with default_storage rather than os functions, which of course does not work when using S3 for media storage.

Note that this fix also appears to be implemented in a couple of other PRs: #1951 and #1922

…ilesystem directly, in order to allow for remote media storage.
This can be used to aid Media Library performance when using a remote object storage system such as S3.
@cjh79
Copy link
Author

cjh79 commented Oct 30, 2019

Note that I have created a PR in the filebrowser-safe project which corresponds to this change.

stephenmcd/filebrowser-safe#132

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant