Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix fullPage rendering and loading #65

Closed
wants to merge 4 commits into from
Closed
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 28 additions & 27 deletions index.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
/* global document */
/* global document, window */
'use strict';
const {promisify} = require('util');
const fs = require('fs');
Expand Down Expand Up @@ -319,36 +319,37 @@ const captureWebsite = async (input, options) => {
}

if (screenshotOptions.fullPage) {
// Get the height of the rendered page
const bodyHandle = await page.$('body');
const bodyBoundingHeight = await bodyHandle.boundingBox();
await bodyHandle.dispose();

// Scroll one viewport at a time, pausing to let content load
const viewportHeight = viewportOptions.height;
let viewportIncrement = 0;
while (viewportIncrement + viewportHeight < bodyBoundingHeight) {
Copy link
Contributor Author

@brandon93s brandon93s Nov 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This loop was inaccessible before due to bodyBoundingHeight being an object. autoScroll below accomplishes the same goal of scrolling to force content load.

const navigationPromise = page.waitForNavigation({waitUntil: 'networkidle0'});
/* eslint-disable no-await-in-loop */
await page.evaluate(_viewportHeight => {
/* eslint-disable no-undef */
window.scrollBy(0, _viewportHeight);
/* eslint-enable no-undef */
}, viewportHeight);
await navigationPromise;
/* eslint-enable no-await-in-loop */
viewportIncrement += viewportHeight;
const autoScroll = async () => {
const isBottom = await page.evaluate(() => {
window.scrollBy(0, window.innerHeight);
return window.scrollY >= document.body.clientHeight - window.innerHeight;
});

await page.waitForFunction(imagesHaveLoaded, {timeout: timeoutInSeconds});

return !isBottom;
};

while (await autoScroll()) { /* eslint-disable-line no-await-in-loop */
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this have a timeout? Otherwise, it would just go on forever on sites with infinite auto-loading.

// noop
}

// Scroll back to top
await page.evaluate(_ => {
/* eslint-disable no-undef */
window.scrollTo(0, 0);
/* eslint-enable no-undef */
// Workaround for chromium height limitations: https://bugs.chromium.org/p/chromium/issues/detail?id=770769#c12
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Workaround for chromium height limitations: https://bugs.chromium.org/p/chromium/issues/detail?id=770769#c12
// Workaround for Chromium height limitations: https://bugs.chromium.org/p/chromium/issues/detail?id=770769#c12

const height = await page.evaluate(() => document.documentElement.scrollHeight);
const maxTextureSize = await page.evaluate(() => {
const canvas = document.createElement('canvas');
const webGL = canvas.getContext('webgl');
return webGL.getParameter(webGL.MAX_TEXTURE_SIZE);
});
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAX_TEXTURE_SIZE is the number of pixels we can go in any dimmension before duplicate rendering will occur. This is determined at runtime as it can/will be different per environment. Using this, we can adjust our deviceScaleFactor to fit the page within this dimension.

Note: The same issue can technically happen horizontally. If there's a use case out there, we should scroll and scale in both dimensions ( maybe a follow-up PR? ).


// Some extra delay to let images load
await page.waitForFunction(imagesHaveLoaded, {timeout: timeoutInSeconds});
// Adjust screenshot to fit entire page within max canvas dimensions
const maxScaleFactor = Number.parseFloat((maxTextureSize / height).toFixed(2));
if (viewportOptions.deviceScaleFactor > maxScaleFactor) {
await page.setViewport({
...viewportOptions,
deviceScaleFactor: maxScaleFactor
});
}
}

const buffer = await page.screenshot(screenshotOptions);
Expand Down