Skip to content

Latest commit

 

History

History
1500 lines (1088 loc) · 46.5 KB

ch4.asciidoc

File metadata and controls

1500 lines (1088 loc) · 46.5 KB

Grow with Google Challenge Notes: Chapter 4

IndexedDB and Caching notes
To go back to the README where all the chapters are: click here.

Table of Contents

4.1 : Introducing the IDB Promised Library

  • When a user opens wittr, we want to start by showing the latest posts the device received, before going to the network.

  • We make a web socket connection and we start receiving new posts one by one.

  • When we receive these posts, we want to display them, but we also want to add them to the set of posts we already have stored.

  • We also want to remove entries that are too old to be worth keeping.

Introducing Databases

Databases are great for adding and removing individual posts, iterate over them, and query the data.

The web platform has a database called IndexedDB. We will be learning with that for the offline database approach. If we’ve used noSQL before, indexeddb will be familiar. If we’ve used relational databases before, then this will be a little weird.

IDB can have many different databases with any name we give them, but we’ll only be creating one for this lesson.

  • It’s best to only have one database per app.

  • The one data will have multiple objects stores, generally one for each kind of thing you want to store.

  • The object store contains multiple values.

  • The values can be JavaScript objects, strings, numbers, dates, or arrays.

  • The items in the object’s store can have a separate primary key or you can assign a property of the values to be the key.

Example:

Out-of-line Keys

In-line Keys

Key

Value

Entries

Jane

41

{name: 'Jane', age: 41}

Natalie

24

{name: 'Natalie', age: 24}

Pete

32

{name: 'Pete', age: 32}

  • The key must be unique within an object store because it is the way to identify a particular object.

  • Later we can get, set, add, remove, iterate over items in object stores as part of a transaction.

  • All read or write operations in indexDB must be part of a transaction.

Transaction:

Here’s an example of a series of steps: * [ ] Add "hello:world" to "keyval' store. * [ ] Get the first value from the "people" store * [ ] Change the person’s name to "james" * [ ] Wwrite it back to the "people" store.

If one of these steps fail, none of them will be applied. The state of the database would be as if none of the steps happened.

We can also create indexes within an object store, which provides a different view of the same data ordered by particular properties.

People Store

…​indexed by age

Entries

Entries

{name: 'Jane', age 41}

{age: 24, name: 'Natalie'}

{name: 'Natalie', age: 24}

{age: 32, name: 'Pete'}

{name: 'Pete', age: 32}

{age: 41, name: 'Jane'}

Example here is similar to a lot of databases which makes a lot of sense. The browser support is good as well with every major browser supporting it.

IndexedDB’s bad reputation

The API for IDB is horrid, and creates spaghetti code. Though it is asynchronous which is fine, but it predates promises. It created its own event-based promise system which creates really confusing code.

The instructor says he’s going to chicken out of teaching us the IDB’s API. So instead we’re going to use IndexDB Promised. https://github.com/jakearchibald/idb/blob/master/README.md

It’s a small library that mirrors the IndexDB API, but uses promises rather than events. Other than that it’s the same as IndexedDB.

4.2 : Getting STarted with IDB

First you head over to http://localhost:8888/idb-test/
It should be a blank page. The script for idb-test is in public > js > idb-test > index.js

All that is in there is an import for the idb library that we saw before.

import idb from 'idb';

Creating our database

To create a database, we use

idb.open(name, version, upgradeCallback)

idb.open() takes in 3 parameters:

  • name

  • version

  • upgradeCallback - a callback to set the database up.

Creating our Database START

under the import, we create our database

idb.open('test-db', 1, function(upgradeDb) {})

  • The function will be called if the browser hasn’t heard about this database before or if the version it knows about is less than this numbe here.

  • The function uses the parameter upgradeDb which we use to define the database.

  • To ensure the DB integrity, this is the only place we can create and remove object stores and indexes.

Creating an object store

The original syntax for creating goes something like this:

var objectStore = db.createObjectStore("toDoList", { keyPath: "taskTitle" });

The original syntax for adding an item inside.

var request = objectStore.put(myItem, optionalKey);
Note
 It is value, key instead of the usual key, value.

For our project, we will create an object store called keyVal. This store has a key that’s separate to the data and does this by default, which is what we want for a keyValStore.

var keyVal = upgradeDb.createObjectStore('keyval');

We want to add some content.

In the library docs that an object store has methods which behave the same as IDB, except they return a promise. The library is way more usable than plain IDB.

keyValStore.put('world', 'hello')

  • We finished setting up our database. .open returns a promise that resolves with a database object.

  • Jake stored the database in the variable dbPromise. Now we can use that database object to get and set items in the database.

Here’s how it would look like alltogether so far:

const dbPromise = idb.open('test-db', 1, (upgradeDb) => {
  const keyValStore = upgradeDb.createObjectStore('keyval');
  keyValStore.put("world", "hello");
  return;
});

and in dev tools the result should look like this:

idb1

Reading the Database

So now for reading the database!

  • we need to create a transaction. The function to do this is db.transaction() with the keyval object store.

const tx = db.transaction('keyval');

  • Then we call the object store (keyValStore), passing in the name of the object store I want, keyval.

const keyValStore = tx.objectStore('keyval')
Note
 It may be repetitive, but there’s a possibility that you’ll have a transaction that uses multiple objects stores.

  • we call .get() on the object store and pass the key I’m interested in such as "hello".

return keyValStore.get('hello');

It will return a promise, which resolves to the value I’m looking for.

.then( val => console.log(`The value of "hello" is: ${val}`))

or

.then(function(val) {
    console.log('The value of "hello" is:', val);
}

Here’s how they look all together for reading the object store

dbPromise.then(db => {
  const tx = db.transaction('keyval');
  const keyValStore = tx.objectStore('keyval');
  return keyValStore.get('hello');
}).then(val => console.log(`The value of "hello" is: ${val}`))

When you refresh in console in devtools, you should get:

The value of "hello" is: world

Adding another value to the object store

Now if we want to add another value to the object store. To do that, we need to create a transaction just as we did before, but this time we specify that we want to read and write this time.

dbPromise.then(function(db) {
    var tx = db.transaction('keyval', 'readwrite' );
    var keyValStore = tx.objectStore('keyval');
    keyValStore.put('bar', 'foo');
}

when using .put, it returns a promise. This promise doesn’t mean it will work. As a reminder, if any part of the operation fails, the whole operation will fail. Which is kind of a good thing because none of the operation will be in a half finished state. So either all happens or none of it happens.

return tx.complete;

transaction.complete is a promise that filfills if and when the transaction completes, and it rejects if it fails.

Once the transaction completes, I’m going to log a success message:

.then(function() {
    console.log('Added foo:bar to keyval')
})

Here’s how they look like all together in an ES6 practice version.

dbPromise.then(db => {
  const tx = db.transaction('keyval', 'readwrite');
  var keyValStore = tx.objectStore('keyval');
  keyValStore.put('bar', 'foo');
  return tx.complete;
}).then(_ => console.log(`Added foo:bar to keyval`));

and get this result in console: Added foo:bar to keyval and this in the idb database
idb2

4.3 : Quiz: Getting Started with IDB

You don’t have to, but if you want to ready the template for the quiz, you can type in:

git reset --hard
git checkout page-skeleton

  • Just in case you forget where the IDB index is for editing, it’s in:
    public > js > idb-test > index.js

  • TODO: in the keyval store, set "favoriteAnimal" as the key and an animal as your value. eg: cat or dog.

Code Refresher:

  • Create a function for dbPromise with a read and write transaction.

dbPromise.then(db => {
  const tx = db.transaction('keyval', 'readwrite');
})

  • Then we have to have a place to store the information to.

const keyValStore = tx.objectStore('keyval');

  • The process of actually adding the key and value. Don’t forget to return the information.

keyValStore.put('animalOfChoice', 'favoriteAnimal');
return tx.complete;

  • Once you’ve completed the task, check to see if the entry was submitted into the devtool’s database. If you don’t see it right away, try refreshing it.

Note
 Make sure you’re in localhost:8888/idb-test.

  • Once you see the entry, head on over to the setting’s page and type in test ID: idb-animal and you should see the message: Yay! Your favorite animal is "animalYouPicked

The answer should have been:

dbPromise.then(function(db) {
    const tx = db.transaction('keyval', 'readwrite');
    const keyvalStore = tx.objectStore('keyval');
    keyvalStore.put('manatee', 'favoriteAnimal');
    return tx.complete;
}).then(_ => console.log("added an animal"));

Adding People objects to objectStore

So far we’ve created a key/value objects store, but now we want to create a different store with objects all of the same kind. Such as people. To do that, we need to create another ObjectStore. To create a different objectstore, we need to do that in indexes within the upgrade function.

Note
 You need to bump the version of the .open() for the upgradeDb function to run again for the new addition we’re going to put in.

  • We create a new objectStore called People. It’s not going to have separate keys, instead the name property of the objects inside will be the key.

keyValStore.put('people', { keyPath: 'name'});

Here Jake mentions that in the real world, people will have the same name, but in this case, we’re just going to assume people have different names.

Warning
 If we try to run the code now, it will fail because createObjectStore has already been created.

IDB has a workaround to that problem.
Introducing: oldVersion in conjuction with switch() statement to let you know which to run if a certain version.
We use the switch and oldversion to surround each of the createObjectStore to control which ones to run when.

switch(upgradeDb.oldVersion) {
  case 0;
    var keyValStore = upgradeDb.createObjectStore('keyval');
    keyValStore.put("world", "hello");
  case 1;
    upgradeDb.createObjectStore('people', { keyPath: 'name'})
}

So if the version is 0, it sets up the 'keyval' store, if the version is 1, we set up the 'people' store.

Note
 Usually with switch statements, there’s a break after each case, but we don’t want to do that here because if the browser hasn’t set up this database at all before, it’ll start with case 0. It will create the key object store, but it will continue and create the object store.

Adding people to the People store.

  • Step 1 : Create the transaction for people and make it read/write.

dbPromise.then(function(db) {
  var tx = db.transaction('people', 'readwrite');
  var peopleStore = tx.objectStore('people');
})

  • Step 2 : Adding a person. Putting in their name, age, and their favorite animal.

peopleStore.put({
  name: 'Sam Munoz',
  age: 25,
  favoriteAnimal: 'dog'
});
return tx.complete
Note
 we just put in .put() without a key this time. Because when we created the objectStore, we specified the key was { keyPath: 'name'}. So the name of the object is the key.

  • Step 3: Now we can add a success console message.

.then(function() {
  console.log('People added')
})

Here, Jake added a lot more people objects into the list…​

Reading people in object store.

We have to create a transaction for people again.

  • Step 1 : We get ahold of the people object store with transaction again.

db.Promise.then(function(db) {
  var tx = db.transaction('people');
  var peopleStore = tx.objectStore('people')
})

  • Step 2 : We use .getAll() Which returns a promise for all the object in the store.

return peopleStore.getAll();

  • Step 3: Then we log the information.

.then(function(people) {
  console.log('People:', people);
})

By default it will be sorted alphabetically by their name since that is the key.

Changing the sorting to their favorite animal.

This is where indexes come in. Indexes can only be created as part of a version upgrade and put inside the .open() function.

  • Step 1 : Bump the version number.

  • Step 2 : Add an index to our switch case.

  • Step 3 : First we need to get ahold of the person object store using transaction again.

case 2:
  var peopleStore = upgradeDb.transaction.objectStore('people')

  • Step 4 : Now that we have the store, we have to create the index called animal which will sort by 'favoriteAnimal' property.

peopleStore.createIndex('animal', 'favoriteAnimal')

  • Step 5 : Now for actually using it. We go back to where we were reading people
    here’s the original that we wrote:

db.Promise.then(function(db) {
  var tx = db.transaction('people');
  var peopleStore = tx.objectStore('people');

  return peopleStore.getAll();
}).then(function(people) {
  console.log('People:', people);
})

  • Step 6 : first we create a new index from the object store by animal

var animalIndex = peopleStore.index('animal')

  • Step 7 : Then we modify .get(). Instead of returning peopleStore, we’re returning animalIndex.

Now when we refresh the browser to see the changes, they’re sorted by their favoriteAnimals.

How to execute queries on the index.

Using .getAll() you can put a specific key to search for. Such as .getAll('cat').

4.4 : Quiz: More IDB

This we need to get the same template as Jake’s

git reset --hard
git checkout task-idb-people

  • We need to create an index for people ordered by age inside the upgrade function.

  • At the bottom of the code we need to log out all the people in that order.

Code Refresher:

This is kind of a spoiler, but I figured it was copying what you last did anyway.

  • Add an index to the createObject function and use switch.

  case 3:
  var peopleStore = upgradeDb.transaction.objectStore('people'); // first access the people database.
  peopleStore.createIndex('age', 'age');  //Then create a new index (create the new name, the key that we'll sort with)

To read and console.log our result.

dbPromise.then(function(db) {
    var tx = db.transaction('people');
    var peopleStore = tx.objectStore('people'); // first access the people objectstore.

    var ageIndex = peopleStore.index('age'); // We also access the index we created earlier and we store it in ageIndex.

    return ageIndex.getAll();  // return what we stored in ageIndex.
  }).then(function(age) {
    console.log('age:', age);    // the logged info and sorted by age.
  });
Note
 Be sure to change the version # and also, the TODO: in the createObject function was after the curly bracket. Your new created Index should be inside with the others.

  • once done, we should see the changes in the browser’s console and there will be age section in people’s database.

  • To confirm the changes, go to the setting’s page and type in the test ID: idb-age. You should see the message Yay! The age index is working.

Introducing cursors

We’ve been getting items out of the store, but now we can go through them one at a time using cursors.

Using the age property that we created, instead of calling getAll(), we’re going to open a cursor.

return ageIndex.openCursor();

That will return a promise for a cursor object representing the first item in the index or undefined if there isn’t one. But if it is undefined, we’re going to do a usual return.

.then(function(cursor) {
  if {!cursor) return;
})

otherwise we’ll just log it

console.log('Cursored at:', cursor.value.name);

The first person in the index is in cursor.value.

Next we insert this code to move on to the next item.

return cursor.continue();

This returns a promise for a cursor representing the next item or undefined if there isn’t one.

Using cursor to keep going until undefined

Now if we want this to keep going until it becomes undefined, this is where it gets trickly.

Step 1 : you can name the function we’re in.

.then(function logPerson(cursor) {...})

Step 2 Then we can call it once cursor.continue resolves.

return cursor.continue().neth(logPerson);

What this does is that it creates an asynchronous loop until cursor is undefined which is the end of the list.

.then(function() {
  console.log('Done cursoring');
})

Skipping items with cursor.

Let’s say you want to skip the first two items, here is what you’d put.

.then(function(cursor) {
  if (!cursor) return;
  return cursor.advance(2);
})

Cursor Summary

So far it just shows a complicated way of using .getAll(), but cursors become really useful when you want to modify items as you’re looping through. You can use your cursor to:

  • cursor.update(newValue) to change the value.

  • cursor.delete() to remove it.

This is the basics for what we’ll be covering in the lesson. It’s the basic API.

If you want to play with the code that Jake was writing…​

git reset --hard
git checkout idb-cursoring

4.5 : Using the IDB Cache and Display Entries

The objective is to create a database that stores the posts.

When wittr loads via a service worker, it does so without going to the network. It fetches the page skeleton and assets straight from the cache.

At the moment we have to go to the network for posts. We’re going to change that. We want to get the posts from the offline stored database and display them. Then we want to connect the web socket to get updated posts once we’re online. Web sockets bypass both the service worker and the http cache. As the new posts arrive, we’ll add them to our database for next time.

  • Step 1 : We need to populate the database, but deal with displaying the contents later. First we need to inspect our websocket code. Head to public>js>main>inddexController.js

    1. There is a method that is called to open the web socket.

      this._openSocket();

open a connection to the server for live updates

IndexController.prototype._openSocket = function() {
  var indexController = this;
  var latestPostDate = this._postsView.getLatestPostDate();

In this methodd, we can see a listener for the message event. var ws = new WebSocket(socketUrl.href);

And that hands off to onSocketMessage, passing in the data it receives.

ws.addEventListener('message', function(event) {
  requestAnimationFrame(function() {
    indexController._onSocketMessage(event.data)
  })
})

Then .\_onSocketMessage parses the data with JSON, then passes it to addPost.

IndexController.prototype._onSocketMessage = function(data) {
  var messages = JSON.parse(data);
  this._postsView.addPosts(messages);
}

  • Step 2 : We are going to look at the data that was received by adding in a console.log.

IndexController.prototype._onSocketMessage = function(data) {
  var messages = JSON.parse(data);
  console.log(messages);
  this._postsView.addPost(message);
}

Once you select update on reload for service worker and refresh the page, you’ll receive this into console:
idb3

And more keeps getting added into console when wittr adds a new post. What we want to do is pass this information to IndexedDB.

There’s an obvious primary key here, id

idb4

And we want to display this information in the order of their date so we’ll need to create an index based on their time.

4.6 : Quiz: Using IDB Cache

We’re going to create a database for wittr! Yay! The moment I’ve been waiting for.
Okay, so first we need to ready the template.

git reset --hard
git checkout task-idb-store

We’re going to be editing wittr right in index Controller. Which is in:
public > js> main > IndexController.js

Inside a constructor function IndexController(container) {…​.}, they’ve created a promise for our database by calling openDatabase

this._dbPromise = openDatabase();

The openDatabase() function is incomplete and it’s our job to complete it.

  • Step 1 : First we create our database

    1. Inside openDatabase()

    2. Create a database called wittr.

    3. It has an objectStore called wittrs

    4. id as its key

    5. index is called by-date which will sort by the time property.

In \_onSocketMessage the database has been fetched.

  • Step 2 : We need to add each of the messages to the wittr store.

  • Step 3 : Confirm that the changes were made by searching for the database that was added by date.

Should look something like this:
idb5

  • Step 4 : Confirm again in the setting’s page(localhost:8889) in the test ID enter: idb-store and you should see the message The database is set up and populated!

Code Refresher

To actually create the database

idb.open('name_of_database', version_#, The function that gets run when starting the database for the first time or if the version number is more than the last time this was run.)

Adding the database

var name = upgradeDb.createObjectStore('aSubName', { keyPath:'id'});  // The *ID* will be the primary key.

Adding an index from original

name.createIndex('by-date', 'time');

Now what to put in it.

We always have to do the usual retrieving the database.

var tx = db.transaction('aSubName', 'readwrite');
var keyValStore = tx.objectStore('aSubName')

This lesson they want you to store messages into the database.
We need to iterate the messages array and put it in keyValStore. There are a number of ways to do this.

ForEach

messages.forEach(function(message) {
  keyValStore.put(message);
})

For...of loop

for (const message of messages) {
  keyValStore.put(message);
}
Note

When creating the database. You don’t need to store it into a variable with a name since we’re not going to be calling it by its name. Since it’s in a function, you just need to return it. So it’ll look like this:

return idb.open(){...}

How Jake completed the quiz

Creating the database

return idb.open('wittr', 1, function(upgradeDb) {
  var store = upgradeDb.createObjectStore('wittrs', {keyPath: 'id'});
  store.createIndex('by-date', 'time');
})

Everything works. The database is open and working, but now we just need to put messages in it.

Inside \_dbPromise.then's function

var tx = db.transaction('wittrs', 'readwrite');
var store = tx.objectStore('wittrs');
messages.forEach(function(message) {
  store.put(message);
})

4.7 : Quiz: Using IDB 2

Now that we’ve put messages into the database, we want to show them. Now we want to get posts that are in the database and display them before connecting to the web socket that gets us newer posts.

Let’s ready the template!

git reset --hard
git checkout task-show-stored

We are still working with wittr, so we’ll be editing in indexController.js. On the previous lesson we were calling _opensocket in the constructor. Now we’re calling \_showcachedMessages then we will open the socket.

this._showCachedMessages().then(function() {
  indexController._openSocket();
})

Currently our showCachedMessages is rather empty. This is where we come in.

  • Step 1 : We have to get the messages out of the database and pass them to this method:

indexController._postsView.addPosts(messages)

code refresher

The usual grab

var tx = db.transaction('wittrs');
var store = tx.objectStore('wittrs');

Now for the index that we want to use

var dateIndex = store.index('by-date');

Now we want to get all of the data from our index

return dateIndex.getAll()

getAll returns a promise so we want to pass them to the indexController and we want it starting with latest.

.then(function(messages) {
  indexController._postsView.addPosts(messages.reverse());
})

  • Step 3 : Once you’re done making changes to the code, make sure you bump the version inside the service worker script.

  • Step 4 : Once done with that, you can test it out by going to setting’s page and set offline mode and should still see the posts on wittr.

  • Step 5 : To confirm again, in the settings page go back to online mode and in the test ID enter: idb-show and should see the message Page populated from IDB!

How Jake did it

var index = db.transaction('wittrs').objectStore('wittrs').index('by-index');
return index.getAll().then(function() {
  indexController._postsView.addPosts(messages.reverse());
})

It looks like you can cram transaction, objectstore, and index into one variable. good to know!

4.8 : Quiz: Cleaning IDB

The thing with databases is that there will be a limit. So to work with that, we only want 30 wittr items in it at any single time.

To ready the template, you type in:

git reset --hard
git checkout task-clean-db

  • Step 1 : We will edit our code inside indexController.js again.
    public>js>main>indexcontroller.js

  • Step 2 : We are editing in

IndexController.prototype._onSocketMessage = function(data) {...}

We already added items to the database, we just need to make it limit to only 30. We will have to use cursors for this.

code refresher.

First we need to open the cursor for store.

return store.openCursor(null, 'prev');

It will return a promise so we’re going to use it to only display 30.

.then(function(cursor) {
    return cursor.advance(30);

advance is also a promise so we can use that to use the other cursor method delete().

.then(function deleteRest(cursor) {
    cursor.delete();
return cursor.continue().then(deleteRest);

  • Step 3 : See the changes in the database.

  • Step 4 : Confirm the change by going to the setting’s page and type in the test ID: idb-clean and you should get this message: Looks like the database is being cleaned!

Here’s how Jake did it.

store.index('by-date').openCursor(null, 'prev').then(function(cursor) {
  return cursor.advance(30);
}).then(function deleteRest(cursor) {
  if (!cursor) return;
  cursor.delete();
  return cursor.continue().then(deleteRest);
})

He’s going to go by-date because he wants to remove the oldest posts.

  • null, 'prev' makes it so it goes backwards in the index starting with the newest posts.

  • Starting with the newest post, he wants to keep the top 30 posts.

  • if cursor is undefined, we’re done.

  • With the rest of the information, he wants it deleted with .delete().

  • After it’s done deleting, continue the cursor from start, and then run the same deleteRest function again to loop through the remaining entries.

Now we can check off another todo list that I completely forgot about!

  • ✓ Unobtrusive app updates.

  • ✓ Get the user onto the latest version.

  • ✓ Continually update cache of posts.

  • ❏ cache photos.

  • ❏ cache avatars.

4.9 : Cache Photos

At the moment we’re only caching resources at install time. We want to cache photos too. we want to cache photos as they appear. We could put these photos in IDB along with the rest of the phost data, but that would mean we need to read the pixel data and convert it into a blob. That’s a little too complicated and it loses streaming, which has a performance impact.

At the moment, when we get an item from a database, we have to take the whole thing out in one lump, then convert it into image data, then add it to the page.

Though if we get the image from a cache, it will stream the data. So we don’t need to wait for the whole thing before we display anything. Which makes it more energy efficient and leads to faster render time.

Here’s the code for the image which is responsive image. Which means it can appear at a veriety of different widths.

<img src="/photos/65152-800px.jpg"
srcset="/photos/65152-1024px.jpg 1024w,
        /photos/65152-800px.jpg 800w,
        /photos/65152-600px.jpg 640w,
        /photos/65152-320px.jpg 320w"
  sizes="(min-width: 800px) 765px,
         (min-width: 600px) calc(100vw - 32px),
        calc(100vw - 16px)">

Because images can appear at a variety of different widths, the responsive image lets the browser decide which image to load based on the width of the window and also the network conditions.

So when the posts arrives through the web socket, which version do we cache?

  • First we wait until the browser makes the request.

  • Then we hear about it in the serviceWorker.

  • We go to the network for the image.

  • once we get a response, we put it in the cache.

  • At the same time, we send it on to the page.

  • At the moment we put the image into a separate cache to the rest of the other static content.

  • We reset the content of our static cache whenever we update our javaScript or css, but we want these photos to live between versions of our app.

next time we get a request for an image that we already have cached, we simply return it.

The trick here is that we’ll return image from the cache even if the browser requests a different size of the same image.

Posts on wittr are short lived, so if the browser requests a bigger version of the same image, returning a smaller one from the cache isn’t really a problem.

You can only use the body of a response, once.

Meaning if we read the responses json, you cannot then read it as a blob.

  • response.json();

  • response.blob();

That’s because the original data has gone, keeping it in memory would be a waste. Also keep in mind that respondWith uses the body of the response as well so which means we cannot later read it again.

  • event.respondWith(response)

This can be a good thing. If the responses was like a 3 gigabyte video going to a video element on the page, the browser doesn’t need to keep the whole 3 gigabytes in memory. It only needs to keep the bit that it’s currently playing. Plus a little bit extra for buffering.

However this is a problem for our photos.

We want to open a cache,
fetch from the network,
and send the response to both the cache and back to the browser.

event.respondWith(
  caches.open('wittr-content-imgs').then(function(cache) {
    return fetch(request).then(function(response) {
      cache.put(request, response);
      return response;
    })
  })
)

Using the body twice like this doesn’t work, but there is a workaround by using clone(). Now the clone goes to the cache and the original gets sent back to the page.

event.respondWith(
  caches.open('wittr-content-imgs').then(function(cache) {
    return fetch(request).then(function(response) {
      cache.put(request, response.clone());
      return response;
    })
  })
)

The browser keeps enough of the original request around to satisfy all of the clones.

4.10 : Quiz: Cache Photos Quiz

Jake starts us off.

We’re actually taking a break from IDB and going back to caches for storing images.

  • Step 1 : We go back to service worker in:
    public>js>sw>index.js

  • Step 2 : We need to set up our image cache in our service worker. We start that off by creating a variable contentImgsCache and setting it to 'wittr-content-imgs';

  • Step 3 : Next we’re going to create another variable allCaches that contains an array of the cache names we’ve already created. staticCacheName and contentImgsCache

var allCaches = [
  staticCacheName,
  contentImgsCache
]

  • Step 4 : Back in Chapter 3, our activate event that we wrote, we deleted any cache that isn’t staticCacheName.

cacheName != staticCacheName;

Now that we created contentImgsCache, we want that to be included as well somehow. So instead of just checking staticCacheName alone, we are going to make it check our array that we made which was called allCache.

!allCaches.includes(cacheName);

  • Step 5 : Now we are going to handle our photo requests. Go over to our fetch handler. We are going to handle URLs that have the same origin and have a path that starts with '/photos/'. And when it does see one of those, it will respond with whatever returns from serve photo.

if (requestUrl.pathname.startsWith('/photos/')) {
  event.respondWith(servePhoto(event.request));
  return;
}

  • Step 6 : Now that we created a fetch listener for photos, it called servePhoto. Now we just need to implement servePhoto. We only want to store 1 copy of each photo, and the responsive photos that was stored look like this:

/photos/9-8028-75277734776-e1d2bda28e-800px.jpg

They have their width information at the end.

What we’re going to do is create a storage URL that doesn’t have the size info. We are going to do that using the regular expressions and .replace() .

var storageUrl = request.url.replace(/-\d+px\.jpg$/, '');

We’re matching on - | any digit | px.jpg and then we replace it with nothing. We have the URL, but missing the size-specific stuff. That is the URL we will use on the cache.

for example

original: /photos/9-8028-75277734776-e1d2bda28e-800px.jpg

the changed : /photos/9-8028-75277734776-e1d2bda28e

Here is where we take over inside servePhoto

  • Step 1 : To ready the template

git reset --hard
git checkout task-cache-photos

  • Step 2 : We will still be working in the serviceworker script.
    public> js> sw > index.js

  • Step 3 : What we need to do is to serve the photos from the cache if they’re there, otherwise we get the photos from the network and also put them in the cache for next time. Also be sure to use the variable we created storageUrl when matching and putting stuff into the image cache so you only end up with one photo in cache no matter how many different sizes are requested.

Code Refresher
(This is a major spoiler. It has been a while since I worked on SW so I had to get Jake!)

First we need to open up the cache that we created.

return cache.open(contentImgsCache)

Then we need to serve the photos from the cache if they’re there

return cache.match(newVarWeMade).then(function(response) {
  if (response) return response;
})

otherwise we get it from the network

return fetch(request)

store it into the cache

.then(function(networkResponse) {
  cache.put(newVarWeMade, networkResponse.clone());
})

and send the original to the browser.

return networkResponse;
Note
 As a reminder. You can only read the messages once, so that is why we used clone() to send to the cache and send it to the browser…​

  • Step 4 : You can confirm that it’s working when you go to your cache in devtools and see a wittr-content-cache and all the images that are saved are missing the width and their .jpg.

swphotos

  • Step 5 : Confirm a second time by going to the settings page and set it to offline mode to check to see if images are still loaded when you refresh the wittr page.

  • Step 6 : To finalize your confirmation, go to the setting’s page and set it back to online and type in the Test ID: cache-photos. You should get the message: Photos are being cached and served correctly!

Here’s how Jake did it

  return caches.open(contentImgsCache).then(function(cache) {
    return cache.match(storageUrl).then(function(response) {
      if (response) return response;
      return fetch(request).then(function(networkResponse) {
        cache.put(storageUrl, networkResponse.clone());
        return networkResponse;
      })
    })
  })

Jake tests the images by resizing the window to see if the images would still load. And it does because it will still send the image that is in the cache no matter the size that it requests.

Okay, that part is a bit confusing since I don’t know about responsive images.

4.11 : Cleaning Photo Cache

Okay now that we’ve added a cache for images, we need to delete the old ones we don’t need anymore.
It won’t be the same way we did with databases. It’s going to be different and specific for caches.

  • If we want to remove specific entries from the cache, we can use cache.delete

cache.delete(request);

  • There is .keys() that returns a promise that gives us all the requests for entries in the cache.

cache.keys().then(function(requests) { // }

All of this is available from pages as well as service workers. In the next chapter we will use these methods to clean the image cache.

4.12 : Quiz: Cleaning Photo Cache Quiz

Jake helps setup our _cleanImageCache

  • Step 1 : We are going to be working in our indexController for this one.
    public >js > main>indexController.js

  • Step 2 : We are going to create a new method called \_cleanImageCache.

IndexController.prototype._cleanImageCache = function() {//}

  • Step 3 : When the page loads, it starts IndexController so inside there, we’re going to add a call for our new \_cleanImageCache method.

  • Step 4 : The cache can still go out of control if the user keeps the page open for ages. We can make it so it’ll call for the clean every five minutes.

setInterval(function() {
  indexController._cleanImageCache();
}, 1000 * 60 * 5)

  • Step 4 : Now we just need to implement our \_cleanImageCache.

Now we have to complete our _cleanImageCache

Our new _cleanImageCache is going to bring together IDB and cache API.

  • Step 1 : Ready the template

git reset --hard
git checkout task-clean-photos

  • Step 2 : We will be working in indexController.js
    public>js>main>indexController.js

  • Step 3 : We have to implement \_CleanImageCache.

Our TODO: We have to get all the messages from the database. Looking at what photos they need, then going through the images cache and getting rid of ones you don’t need anymore. Important note: The photo’s property may not match the URL in the cache.

Tip
 When you’re done editing in the code, make sure you clear everything. Clear your cache and delete your database! There’s a special way to delete your cache on the page here:
cleardata
After you’ve cleared the cache, make sure you actually have some images in the cache for the test ID to check if some changes have been made when you refresh the wittr page. So which means you may have to stay idle for a bit to collect some images.
If you’re still getting a fail, you may need to clear site data on your setting’s page as well. That worked for me.

Code Refresher

  • Step 4 : To confirm that it’s working, head over to the image cache and you should only see a list of images that are currently on the page. When you refresh the page again, you would notice that the images inside the cache would be different but still limited to what is on the page.

  • Step 5 : To confirm again, go over to the setting’s page and type in the test ID: cache-clean. We have 8 seconds to test the clean up so the best way to do that is by refreshing wittr. You should get the message: Yay! The image cache is being cleaned!

How Jake did it!

  • First he created the array of images he wants to keep.

var imagesNeeded = [];

  • Then create a transaction to look at the 'wittrs' store.

var tx = db.transaction('wittrs');

  • Then we want to return a promise that actually opens the 'wittrs' store. We will use that to .getall()

return tx.objectStore('wittrs').getAll()

  • Now we can look into the database. For each message, I’ll look to see if it has a photo property.

.then(function(messages) {
  messages.forEach(function(message) {
    if (message.photo)

});

This contains the photo URL, but without the width bit at the end.

  • Now with that, I’ll add the photos to the array of images that I want to keep.

imagesNeeded.push(message.photo);

  • After the forEach, we want to open our images cache

return caches.open('wittr-content-imgs')

  • and get the requests that are stored in it using cache.keys.

.then(function(cache) {
  return cache.keys().then(function(requests) {
    //
  })
})
Note
 The URLs on request objects are absolute so which means it’ll include the local host port 8080 bit.

  • For each requests, we want to pass a URL

requests.forEach(function(request) {
  var url = new URL(request.url);
})

  • If the path name of the URL isn’t in our array of images needed, we’ll pass a request to cache.delete.

if (!imagesNeeded.includes(url.pathname)) cache.delete(request);

  • All together should look like this!

IndexController.prototype._cleanImageCache = function() {
  return this._dbPromise.then(function(db) {
    if (!db) return;

    var imagesNeeded = [];

    var tx = db.transaction('wittrs');
    return tx.objectStore('wittrs').getAll().then(function(messages) {
      messages.forEach(function(message) {
        if (message.photo) {
          imagesNeeded.push(message.photo);
      });
      return caches.open('wittr-content-imgs');
    }).then(function(cache) {
      return cache.keys().then(function(requests) {
        requests.forEach(function(request) {
          var url = new URL(request.url);
          if (!imagesNeeded.includes(url.pathname)) cache.delete(request);
        });
      });
    });
  });
};

4.13 : Quiz: Caching Avatars

Setting up for adding avatars into our cache

The previous one was a tough one, so I believe caching avatars might actually help us practice.

The difference here is that avatars actually change often. We don’t want people to be stuck with their old avatar after changing theirs.

What we’ll do with avatars is that we’ll fetch a particular avatar from the cache, we’ll also fetch it from the network and update the cache.

Avatars are also responsive images, but they vary by density rather than width. They’re a bit like the photos, but a slighltly different URL pattern.

example:

<img width ="40" height="40" src="/avatars/sam-1x.jpg"
srcset="/avatars/sam-2x.jpg 2x,
/avatars/sam-3x.jpg 3x">

We will cache the avatars in the same cache as photos. In \_cleanImageCache in indexController.js.
The first thing we do is edit our cleanup code to include avatars for things we want to keep. We don’t want our cleanup to delete them all.

images.Needed.push(message.avatar);

Now it’s our job to finish the rest.

  • Step 1 : To ready the template

git reset --hard
git checkout task-cache-avatars

  • Step 2 : We will be working on service worker.+ public > js> sw> index.js

  • Step 3 : There are two TODOS.

    1. inside the fetch event listener, we need to call serveAvatar for avatar URLs.

    2. Then we need to implement the actual serveAvatar.

      1. It should return them from the cache if they’re there.

      2. If not, get it from the network and put it in the cache.

      3. The difference here is that: Even if you return it from the cache, you need to go to the network to update it for the next fetch.

      4. Reminder, we’re removing the size specific parts of teh URL so use storageUrl to put and match in the cache.

      5. The solution will be similar to servePhoto, but not exactly the same.

Code Refresher

Here’s what servePhoto looks like:

function servePhoto(request) {
  var storageUrl = request.url.replace(/-\d+px\.jpg$/, '');

  return caches.open(contentImgsCache).then(function(cache) {
    return cache.match(storageUrl).then(function(response) {
      if (response) return response;

      return fetch(request).then(function(networkResponse) {
        cache.put(storageUrl, networkResponse.clone());
        return networkResponse;
      });
    });
  });
}

  • Step 4 : Once you’re done, you can confirm that that it’s working when you see avatars being saved into wittr-content-imgs cache. When you go to setting’s page to test out offline mode and refresh the wittr page, you should still be able to see the avatars.

cleardata2

  • Step 5 : To confirm again, go back over to the setting’s page. Make sure you go back to online mode and type in the test ID: cache-avatars and should get the message: Avatars are being cached, served and updated correctly!

Here’s how Jake did it!

Inside our fetch listener, we are going to react to URLs that starts with /avatars/ and respond with serveAvatar.

if (requestUrl.pathname.startsWith('/avatars/')) {
  event.respondWith(serveAvatar(event.request));
  return;
  }

Over in serveAvatar we need to open up the image cache then look for a match for the storageUrl

return caches.open(contentImgsCache).then(function(cache) {
  return cache.match(storageUrl)

The difference here is that we need to do a network fetch for the avatar. If we get a response, we put a clone in the cache using storageUrl and also return the original response.

.then(function(response) {
  var networkFetch = fetch(request).then(function(networkResponse) {
    cache.put(storageUrl, networkResponse.clone());
    return networkResponse;
    })

Now that we’ve got a response from the cache, though it may be undefined if there’s no match for this particular request. We also got a promise for the network response. So we’ll return the cache response or the network response, and that’s it.

return response || networkFetch;

Here’s how it looks together

return caches.open(contentImgsCache).then(function(cache) {
  return cache.match(storageUrl).then(function(response) {
    var networkFetch = fetch(request).then(function(networkResponse) {
      cache.put(storageUrl, networkResponse.clone());
      return networkResponse;
    });
      return response || networkFetch;
  })
})

4.14 : Outro

That’s it!