Hey Developers, I am sure most of you are already aware of Service Workers and their applications. While those who don't know just think of it like a script that your browser runs in the background, separate from a web page, opening the door to features that don't need a web page or user interaction like caching etc. So, In our applications at BharatPe, we were trying to implement caching to optimize the performance of our applications and while doing so we faced blockers in cache management.
Let's learn how these service workers work by default and at which stage we faced this issue and then how we tackled this issue in our BharatPe applications.
How Service Workers works w.r.t Caching
With the help of service workers, we implement caching and cache the static files in the browser. When you cache your static files, your application becomes faster and is helpful for even running your application offline. Let's understand this using below data flow diagram.
- Now, when the user goes to a website/application, then they receive the information from the cached stored data file and in the background, the service worker will connect with the server and check whether there is any update that is pushed into the server. Now if there is no update, then there is everything great. But if there is an update then the service worker fetch the updated data files and store them (they are not served yet, they are in the waiting phase). Have a look at the below diagram showing data flow.
- Now the updated files are in a waiting state until it's required to get swapped with the older one. Now you might be thinking when should that swap happen? When the user refreshes the page or anything else? This is what the problem is, the default behaviour of the service workers is that when the page refreshes, it fetches the data from cached files only and our newly updated content is in waiting for state only. Have a look at the below diagram showing data flow once the user refreshes.
- Now this could be accomplished by closing the tab of the application (if it's a web application) and then opening it again. There are other conditions also that these service workers see other than the closing tab. These service workers also have a function called skipWaiting(), which skips the waiting time of the updated data and renders the updated data once the user refreshes the page i.e user is delayed by just n+1 state.
This concept sounds amazing theoretically, but as I already mention, there are several other parameters also that these service workers see, and the result is that even after multiple refreshed and new tab sessions, the latest updates were not rendering. The in-build service worker of React JS was also discarded due to this reason only and they stop providing inbuilt service workers support.
In the case of BharatPe applications, the problem is same, as we use web view to render our applications into mobile screens, hence we are not able to close the tab, and then the refresh thing is not working practically in either of the cases.
This thing is possible only when you clear caches and browser history, but we can't expect all users are technically sound.
Solution 1 is that we monitor the latest files or versions via our code and whenever there is any new version available i.e the current version and the last updated versions are mismatched then we can give a popup to the user that a new update is available.
You might have seen such kinds of pop-up messages in a few applications like Gmail etc.
But this is not a good solution with respect to the UX point of view, this looks less prominent way to deal with this problem. Because every time the user comes to our application and we release something new, this pop-up occurs and the user has to do it again and again. It's so awkward :(
⚠ Warning ⚠ - There are possibilities that you might deliver a bad or malfunctioned service workers, which is really a very bad impression to your users. Because delivering bad service worker is something which you can't resolve even after releasing any new update. It will be a huge problem with a very bad impact. The only way to revert back is to release a new deployment with either a good service worker or remove that and then ask your users to clear the cache of browsers and cookies and then use your application. Which is so bad w.r.t the user's point of view.
We have created our own service worker and we are resolving this issue by EVH (External version handler). The main steps of our service worker include SW Registration, SW Installation, and SW Activation.
Main Idea is to get the version from version.json of the new build. And compare it with the existing version inside the cache. if they are the same then do nothing else clear the old cache, unregister the service worker and re-register it.
Note - I will be demonstrating for the React JS + Typescript project. You can change it as per your stack.
Step 1
Configure your service worker which will be used to generate the final service worker script with all the details of the assets that are to be cached.
- Service Worker Config, name the file as "src/service-worker.ts" (We are using workbox for config)
import { clientsClaim } from 'workbox-core';
import { ExpirationPlugin } from 'workbox-expiration';
import { precacheAndRoute, createHandlerBoundToURL } from 'workbox-precaching';
import { registerRoute } from 'workbox-routing';
import { StaleWhileRevalidate } from 'workbox-strategies';
clientsClaim();
precacheAndRoute(self.__WB_MANIFEST);
const fileExtensionRegexp = new RegExp('/[^/?]+\\.[^/]+$');
registerRoute(
({ request, url }) => {
if (request.mode !== 'navigate') {
return false;
}
if (url.pathname.startsWith('/_')) {
return false;
}
if (url.pathname.match(fileExtensionRegexp)) {
return false;
}
return true;
},
createHandlerBoundToURL(process.env.PUBLIC_URL + '/index.html')
);
registerRoute(
({ url }) => url.origin === self.location.origin && (url.pathname.endsWith('.png') || url.pathname.endsWith('.jpg') || url.pathname.endsWith('.sv')),
new StaleWhileRevalidate({
cacheName: 'images',
plugins: [
new ExpirationPlugin({ maxEntries: 5 }),
],
})
);
self.addEventListener('message', (event) => {
if (event.data && event.data.type === 'SKIP_WAITING') {
self.skipWaiting();
}
});
Step 2
Now we need to have a script that generates version on every build release. Which is a external version file (version.json)
- Create a new folder named "script" and add the below script inside this folder with the name "pre-release.js"
const path = require('path');
const fs = require('fs-extra');
const PACKAGE_PATH = process.cwd();
const BUILD_PATH = path.join(PACKAGE_PATH, './public');
const getVersion = async() => {
return new Promise((resolve, reject) => {
const packageFile = path.join(PACKAGE_PATH, 'package.json');
console.log('reading package');
fs.readFile(packageFile, (err, data) => {
if (err)
resolve(null)
const packageJson = JSON.parse(data);
resolve(packageJson.version);
})
});
}
async function createVersionFile(buildPath) {
const version = await getVersion();
if (!version) return;
const versionContent = {
version
};
console.log(versionContent, "JSON");
const versionJsonPath = path.join(BUILD_PATH, 'version.json');
await Promise.all([
fs.writeFile(versionJsonPath, JSON.stringify(versionContent, null, 2)),
]);
}
createVersionFile(BUILD_PATH);
- Update the build script in your package.json as
"build": "node scripts/pre-release.js && react-scripts build"
This will generate the version as planned and then the build will generate.
Step 3
Service worker handler, This is to register, unregister, version checker, handling cache etc. This is like a service which manages the service worker flow
function registerValidSW(swUrl: string, config: any) {
navigator.serviceWorker
.register(swUrl)
.then((registration) => {
registration.onupdatefound = () => {
const installingWorker = registration.installing;
if (installingWorker == null) {
return;
}
installingWorker.onstatechange = () => {
if (installingWorker.state === 'installed') {
if (navigator.serviceWorker.controller) {
console.log('New content is fetched but yet to service!!');
} else {
console.log('Content is precached now !!');
}
}
};
};
})
.catch((error) => {
console.error('Error during service worker registration:', error);
});
}
function checkValidServiceWorker(swUrl: string, config: any) {
fetch(swUrl, {
headers: { 'Service-Worker': 'script' },
})
.then((response) => {
const contentType = response.headers.get('content-type');
if (response.status === 404 || (contentType != null && contentType.indexOf('javascript') === -1)) {
navigator.serviceWorker.ready.then((registration) => {
registration.unregister().then(() => {
window.location.reload();
});
});
} else {
registerValidSW(swUrl, config);
}
})
.catch(() => {
console.log('No internet connection found. App is running in offline mode.');
});
}
const isLocal = Boolean(
window.location.hostname === 'localhost' ||
window.location.hostname === '[::1]' ||
window.location.hostname.match(/^127(?:\.(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}$/),
);
export function register(currentBuildVersion: any) {
if (process.env.NODE_ENV === 'production' && 'serviceWorker' in navigator) {
const publicUrl = new URL(process.env.PUBLIC_URL, window.location.href);
if (publicUrl.origin !== window.location.origin) {
return;
}
window.addEventListener('load', async () => {
const versionJSON = `${process.env.PUBLIC_URL}/version.json`;
const JSONResponse = await fetch(versionJSON);
const versionData = await JSONResponse.json();
const buildVersion = versionData.version;
const cacheStatus = await clearCacheOnNewBuild(currentBuildVersion, buildVersion);
if (cacheStatus == 'deleted') {
await unregister();
document.location.reload();
}
console.log('resolved', cacheStatus);
const swUrl = `${process.env.PUBLIC_URL}/service-worker.js`;
if (isLocal) {
checkValidServiceWorker(swUrl, null);
navigator.serviceWorker.ready.then(() => {
console.log('Service Worker Activated');
});
} else {
registerValidSW(swUrl, null);
}
});
}
}
export function unregister() {
return navigator?.serviceWorker?.ready
.then((registration) => {
return registration.unregister();
})
.catch((error) => {
return console.error(error.message);
});
}
export const clearCacheOnNewBuild = (currentBuildVersion: string, buildVersion: string) => {
return new Promise((resolve, reject) => {
console.log('New BuildVersion', buildVersion, 'Current BuildVersion', currentBuildVersion);
if (currentBuildVersion !== buildVersion) {
console.log('New version found.');
if (buildVersion) {
if ('caches' in window) {
caches.keys().then((keyList) => {
return Promise.all(
keyList.map((key) => {
return caches.delete(key);
}),
);
});
resolve('deleted');
}
}
}
resolve('loading');
});
};
Step 4
Initialise the service worker in your application. Add the below code into your index file along with your existing code.
import * as ServiceWorker from './ServiceWorkerCore';
const pjson = require('../package.json');
try {
ServiceWorker.register(pjson.version);
} catch (e) {
console.error('Service Worker Failed', e);
}
With these 4 steps, We can have a perfect service worker which effiencetly handles cache & improves overall performance.
This is for CRA based react projects. For web pack based projects the workbox part slightly changes a bit. We don't need the service-worker.ts file, Instead add these in you webpack config.
Generating service worker
new WorkboxPlugin.GenerateSW({
clientsClaim: true,
skipWaiting: true,
exclude: ['version.json']
}),
Copying the version file from public to build folder
new CopyWebpackPlugin({
patterns: [{
from: path.resolve(__dirname + '/public', "version.json"),
}]
}),
This should work the same way as CRA.
This is how we were able to achieve this great performance with our own custom worker. I hope after reading this the fear inside you of the service workers vanishes and you were able to increase the performance of your application. In case of any doubts feel free to drop us your doubts at tech.queries@bharatpe.com or you can comment below and our team will try to respond to you as soon as possible.
Do follow us on Twitter, LinkedIn, we post our geeky stuff very frequently here. Also, if you are looking to join our geeky engineering team, do check out https://tech.bharatpe.com/ we have a lot of openings for the candidates who believe in building a new generation of revolutionary fintech products and building a new INDIA together.
