In the startup hassle, product development is always the highest priority, and due to this race to build and ship products in a very short period, we generally forget to follow some basic coding practices. One of the problems among them is the use of the optimal size of the images.
Problem Statement - To automate the blocking of the use of assets that are higher than a threshold value (eg. 100kb).
The Approach - To solve this problem using some automation we identified 2 break points where some automation can be added i.e. at the developer's end and the PR reviewer's end. The idea is to first warn the developer before pushing it into our VCS that your changes contain some assets that are greater than 100kb and secondly, to provide some information about the assets in Pull Request so that the reviewer will get to know. This will help us to determine if the developer ignored and pushed directly into VCS.
Breakpoint 1: To make sure the developer will get the warning while pushing his changes into GitHub, I created a script that will run as a pre-commit. To implement this, I used a husky and a bash script.
In the bash script, I first checked the count of all the assets that are greater than 100kb and stored it. Now, if the count is greater than zero, I will post a warning instruction in the terminal and if the count is 0, then I will print out a congratulations message in the terminal.
NC='\033[0m'
YELLOW='\033[0;33m'
GREEN='\033[0;32m'
count=$(find ./src/assets/ -type f -size +100k -exec ls -lh {} \; | wc -l)
if [[ "${count}" -gt 0 ]]; then
find ./src/assets/ -type f -size +100k -exec ls -lh {} \; | awk '{ print "\033[31m" $9 " || Size : " $5 }'
printf "${YELLOW}INFO: Please resize above assets and then proceed${NC}\n\n"
else
printf "${GREEN}Congratulations, all images are less than 100Kb.${NC}\n\n";
fi
# Script Ends
In the package.json, add husky logic and script command.
"husky": {
"hooks": {
"pre-commit": "yarn run asset-test"
}
}
Add script in dependencies. (Note - You can replace the "scripts/pre-asset-check.sh" path to your file where it's located in your code).
"asset-test": "chmod +x scripts/pre-asset-check.sh && bash scripts/pre-asset-check.sh"
Breakpoint 2: Now when the developer has raised the pull request, I created a new GitHub action which will automatically triggered when any new PR is raised or new changes in PR are pushed.
To create a new GitHub action, I followed an amazing article by FreeCodeCamp https://www.freecodecamp.org/news/build-your-first-javascript-github-action/ and created my files as per requirement.
Action metadata file - This will help us to define the interface of our Action.
name: 'Assets Checker'
description: 'A Github Action to Analyse your static image files and warns if the size increase the threshold size. It check for .jpg, .svg, .png, .gif, .jpeg files.'
author: 'BharatPe'
inputs:
target_folder:
description: 'The path of directory which contains all your assets ex. src/assets'
required: true
default: src/assets
thrashold_size:
description: 'Maximum size of assets allowed in Kb ex. 100'
required: true
default: 100
token:
description: 'The token to use to access the GitHub API'
required: true
runs:
using: 'node16'
main: 'dist/index.js'
Now, using handy toolkits from GitHub i.e. actions@core, and actions@gituhub, we will make the heart of our action i.e. action file.
const core = require("@actions/core");
const github = require("@actions/github");
const exec = require("@actions/exec");
const { Octokit } = require("@octokit/rest");
const fs = require('fs');
const convertBytes = function(bytes) {
const sizes = ["Bytes", "KB", "MB", "GB", "TB"]
if (bytes == 0) {
return "n/a"
}
const i = parseInt(Math.floor(Math.log(bytes) / Math.log(1024)))
if (i == 0) {
return bytes + " " + sizes[i]
}
return (bytes / Math.pow(1024, i)).toFixed(1) + " " + sizes[i]
}
const main = async () => {
try {
const inputs = {
token: core.getInput("token"),
target_folder: core.getInput("target_folder"),
thrashold_size: core.getInput("thrashold_size")
};
const {
payload: { pull_request: pullRequest, repository },
} = github.context;
if (!pullRequest) {
core.error("This action only works on pull_request events");
return;
}
const { number: issueNumber } = pullRequest;
const { full_name: repoFullName } = repository;
const [owner, repo] = repoFullName.split("/");
const octokit = new Octokit({
auth: inputs.token,
});
let ignoreArray = [];
let myOutput = '';
let myError = '';
const options = {};
options.listeners = {
stdout: (data) => {
myOutput += data.toString();
},
stderr: (data) => {
myError += data.toString();
}
};
/**
* Check if array assets file name contains inside .ignore-assets file or not.
* If its contains then remove those images from sourceArray and return new array.
*
* @param {Array} sourceArray Array of all assets files.
* @returns Array of files.
*/
function getAssetsIgnoreFiles(sourceArray) {
const file=`.assets-ignore`;
try {
ignoreArray = fs.readFileSync(file).toString().split("\n");
if (ignoreArray.length > 0) {
return sourceArray.filter (v => {
const fileName = v.split(" ").slice(-1).pop()
if (!fileName) return true;
return ignoreArray.indexOf(fileName) === -1;
})
}
} catch (e) {
// File not found exception.
}
return sourceArray;
}
await exec.exec(`find ${inputs.target_folder} -type f \( -name "*.jpeg" -o -name "*.png" -o -name "*.svg" -o -name "*.gif" -o -name "*.jpg" \) -size +${inputs.thrashold_size}k -exec ls -lh {} \;`, null, options);
const arrayOutput = getAssetsIgnoreFiles(myOutput.split("\n"));
const count = arrayOutput.length - 1;
const invalidFiles = [...arrayOutput];
const successBody = ` Woohooo :rocket: !!! Congratulations, your all assets are less than ${inputs.thrashold_size}Kb.`
const errorBody = `Oops :eyes: !!! You have ${count} assets with size more than ${inputs.thrashold_size}Kb. Please optimize them. If you unable to optimize these assets, you can use .assets-ignore file and add these assets in .assets-ignore file. For more details read readme`
const getTableDataString = (invalidFiles) => {
let filteredFiles = [];
for(let item of invalidFiles) {
const fileName = item.split(" ").slice(-1).pop();
const fileSize = item.split(" ")[4];
if(fileName && fileSize) filteredFiles.push([fileName, fileSize]);
}
let res = `### Invalid Files\n|File Name|File Size|\n|-----|:-----:|\n`;
for(let item of filteredFiles) {
res += `|${item[0]}|${item[1]}|\n`
}
return res;
};
/**
* Get all Ignored file data as github comment string format.
*
* @param {Array} ignoreArray array of files which is added in .assets-ignore file.
* @returns Promise of github comment string.
*/
const getAllIgnoredFileString = (ignoreArray) => {
return new Promise((resolve, reject) => {
let res = `### All .assets-ignored Files\n|File Name|File Size\n|-----|:-----:|\n`;
for(let index=0; index < ignoreArray.length; index++) {
const item = ignoreArray[index];
fs.stat(item, (err, fileStats) => {
if (err) {
res += `|${item}|None|\n`
} else {
const result = convertBytes(fileStats.size)
res += `|${item}|${result}|\n`
}
if (index === ignoreArray.length-1) {
resolve(res);
}
})
}
})
};
/**
* Publish .assets-ignore entries in github comment.
*
* @param {Array} ignoreArray array of files which is added in .assets-ignore file.
*/
const publishIgnoreAssetsTable = async (ignoreArray) => {
if (ignoreArray.length) {
const body = await getAllIgnoredFileString(ignoreArray);
return octokit.rest.issues.createComment({
owner,
repo,
issue_number: issueNumber,
body,
});
}
}
if(count > 0) {
octokit.rest.issues.createComment({
owner,
repo,
issue_number: issueNumber,
body: errorBody,
});
octokit.rest.issues.createComment({
owner,
repo,
issue_number: issueNumber,
body: getTableDataString(invalidFiles),
});
await publishIgnoreAssetsTable(ignoreArray);
core.setFailed('Invalid size assets exists !!!');
}else {
octokit.rest.issues.createComment({
owner,
repo,
issue_number: issueNumber,
body: successBody,
});
await publishIgnoreAssetsTable(ignoreArray);
}
} catch (error) {
core.setFailed(error.message);
}
};
main();
Now, after putting this final logic and building, we get our GitHub action ready to be used across any repository. To test and use our action, you can simply create a workflow file in your application under the .github folder ex. .github/workflows/asset-checker.yml, and add below code into your yml file.
name: 'Assets Checker by BharatPe'
on: [pull_request]
jobs:
Assets-Checker:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@master
- uses: actions/setup-node@v1
with:
node-version: '14.15.0'
- uses: bharatpe/assets-checker@main
with:
token: ${{ secrets.GITHUB_TOKEN }}
target_folder: src/assets
thrashold_size: 100
You can configure it according to your use case by providing input values in target_folder, thrashold_size, etc.
Note - There might be some cases where you have to use files greater in size, and for that, we have provided support with our asset-ignore config. Thanks, Sandeep Rawat for this feature.
For more information, check our repository at https://github.com/bharatpe/assets-checker and give it a star ⭐ if you like it.
Hence, my first GitHub action for BharatPe is built and I am super proud that it is being used by almost all frontend repositories across our organization. In case of any doubts feel free to comment below and our team will try to respond to you as soon as possible or connect with me at Akshay Sharma.
Do follow us on Twitter, and LinkedIn, we post our geeky stuff very frequently here. Also, if you want to join our geeky engineering team, check out the careers page. We have openings for candidates who believe in building a new generation of revolutionary fintech products and building a new INDIA together.