ReactJS Infinite Scrolling: The Ultimate Guide to Optimization

Infinite Scrolling with Collapsible Headers: The Virtual Approach

UI Scroll:

Scrolling is a method used to display data when it is not possible to fit all the content onto a single page. This implementation is commonly seen on most websites, such as Twitter and Facebook feeds or lists of items on an e-commerce app like Zillion, where the content cannot be accommodated entirely on the computer screen.

The naive way and its consequences:

To implement this concept easily, we can initially display all the data on the screen. However, a challenge arises when the number of items becomes substantial. For instance, consider Facebook feeds with a large list of items. Displaying all of them on the screen is not practical due to the potentially huge number, which could reach hundreds, thousands, or even more. Creating many DOM nodes to show the data is not an optimal solution in such cases.

Improved Approach:

One possible solution for this is implementing Pagination. In this approach, our documents are organized into pages, each containing a specific number of items. Users can navigate forward and back through the pages using either the page number or the arrow icon, similar to flipping through pages in a manual.

From a technical standpoint, this approach is logical, as we organize items by content page in our database, and we serve the correct data based on the user's input. However, we can explore a more efficient and user-friendly solution. What if we dynamically change the pages for the user as they scroll?

Infinite Scroll offers a solution to this issue. With this method, we can dynamically append pages as the user scrolls, incrementing the page one at a time until all pages are displayed. By adopting this approach, we can reduce interaction costs, particularly on mobile devices, by eliminating the need for extra clicks from users.

Implementation:

To enable infinite scroll on our list, we can attach an event listener to the scroll event. This will allow us to detect when we reach the bottom of the page and subsequently trigger the loading of more items for our list. However, doing this can result in both noise and non-performant Javascript code because of the event listeners' nature. When optimizing the front-end code, our goal should be to minimize the number of listeners whenever feasible.

Here comes Intersection Observer to save us:

The Intersection Observer is a JavaScript API that allows you to efficiently observe changes in the visibility of an element within the viewport or a specific container. It provides a way to track when an observed element enters or exits the visible part of the user's browser window.

The Intersection Observer is especially useful for implementing lazy loading of images, infinite scrolling, and other performance optimizations in web applications. It helps avoid costly computations and improves the performance of websites by reducing the need for continuous manual checks of element visibility.

The basic idea behind Intersection Observer is that you define a callback function that will be executed whenever an observed element crosses a specified threshold (i.e., enters or exits the viewport). This callback function receives a list of IntersectionObserverEntry objects that contain information about the observed element, such as its visibility percentage, bounding rectangle, and intersection ratio.

One significant advantage of using this API compared to event listeners is that the computation for both the target elements and observing the behaviour of intersecting elements is not executed on the main JavaScript thread. This means we are freed from the potential noise and blocking nature of the language.

Mechanism

However, before we proceed with the solution, we must consider a few points. While Intersection Observer offers advantages over a regular scroll event listener, it alone will not completely solve the problem. If we simply append the next data to the end of the existing list, we will still create a large number of DOM nodes, which is not an optimal solution.

We should aim to minimize the count of visible DOM elements while ensuring it is larger than the viewport (our root element). For this purpose, we will set the count to 20 in our specific case.

As observed, the total number of DOM elements remains constant, as we navigate through the complete list. To create an illusion of a large scrolling list, we dynamically modify the attributes, contents, and paddings of these elements.

We will become acquainted with several keywords related to constructing an infinite scroll feature, and then I will explore each one in detail.

Root Element
Watcher
Virtual DOM
Virtual Scrolling

The Root Element serves as our observation target for the sentinels. Whenever the sentinels intersect with this element, we initiate the computation process to either fetch new items from an API or retrieve them directly from the cache. This term is derived from the Intersection Observer API. By extending the root margin at the bottom, we ensure that the fetch call occurs well before the content reaches the viewport, thereby minimizing the waiting time for the user to receive the new data.

The Watchers serve as fixed elements that act as triggers for initiating changes. In the case of implementing an infinite scroll, we utilize top and bottom anchors to detect changes whenever they intersect with our Root Element. The top watcher operates straightforwardly, as it returns the data stored in our cache (in memory, Redis map, etc.) and displays it.

On the other hand, the bottom watcher presents a more intricate scenario. Depending on the sliding window, we must decide whether to make a network request for more data or retrieve it from the existing cache (if the user scrolls up and then back down). For our demonstration purposes, we will simplify this process by assuming that all data already exists in the cache. However, it's important to remember that in real-world scenarios, we need to include a fetch call when the data from the cache is exhausted.

The Virtual DOM plays a crucial role in preventing the need to repaint the entire browser when displaying new items. Instead of performing a full re-render, we can update the attributes to show the new data. Additionally, by strategically adjusting the container's paddings, we can create the illusion of extending the list to the user without actually expanding it in the DOM. In the demo, I will provide a detailed explanation of this implementation.

Virtual Scrolling is related to the intervals between two indices while scrolling up or down, which includes the number of visible DOM nodes. By employing this sliding window approach, we can restrict the display to a fixed amount of data, leaving the rest of the content outside the current viewport.

Now that we have covered those points, let's proceed to the actual demo.

Few notable functions:

topWatcherElmRef
bottomWatcherElmRef
topReachedCallback
bottomReachedCallback
reDrawDOM
updateTopBottomPaddings

  // When top of list is reached
  const topReachedCallback = useCallback(
    (entry) => {
      if (currentIndex.current === 0) {
        const container = document.querySelector(".box-list");
        container.style.paddingTop = "0px";
        container.style.paddingBottom = "0px";
      }
      if (entry.isIntersecting && currentIndex.current !== 0) {
        const firstIndex = getVirtualScrollArea(false);
        updateTopBottomPaddings(false);
        reDrawDOM(firstIndex);
        currentIndex.current = firstIndex;
      }
    },
    [updateTopBottomPaddings, reDrawDOM]
  );

  // When bottom of list is reached
  const bottomReachedCallback = useCallback(
    (entry) => {
      if (currentIndex.current === maxItemCount - domPageSize) {
        return;
      }
      if (entry.isIntersecting) {
        const firstIndex = getVirtualScrollArea(true);
        if (list[firstIndex + chunkSize]) {
          currentIndex.current = firstIndex;
          updateTopBottomPaddings(true);
          reDrawDOM(firstIndex);
        } else {
          if (hasMore) {
            setPage((pg) => pg + 1);
            return;
          }
        }
      }
    },
    [updateTopBottomPaddings, hasMore, maxItemCount, reDrawDOM, list]
  );

  // Top element that act as triggers for initiating changes
  const topWatcherElmRef = useCallback(
    (node) => {
      if (startElmObserver.current) startElmObserver.current.disconnect();
      startElmObserver.current = new IntersectionObserver((entries) => {
        topReachedCallback(entries[0]);
      });
      if (node) startElmObserver.current.observe(node);
    },
    [topReachedCallback]
  );

  // Bottom element that act as triggers for initiating changes
  const bottomWatcherElmRef = useCallback(
    (node) => {
      if (lastElmObserver.current) lastElmObserver.current.disconnect();
      lastElmObserver.current = new IntersectionObserver((entries) => {
        bottomReachedCallback(entries[0]);
      });
      if (node) lastElmObserver.current.observe(node);
    },
    [bottomReachedCallback]
  );

topWatcherElmRef and bottomWatcherElmRef will point to the top and bottom elements of our list, respectively. They will be used to set up observers for both elements. The callbacks, topReachedCallback and bottomReachedCallback, will be executed whenever the watchers intersect the current viewport.

  // Append next elements to list
  const reDrawDOM = useCallback(
    (firstIndex) => {
      const items = [];
      for (let i = 0; i < domPageSize; i++) {
        if (list[i + firstIndex]) {
          items.push(list[i + firstIndex]);
        }
      }
      setListItems([...items]);
    },
    [list]
  );

The reDrawDOM function is utilized to recycle existing DOM nodes. Depending on your requirements, you may need to make modifications or even force a fresh render on specific elements. Nevertheless, the crucial aspect to remember is that we are only replacing content, not entirely mounting or unmounting new elements each time the Sentinels intersect with the root element.

  // Update top and bottom pading to virtualize scrolling
  const updateTopBottomPaddings = useCallback((isScrollDown) => {
    if (currentIndex.current === 0) {
      return;
    }
    const container = document.querySelector(".box-list");
    const currentPaddingTop = extractNumber(container.style.paddingTop);
    const currentPaddingBottom = extractNumber(container.style.paddingBottom);
    const remPaddingsVal = 170 * chunkSize + 10;
    if (isScrollDown) {
      container.style.paddingTop = currentPaddingTop + remPaddingsVal + "px";
      container.style.paddingBottom =
        currentPaddingBottom === 0
          ? "0px"
          : currentPaddingBottom - remPaddingsVal + "px";
    } else {
      container.style.paddingBottom =
        currentPaddingBottom + remPaddingsVal + "px";
      container.style.paddingTop =
        currentPaddingTop === 0
          ? "0px"
          : currentPaddingTop - remPaddingsVal + "px";
    }
  }, []);

The updateTopBottomPaddings method is utilized to obtain new paddings and directly modify the container's size. This is necessary because, as we recycle our DOM nodes, the actual content remains fixed with a specific number of elements. By expanding the paddings in both directions, we simulate the list extending larger or becoming smaller as we scroll up and down, precisely by the same amount as the heights of the items being removed.

In certain scenarios, we may also need to calculate and cache the heights of the elements being removed to add paddings accordingly. For our demonstration's convenience, each row's height is set to 140px, with a margin of 10px at the top and bottom. Consequently, we are removing and adding (140 + 10 + 10) * the number of items being removed on scrolling

Add-On: Collapsible Headers on Scroll

So far, we have covered infinite scroll most optimally. However, what if we also want the header elements to collapse when the user starts scrolling down, to increase the screen size for the viewers? Conversely, when the user starts scrolling up, the headers should become visible again, making filters and similar features easily accessible to the user just like most e-commerce platforms do on mobile devices. We successfully implemented this feature in BharatPe's Loyalty App - Zillion.

Mechanism:

When the user starts scrolling down, we detect the scroll direction and calculate the offset. Using this information, we gradually collapse the header DOM element pixel by pixel. The headers are stacked on top of each other, creating a visually pleasing effect. For instance, if there are three header elements, and we have set two of them as collapsible, these two headers will move below the third one. However, if all three headers are set as collapsible, they will all stack above the viewport, becoming invisible to the user.

Conversely, when the user starts scrolling up, we once again capture the scroll direction and offset. Utilizing this data, we begin to unstack the header elements in the same order in which they collapsed. As a result, the headers become visible again, and the user can easily access filters and other features. This dynamic collapsing and uncollapsing of headers enhance the user experience, providing more screen space when needed and re-displaying the headers as the user scrolls back up.

As you can see, the header elements are translated on the y-axis and move out of sight as the user scrolls through the list.

Let’s check out the above mechanism in the code

We have created a useScrollDirection hook, which we use to get scroll direction and scroll offset

import { useEffect, useState } from "react";

function useScrollDirection(element) {
  const [scrollOffset, setScrollOffset] = useState(0);
  const [lastScroll, setLastScroll] = useState(0);
  const [scrollDirection, setScrollDirection] = useState("UP");

  useEffect(() => {
    let updateScrollDirection = () => {
      /**/
    };
    let lastScrollY = 0;
    if (element && element.addEventListener) {
      lastScrollY = element.scrollTop;
      updateScrollDirection = () => {
        const scrollY = element.scrollTop;
        const direction = scrollY > lastScrollY ? "DOWN" : "UP";

        if (
          direction !== scrollDirection &&
          (scrollY - lastScrollY > 0 || scrollY - lastScrollY < 0)
        ) {
          setScrollDirection(direction);
        }
        setLastScroll(lastScrollY);
        setScrollOffset(scrollY);
        lastScrollY = scrollY > 0 ? scrollY : 0;
      };
      element.addEventListener("scroll", updateScrollDirection);
    }
    return () => {
      if (element && element.removeEventListener) {
        element.removeEventListener("scroll", updateScrollDirection);
      }
    };
  }, [scrollDirection, element]);

  return {
    scrollDirection,
    scrollOffset,
    lastScroll
  };
}

export default useScrollDirection;

Initially, the HeaderElements component creates a reference for each header element. Then, it calculates the height of each header element. Using this height information, the component positions each header at its correct place by assigning the translateY() property. For example, if there are three header elements with heights of 40px, 50px, and 60px respectively, the positioning would be as follows:

Header element 1: transform: translateY(0px)
Header element 2: transform: translateY(40px) (height of header 1)
Header element 3: transform: translateY(90px) (sum of heights of header 1 and header 2)

  useEffect(() => {
    let ht = -1;
    const offSets = [];
    const tys2 = [];
    if (headerElements && headerElements.length) {
      headerElements.forEach((_data, index) => {
        tys2.push(0);
        const htmlElement = getElement(index);
        if (htmlElement) {
          if (ht < 0 && htmlElement["clientHeight"]) {
            ht = 0;
          }
          ht = ht + htmlElement["clientHeight"];
        }

        let htt = 0;
        for (let i = 0; i < index; i++) {
          const htmlElement = getElement(i);
          if (htmlElement && headerElements[i].collapse) {
            htt = htt + htmlElement["clientHeight"] || 0;
          }
        }
        offSets.push(htt);
      });
    }
    setTopPadding(ht);
    setTransformOffSets(offSets);
    setTYS(tys2);
    setHeaderHeight(ht);
  }, [headerElements, setTopPadding]);

  return (
    <>
      {headerElements && headerElements.length ? (
        <div
          ref={headRef}
          className="header-wraper"
          style={{
            height: `${headerHeight}px`,
            zIndex: headerElements.length + 1
          }}
        >
          {headerElements.map((data, index) => (
            <div
              key={index}
              className="header"
              ref={(element) => (headerRef.current[index] = element)}
              style={{
                zIndex: headerElements.length - index,
                transform: `translateY(${tys[index]}px)`,
                backgroundColor: data.backgroundColor,
                position: "relative"
              }}
            >
              {data.content}
            </div>
          ))}
        </div>
      ) : null}
    </>
  );

Once the user starts scrolling, the system will detect the change, and the header elements will be dynamically updated accordingly. This will cause the headers to either collapse or become visible based on the direction of the scroll. For example, if the user scrolls down 40 px then if header elements will have translateY as follows:

Header element 1: transform: translateY(-40px)
Header element 2: transform: translateY(0px)
Header element 3: transform: translateY(50px)

Hiding header element 1 from the user's view.

  useEffect(() => {
    let nch = 0;
    let ch = 0;
    headerElements.forEach((data, i) => {
      const htmlElement = getElement(i);
      let val = undefined;
      if (htmlElement) {
        const offset = transformOffSets[i];
        const th = offset + (htmlElement["clientHeight"] || 0);
        const style = window.getComputedStyle(htmlElement);
        const matrix = new DOMMatrixReadOnly(style.transform);
        const prevTyVal = matrix.m42;
        if (scrollDirection === "DOWN") {
          if (data.collapse) {
            if (Math.abs(prevTyVal) < Math.abs(th)) {
              const actualO = lastScroll - scrollOffset;
              if (prevTyVal + actualO < -th) {
                val = -th;
              } else {
                val = prevTyVal + actualO;
              }
            }
          } else {
            if (Math.abs(prevTyVal) < Math.abs(offset)) {
              const actualO = lastScroll - scrollOffset;
              if (prevTyVal + actualO < -offset) {
                val = -offset;
              } else {
                val = prevTyVal + actualO;
              }
            }
          }
        } else if (scrollDirection === "UP") {
          if (lastScroll >= scrollOffset) {
            const actualO = lastScroll - scrollOffset;
            if (prevTyVal + actualO > 0) {
              val = 0;
            } else {
              val = prevTyVal + actualO;
            }
          } else {
            val = 0;
          }
        }
        if (!data.collapse) {
          nch = nch + htmlElement["clientHeight"] || 0;
        } else {
          ch = ch + htmlElement["clientHeight"] || 0;
        }
      } else {
        val = 0;
      }
      if (val !== undefined) {
        setTYS((ty) => {
          ty[i] = val;
          return [...ty];
        });
      }
    });

    if (headRef && headRef.current) {
      const th = nch + ch;
      const prevH = headRef.current["clientHeight"] || 0;
      if (scrollDirection === "DOWN") {
        if (prevH > nch) {
          const actualO = lastScroll - scrollOffset;
          if (prevH + actualO < nch) {
            setHeaderHeight(nch);
          } else {
            setHeaderHeight((ht) => {
              ht = prevH + actualO;
              return ht;
            });
          }
        }
      } else if (scrollDirection === "UP") {
        const actualO = lastScroll - scrollOffset;
        if (lastScroll >= scrollOffset) {
          if (prevH + actualO >= th) {
            setHeaderHeight(th);
          } else {
            setHeaderHeight((ht) => {
              ht = prevH + actualO;
              return ht;
            });
          }
        } else {
          setHeaderHeight(th);
        }
      }
    }
  }, [
    scrollDirection,
    headerElements,
    scrollOffset,
    lastScroll,
    transformOffSets
  ]);

Conclusion:

Developing a high-performance infinite scroll feature can be challenging, but the Intersection Observer API makes it achievable and efficient. Aside from infinite scrolling, Intersection Observer can also be utilized for lazy loading images, running animations, and various other tasks. Please feel free to test the complete demo on CodeSandbox given below and share your feedback with us!

https://codesandbox.io/s/infinite-scroll-n5pv6w?file=/src/useScrollDirection.js:55-73

Supercharge Your ReactJS App with Optimized Infinite Scrolling

Comments (1)

Tech

Unfolding The Service Worker

More from this blog

Seamless State Management: Redux in Next.js with Hydration

Next.js Monitoring: From Setup to Visualization with Prometheus and Grafana

AssetGuard: Keeping Assets in Check

My experience as a Software Engineer Intern at BharatPe

UI Scroll:

The naive way and its consequences:

Improved Approach:

Implementation:

Here comes Intersection Observer to save us:

Mechanism

Few notable functions:

Add-On: Collapsible Headers on Scroll

Mechanism:

Conclusion:

Command Palette

Comments (1)

Tech

Unfolding The Service Worker

More from this blog

UI Scroll:

The naive way and its consequences:

Improved Approach:

Implementation:

Here comes Intersection Observer to save us:

Mechanism

Few notable functions:

Add-On: Collapsible Headers on Scroll

Mechanism:

Conclusion: