How to Optimize jqGrid for Large Datasets

I. Introduction

In enterprise systems, displaying large datasets in tables is common. However, performance problems appear when the dataset grows to millions of records. Therefore, without proper optimization, system performance gradually declines.

This article examines the following aspects:

• • Common performance problems in grid components handling large datasets.
  • The root causes of these issues.
  • Practical optimization strategies for production environments.

II. Typical Performance Problems

When a data grid processes large datasets, performance issues typically occur across multiple layers:

• The database executes the query. Poor indexing can significantly increase response time.
• The data travels through the network. Large JSON payloads increase transfer latency.
• In addition, browser stores the response in memory, raising RAM consumption.
• The browser renders the data into DOM elements,triggering reflow and repaint operations.

1. Slow Initial Load

One of the most common problems is a slow initial load. In many cases, the page freezes right after opening. The page may stop responding for several seconds, users think the system is unstable.

Therefore, this usually happens because too many rows are loaded at once. When thousands of records are returned,  the grid tries to render everything immediately. The main causes include:

• A large number of DOM elements
• Heavy layout and repaint operations
• High CPU usage

2. Lag When Paging or Sorting

When a grid handles large datasets, performance issues can affect user interaction. Users may notice slow paging or sorting, increased memory usage, and delays before data reaches the browser. As a result, the grid becomes slow and less responsive.

• Lag When Paging or Sorting:
  Client-side paging and sorting load the entire dataset into the browser. In addition, each interaction forces the grid to reprocess large amounts of data, which can cause slow responses and temporary UI freezes.
• High Memory Usage:
  Storing the full dataset in browser memory consumes significant RAM. Large JSON responses and repeated reloads increase memory pressure, which may degrade performance or even crash the page.
• Network Bottleneck:
  APIs that return unnecessary columns or full datasets generate large payloads. This increases download and parsing time, slowing client-side rendering.

3. High Memory Usage

Another critical issue is high memory usage when processing large datasets. Over time, browser memory usage increases, and in extreme cases, the page may crash, especially on devices with limited resources.

• When the full dataset is stored in the browser’s memory, RAM usage increases rapidly.
• In addition, large JSON responses and internal rendering references create significant memory pressure.
• Client-side rendering slows down, further degrading overall performance.

4. Network Bottleneck

Also, performance issues can occur even before data reaches the browser. Large API responses may slow down data transfer, affecting user experience—even on fast, stable connections.

• As a result, the dataset remains in browser memory, which increases RAM usage.
• Large JSON responses and internal references for sorting and filtering add memory overhead.
• As memory usage grows, rendering slows and system responsiveness declines.

III. Solution Strategy: Migrating to a Server-Side Architecture

To address the limitations discussed earlier, the most effective solution is to shift data processing from the client to the server. This approach uses a lazy loading strategy, loading only the data required for the current page.

1. Server-side Pagination

Instead of transferring the entire dataset to the browser, the system should return only the records required for the current page (for example, 20 or 50 records per request).

In practice, this approach fundamentally changes how the grid handles large datasets.

• Operating Principle:
  jqGrid automatically sends navigation parameters (page, rows, sidx, sord) to the API.
  The backend uses these parameters to construct an SQL query with clauses such as LIMIT and OFFSET.
• Impact:
  This reduces payload size from megabytes to kilobytes and significantly lowers network overhead.

2. Rendering Optimization with gridview and Virtual Scrolling

• Gridview with true Mechanism:
  When enabled, jqGrid does not insert rows into the DOM one by one. Instead, it generates the entire HTML string for the dataset and performs a single insertion operation. This significantly minimizes browser reflow and repaint cycles.
• Virtual Scrolling (scroll: 1):
  This technique loads data dynamically as the user scrolls. It creates the illusion of an infinite list while keeping the number of DOM elements to a minimum.

3. Detailed Comparison Table: Client-Side vs Server-Side Processing

IV. How to implement

Now, let's look at a real-world example. Below, you will find the frontend setup and the backend logic.

1. Frontend Configuration

The following configuration enables jqGrid server-side pagination and improves performance when processing large datasets.

JavaScript

// Collect search parameters
var inputData = {
    status: $('[name="status"]').val(),
    name:   $('[name="name"]').val(),
    id:     $('[name="id"]').val()
};

// Reload jqGrid with server-side config
$('#tablejqGrid').jqGrid('setGridParam', {
    url: '/bpm/logic/api/your_url_api', // The API endpoint to fetch data from
    datatype: 'json',
    mtype: 'POST',
    postData: inputData, // Send the collected search parameters along with the request
    contentType: 'application/json',
    serializeGridData: function (data) {
        return JSON.stringify(data); // Convert the data object into a JSON string before sending
    },
    page: 1, // Reset table grid to the first page when reloading
    jsonReader: { // Define how jqGrid should interpret the JSON response from the server
        root: 'rows',    //the actual data array is located
        page: 'page',    //the current page
        total: 'total',  //the total pages
        records: 'records', //total number of records in the database
        repeatitems: false // Each row is an object, not an array
    }
}).trigger('reloadGrid');  // Trigger reload so the grid fetches new data

{
   page: 1,    //The current page requested by the user.
   rows: 50,   //Number of records per page.
   sidx: "id"  //Sort index (column name used for sorting). 
   sord: "asc" //"asc" = ascending && "desc" = descending
   inputData: { /* search conditions */ }
}

2. Backend Logic

For example, imagine you have a huge table of records , like a spreadsheet with hundreds or thousands of rows. With that you don’t want to show all rows at once , because that would be slow and confusing. Instead, you show them page by page like flipping through pages of a book.

Specifically, the backend receives the following parameters:

Next, calculate the offset using the following formula:

Offset = (page - 1) × rows
Offset = (1 - 1) × 50 = 0

Explanation:

• Start at records 0
• Start returning rows from record 1 .

Think of it like a book:

• Page 1 → rows 1–50
• Page 2 → rows 51–100
• Page 3 → rows 101–150

Finally, the backend executes the SQL query

SELECT status,name,id    --Only get the columns we need 
FROM your_table
WHERE 
-- your condition 
ORDER BY 
/*IF sidx != null && sidx!='' && sord !=null && sord!=''*/
/*$sidx*/id /*$sord*/ASC
LIMIT 50 OFFSET 0;     --Limit: Return 50 records (page size) && OFFSET: Skip the first 0 records (calculated from page number)

3. Front-end JSON Response

JSON

{
  "page": 1,
  "total": 2, //Count total page using
  "records": 95, //Count all recrods to load in table
  "rows": [ 
    { "id": 1, "name": "Cristiano", "status": "Active" },
    { "id": 2, "name": "Lionel", "status": "Inactive" },
    { "id": 3, "name": "Lamine", "status": "Inactive" },
    ....,
  ]
}
 

V. Conclusion

In summary, optimizing a large data grid is not a simple adjustment or quick fix. It requires coordination between the frontend and backend. By implementing jqGrid server-side pagination, enabling gridview optimizations, and controlling JSON payload size, the system remains stable and scalable. These strategies maintain consistent performance, even when processing millions of records.

Whether you need scalable software solutions, expert IT outsourcing, or a long-term development partner, ISB Vietnam is here to deliver. Let’s build something great together—reach out to us today. Or click here to explore more ISB Vietnam's case studies. 

※ References:

(1) JQurery jqGrid table documentation: https://www.guriddo.net/documentation/guriddo/javascript/

(2) Why jqgrid huge data problems: https://stackoverflow.com/questions/6211633/jqgrid-huge-data-load-problems