How to add more data to datatables.net table when you don't know the number of final rows you'll get and you can only pull 100 rows at a time?

I'm using DataTables.net and what I'm doing is pulling the first 100 rows. Each group is only 100. The PHP call is for page 1 at first.

Here's the link to my site what I'm trying to do in a rudimentary fashion, not yet incorporated in my full site:

http://magicdjs.com/getinventoryitemsaddedbetween1.html

Now, what I need to do, to gather all the rows, is keep pulling until 100 is NOT the number of rows. This means, I've reached the end. I'm told there are 70 pages of 100 rows each with the last one being less than 100.

Here's my JS call:

/* global currentTable */

let theRows;
let theTable;
let theAjax;
let theColumns;
let pageNbr = 1;
let newPageNbr = 0;

let dt;
let detailRows = [];
let myTable, myAjax, myRows, myCols;
let myJSON;
let table;
let headers;
let $ = jQuery.noConflict();
let data = [];
let fixedData = [];

'use strict';

headers = {
    'Access-Control-Allow-Origin': 'http, data',
    'Access-Control-Allow-Methods': ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS'],
    'Access-Control-Allow-Headers': 'Content-Type',
    //'Content-Type': 'application/json',
    'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8'
};
// DT Debugger. Uncomment to use
// (function() {
//     var url = 'https://debug.datatables.net/bookmarklet/DT_Debug.js';
//     if (typeof DT_Debug != 'undefined') {
//         if (DT_Debug.instance !== null) {
//             DT_Debug.close();
//         } else {
//             new DT_Debug();
//         }
//     } else {
//         var n = document.createElement('script');
//         n.setAttribute('language', 'JavaScript');
//         n.setAttribute('src', url + '?rand=' + new Date().getTime());
//         document.body.appendChild(n);
//     }
// })();

theTable = "magicdatatable";

$(document).ready(async() => {

    if (sessionStorage.getItem("magic") === null) {

        callMurphs = () => {

            let url = "services.php";
            let user_id = "djpeterb@magicdjs.com";


            try {
                $.ajax({
                    type: 'POST',
                    url: url,
                    dataType: 'json',
                    data: { user_id: user_id, page: pageNbr },
                    success: (thedata) => {
                        console.log("data returned: ", thedata);
                        if (thedata.status == 200) {
                            console.log(thedata.statusText + "data returned: ", thedata);
                            data = [];
                            data.push(thedata);
                            console.log("New Data: ", data);
                            data.map(el => {
                                delete el["status"],
                                    delete el["statusText"]
                            });
                            sessionStorage.setItem("magic1", JSON.stringify(data[0]));
                            data = [];
                            data.push(sessionStorage.getItem("magic0"));
                            callTable(1);
                        } else {
                            console.log("no data returned inside success: ", thedata.statusText);
                            sessionStorage.setItem("magic1", "No Magic Found Captain!" + thedata.statusText);
                        }
                    },
                    complete: function(data) {
                        console.log("All Done!");
                    }
                });
            } catch (e) {
                console.log("An Error occured: ", e);
            }

        }

        callMurphs();

    } else {
        data = [];
        if (data.length === 0) {
            data.push(sessionStorage.getItem("magic1"));
        }
        table = $("#" + theTable).DataTable({

            "data": JSON.parse(sessionStorage.getItem("magic0")).data,
            "columns": [{
                    "className": 'dt-control',
                    "orderable": false,
                    "data": null,
                    "defaultContent": ''
                },
                {
                    "data": "id",
                    className: "producttabletxt"
                },
                {
                    "data": "title",
                    className: "producttabletxt"
                },
                {
                    "data": null,
                    className: "producttabletxt",
                    render: (data, type, row) => {
                        let retVal = "";

                        if (data.categories.string === undefined) {
                            retVal = "<strong>No Categories Listed</strong>";
                        } else {
                            retVal = data.categories.string;
                        }
                        return retVal;
                    }
                },
                {
                    "data": null,
                    className: "producttabletxt",
                    render: (data, type, row) => {
                        let retVal = "";

                        if (data.artist_magician === "") {
                            retVal = "<strong>No Magician Listed</strong>";
                        } else {
                            retVal = data.artist_magician;
                        }
                        return retVal;
                    }
                },
                {
                    "data": "qty",
                    className: "producttabletxt"
                },
                {
                    type: "currency",
                    "data": null,
                    render: (data, type, row) => {
                        return "<strong>$" + parseFloat(data.price).toFixed(2) + "</strong>";
                    },
                    className: 'dt-body-right'
                }
            ],
            "order": [
                [1, "asc"]
            ]
        });
    }

    callTable = (page) => {

            let sessionItemPage;

            if (page === null) {
                sessionItemPage = "magic0";
            } else {
                sessionItemPage = "magic" + page;
            }

            table = $("#" + theTable).DataTable({

                "data": JSON.parse(sessionStorage.getItem(sessionItemPage)).data,
                "columns": [{
                        "className": 'dt-control',
                        "orderable": false,
                        "data": null,
                        "defaultContent": ''
                    },
                    {
                        "data": "id",
                        className: "producttabletxt"
                    },
                    {
                        "data": "title",
                        className: "producttabletxt"
                    },
                    {
                        "data": null,
                        className: "producttabletxt",
                        render: (data, type, row) => {
                            let retVal = "";

                            if (data.categories.string === undefined) {
                                retVal = "<strong>No Categories Listed</strong>";
                            } else {
                                retVal = data.categories.string;
                            }
                            return retVal;
                        }
                    },
                    {
                        "data": null,
                        className: "producttabletxt",
                        render: (data, type, row) => {
                            let retVal = "";

                            if (data.artist_magician === "") {
                                retVal = "<strong>No Magician Listed</strong>";
                            } else {
                                retVal = data.artist_magician;
                            }
                            return retVal;
                        }
                    },
                    {
                        "data": "qty",
                        className: "producttabletxt"
                    },
                    {
                        type: "currency",
                        "data": null,
                        render: function(data, type, row) {
                            return "<strong>$" + parseFloat(data.price).toFixed(2) + "</strong>";
                        },
                        className: "dt-body-right producttabletxt"
                    }
                ],
                "order": [
                    [1, "asc"]
                ]
            });
        }
        // Add event listener for opening and closing details
    $("#" + theTable).on('click', 'td.dt-control', function() {
        var tr = $(this).closest('tr');
        var row = table.row(tr);

        if (row.child.isShown()) {
            // This row is already open - close it
            row.child.hide();
            tr.removeClass('shown');
        } else {
            // Open this row
            row.child(format(row.data())).show();
            tr.addClass('shown');
        }
    });
    // }
});

/* Formatting function for row details - modify as you need */
//console.log("THIS after the FORMAT Datatable: ", clicked);
format = (d) => {

    // `d` is the original data object for the row
    let tblTemp;
    console.log("Original D: ", d);
    //Find out which table we are and pull that detail template...
    switch (theTable) {
        case "magicdatatable":

            if (d.artist_magician === "") {
                d.artist_magician = "NA";
            }

            tblTemp = '<table id="magicproductdet" class="table table-bordered table-responsive table-striped" style="padding-left:150px;">' +
                '<tr>' +
                '<td class="producttabletxt" colspan=2><strong>Magician</strong></td>' +
                '<td class="producttabletxt" colspan=2><strong>' + d.artist_magician + '</strong></td>' +
                '</tr>' +
                '<tr>' +
                '<td class="producttabletxt" colspan=2>' + d.htmldesc + '</td>' +
                '</tr>' +
                '<tr>' +
                '<td colspan=2>' +
                '<img class="productimg" src="https://www.murphysmagicsupplies.com/images/' + d.imgfilename + '" alt="' + d.altimages + '"></td>' +
                '</td>' +
                '</tr>' +
                '</table>';
            break;
        default:
            break;
    }
    return tblTemp;
};

checkAllItems = () => {

    let arrayOfKeys = Object.keys(sessionStorage);

    // for (let i = 0; i < sessionStorage.length; i++) {
    //     console.log("SessionStorage List of Items: ", sessionStorage.getItem(sessionStorage.key(i)));
    // }

    console.log("SessionStorage List of Item Names: ", arrayOfKeys);
    newPageNbr = arrayOfKeys.length + 1;
    addMoreData(newPageNbr)
}

addMoreData = (pgnbr) => {

    let url = "webservices/vendor/murphs/murphs-services.php";
    let user_id = "djpeterb@magicdjs.com";

    console.log("New Page Number: " + pgnbr);

    try {
        $.ajax({
            type: 'POST',
            url: url,
            dataType: 'json',
            data: { user_id: user_id, page: pgnbr },
            success: (thedata) => {
                console.log("data returned: ", thedata);
                if (thedata.status == 200) {
                    console.log(thedata.statusText + "data returned: ", thedata);
                    data = [];
                    data.push(thedata);
                    console.log("New Data: ", data);
                    data.map(el => {
                        delete el["status"],
                            delete el["statusText"]
                    });
                    this.addEntry(data);
                    sessionStorage.setItem("magic" + pgnbr, JSON.stringify(data[0]));
                    data = [];
                    data.push(sessionStorage.getItem("magic" + pgnbr));
                    // callTable(pgnbr);
                } else {
                    console.log("no data returned inside success: ", thedata.statusText);
                    sessionStorage.setItem("magic" + pgnbr, "No Magic Found Captain!" + thedata.statusText);
                }
            },
            complete: function(data) {
                console.log("All Done!");
            }
        });
    } catch (e) {
        console.log("An Error occured: ", e);
    }

}

addEntry = (newEntry) => {

    let newSortedData = {
        data: []
    };

    // Parse any JSON previously stored in allEntries
    let existingEntries = JSON.parse(sessionStorage.getItem("magic0")).data;
    if (existingEntries === null) {
        existingEntries;
    }
    sessionStorage.setItem("magic1", JSON.stringify(newEntry));
    // Save allEntries back to local storage
    let newEntries = existingEntries.concat(newEntry[0].data)

    console.log("existing entries with push for new ones: ", newEntries);

    table.clear().draw();
    newSortedData.data.push(newEntries);
    table.rows.add(_.sortBy(newSortedData, "title")); // Add new data
    table.columns.adjust().draw(); // Redraw the DataTable

    console.log("Existing and new ones sorted: ", newSortedData);

    sessionStorage.setItem("magic1", JSON.stringify(newSortedData));


};

What I'm trying to do is when I click "ADD MORE ROWS" button, I want to pull the next set, based on how many I have, then, concatenate the subsequent sets into the main magic0. So, essentially, I can have 70 JSON objects like you'll see when you open the DEV console.

When I push ADD MORE ROWS, I add to the counter, 1, and send that NEW number as a page number into the PHP to call the very next set of rows until I hit rows that are less than 100, meaning, I've reached the final set.

What's happening, is that I keep overwriting the original magic0 with magic(n). I want to concatenate not replace.

Anyway, the reason for the two callTable calls is that the one buried in the callMurphs() callTable(1); can FIND the callTable function but, cannot when I'm here in the ELSE

} else {
        data = [];
        if (data.length === 0) {
            data.push(sessionStorage.getItem("magic1"));
        }
        table = $("#" + theTable).DataTable({
...

The reason I have to DUPLICATE the entire callTable function again which is nonsense.

It keeps saying, callTable is not a function otherwise.

So, yes, there are a few things but instead of making 70 PHP scripts with 100 rows each, I built one call to the PHP using an incremental system calling it ONLY when I want more rows. BUT, therein lies my problem. I'd like all 7000 rows but in a lazy loading manner which I know DataTables can do "IF" you don't have to make 70 and only 1 call with 7000+ rows. See my issue?

Thank you.

If you need more info, please ask. The PHP is proprietary from Murphy's Magic therefore I cannot show you the KEY phrase.

Via Active questions tagged javascript - Stack Overflow https://ift.tt/2FdjaAW