Parse is great, even if they are planning to shutdown their services in 2017… However, like a lot of other APIs they limit the amount of data you retrieve from them via a single call. The default value is 100 and can be maxed up to 1000 but what if you have 1001+ objects you’d like to retrieve?

Well… some solutions…

A lot of people who are looking for a solution are mentioning limit() alongside with skip() from Parse JS SDK.

https://parse.com/questions/loading-more-than-100-objects http://stackoverflow.com/questions/30562620/api-100-objects-limit

It’s basically like a pagination system where you can use offsets and ask objects skipping the first 1000 and retrieve the next 1000. You would obviously have to repeat that. It’s convenient and you could do that but at some point, the Parse API might throw that nasty error at your face.

Error: Skips larger than 10000 are not allowed.

Under the hood, Parse is using MongoDB database engine, which as you may know, is using the NoSQL technology. Don’t get me wrong, NoSQL is great, especially when reliability and performances are keys. However, one of the limitation or “cons” often mentioned is “pagination”.

Another solution: Server-side scripts - Parse CloudCode!

The CloudCode cloudcode/cloud/main.js.

Parse.Cloud.define("retrieveAllObjects", function(request, status) {
    var result     = [];
    var chunk_size = 1000;
    var processCallback = function(res) {
        result = result.concat(res);
        if (res.length === chunk_size) {
            process(res[res.length-1].id);
        } else {
            status.success(result);
        }
    };
    var process = function(skip) {
        var query = new Parse.Query(request.params.object_type);
        if (skip) {
            query.greaterThan("objectId", skip);
        }
        if (request.params.update_at) {
            query.greaterThan("updatedAt", request.params.update_at);
        }
        if (request.params.only_objectId) {
            query.select("objectId");
        }
        query.limit(chunk_size);
        query.ascending("objectId");
        query.find().then(function (res) {
            processCallback(res);
        }, function (error) {
            status.error("query unsuccessful, length of result " + result.length + ", error:" + error.code + " " + error.message);
        });
    };
    process(false);
});

Disclaimer: this was highly inspired by some snippets we found across multiple websites. We’ll look into adding the links to this article…

Plenty of useful information on how CloudCode works and how to deploy it, directly accessible on the Parse documentation. This is the code you need to deploy on your Parse instance and basically this will create a new endpoint for you to “consume”.

Once it’s deployed, it can be consumed by using either a Promise (or not, it’s up to you really):

Parse.Cloud.run('retrieveAllObjects', {
    object_type: "MyClass", // REQUIRED - string: name of your Parse class
    update_at: moment().toDate(), // OPTIONAL - JS Date object: Only retrieve objects where update_at is higher than...
    only_objectId: true|false // OPTIONAL - boolean: the result will only be composed by objectId + date fields, otherwise all attributes are returned.
}).then(function(objects) {
    /* SUCCESS */
    // if objects.length > 0 objects can be looped through
});

We assume you know how to use Parse JS SDK.

By the way, Promises (promise.then()) and MomentJS (moment().toDate()) are really two awesome tools. Definitely recommend you check those out! We use the Q library for Promises, which is the one supported by the Parse JS SDK as well.

Of course, we’re not saying our solution is the best, we simply wanted to share it with you so you can implement it if you were facing that very same problem! ;)

The big advantage of our approach is that we don’t use skip and limit, we order our result by objectId which is the PK (primary key) on Parse and use a self-invoked function alongside with the PK as a pagination controller for the offset. This way, you won’t ever be limited to 10000 results.

Obviously this takes a bit longer to get executed but the workload stays server-side and at least you know that the response you receive contains everything. Server-side scripts could also be a potential approach for other pagination problems.

That’s all folks! Check out the gist for this and feel free to comment/fork/improve/share it!