Pagination in Ember with a JSON API Backend

Setting up pagination with a JSON API backend is not as straight-forward as it should. What is the recommended way to handle this with Ember Data?

How do we deal with query parameters? How about the UI?

A glance at the spec

First step: check what the JSON API specification has to say about pagination. Basically:

Pagination links MUST appear in the links object that corresponds to a collection. To paginate the primary data, supply pagination links in the top-level links object.

[…]

The following keys MUST be used for pagination links:

  • first: the first page of data
  • last: the last page of data
  • prev: the previous page of data
  • next: the next page of data

There is also an example we can refer to. It’s simple. Given a query to /articles?page[number]=3&page[size]=1, the response shall be:

{
  "data": [
    {
      "type": "articles",
      "id": "3",
      "attributes": {
        "title": "JSON API paints my bikeshed!",
        "body": "The shortest article. Ever."
      }
    }
  ],
  "links": {
    "self": "http://example.com/articles?page[number]=3&page[size]=1",
    "first": "http://example.com/articles?page[number]=1&page[size]=1",
    "prev": "http://example.com/articles?page[number]=2&page[size]=1",
    "next": "http://example.com/articles?page[number]=4&page[size]=1",
    "last": "http://example.com/articles?page[number]=13&page[size]=1"
  }
}

Crystal clear. Let’s jump on to our own sample app.

Consuming the paginated API

Let’s assume we have a working backend.

If you don’t have a compliant backend, here is a quick Rails 5 beta example with Kaminari.

$ gem install rails --pre
# create a new Rails API app specifying the version you just installed
$ rails _5.0.0.beta3_ new articles-api --api

Generate a scaffold for articles

$ cd articles-api
$ bin/rails generate scaffold article title:string body:string

Create seed

# db/seeds.rb

Article.create([
  { title: "Article 1", body: "Lorem 1 ipsum" },
  { title: "Article 2", body: "Lorem 2 ipsum" },
  { title: "Article 3", body: "Lorem 3 ipsum" },
  { title: "Article 4", body: "Lorem 4 ipsum" },
  { title: "Article 5", body: "Lorem 5 ipsum" },
  { title: "Article 6", body: "Lorem 6 ipsum" },
  { title: "Article 7", body: "Lorem 7 ipsum" },
  { title: "Article 8", body: "Lorem 8 ipsum" },
  { title: "Article 9", body: "Lorem 9 ipsum" },
  { title: "Article 10", body: "Lorem 10 ipsum" },
  { title: "Article 11", body: "Lorem 11 ipsum" },
  { title: "Article 12", body: "Lorem 12 ipsum" },
  { title: "Article 13", body: "Lorem 13 ipsum" }
])

Migrate and seed

$ bin/rails db:migrate
$ bin/rails db:seed

Now we have a functional Rails API with data! Next step is to format the data in JSON API and support pagination:

# add (working) gems to our Gemfile
$ echo 'gem "active_model_serializers", github: "rails-api/active_model_serializers", tag: "v0.10.0.rc4"' >> Gemfile
$ echo 'gem "kaminari"' >> Gemfile

# configure AMS for JSON API
$ echo "ActiveModel::Serializer.config.adapter = ActiveModel::Serializer::Adapter::JsonApi" > config/initializers/json_api.rb

# generate article serializer
$ bin/rails g serializer article  # EDIT to include attributes :id, :title, :body

$ bundle

In the controller, make sure to paginate if the parameters request so.

# app/controllers/articles_controller.rb

class ArticlesController < ApplicationController
  before_action :set_article, only: [:show, :update, :destroy]

  # GET /articles
  def index
    if params[:page]
      @articles = Article.page(params[:page][:number]).per(params[:page][:size])
    else
      @articles = Article.all
    end

    render json: @articles
  end
# ...
end

Boot the app (binding to all interfaces will ensure Ember CLI proxy can connect to the API)

$ bin/rails s --binding 0.0.0.0

And query the exact URL used in the JSON API reference example (http://localhost:3000/articles?page[number]=3&page[size]=1) it to verify it responds correctly:

$ curl "http://localhost:3000/articles?page%5Bnumber%5D=3&page%5Bsize%5D=1"
{"data":[{"id":"3","type":"articles","attributes":{"title":"Article 3","body":"Lorem 3 ipsum"}}],"links":{"self":"http://localhost:3000/articles?page%5Bnumber%5D=3\u0026page%5Bsize%5D=1","first":"http://localhost:3000/articles?page%5Bnumber%5D=1\u0026page%5Bsize%5D=1","prev":"http://localhost:3000/articles?page%5Bnumber%5D=2\u0026page%5Bsize%5D=1","next":"http://localhost:3000/articles?page%5Bnumber%5D=4\u0026page%5Bsize%5D=1","last":"http://localhost:3000/articles?page%5Bnumber%5D=13\u0026page%5Bsize%5D=1"}}

Success!

Set up Ember app

Our app will display that list of articles from the backend:

$ ember new articles
$ cd articles
$ ember generate resource articles title:string body:string
$ ember g controller articles

What is a controller?

Sounds like a foreign concept, doesn’t it? Query parameters is really the only (explicit) use we give them. Also, with the upcoming introduction of “routable components” we will kiss controllers goodbye – that’s what we root for here at Ember Igniter.

You do know how components work, right? A Javascript file “backs” an hbs template? Same happens with controllers. Every Handlebars template in your app is backed by one of these.

If you are interested in this subject, read Should we use controllers?

We’ll proceed to define the needed query parameters in our beloved controller. But… precisely which parameters should we use?

Remember the spec mentioned an URL like /articles?page[number]=3&page[size]=1

Problem! Ember query parameters aren’t capable of supporting query objects or hash-like structures. It’s tempting to want to define page: { number: 3, size: 1 } but… nope.

Dang! Is Ember really ready to work with JSON API? Are we doomed?

Oh, don’t be so dramatic. We are dealing with two different sets of query parameters here:

We most definitely can supply hash-like structures to our backend with Ember Data.

For our browser URL concern we’ll settle with page and size which correspond to backend URL page[number] and page[size], respectively.

We’ll only use page in our URLs for now, though.

// app/controllers/articles.js

export default Ember.Controller.extend({
  queryParams: ['page', 'size'],
  page: 1,
  size: 5
});

Moving on to our route…

// app/routes/articles.js

export default Ember.Route.extend({

  model(params) {
    return this.store.query('article', { page: {
        number: params.page,
        size: params.size
      }
    });
  },

  queryParams: {
    page: {
      refreshModel: true
    },
    size: {
      refreshModel: true
    }
  }

});

See what we did there?

We mapped our browser URL query params to our backend URL query params.

Oh and by the way, Ember by default does not call model when query params are changed. We tell it to do so any time page/size changes, through refreshModel: true.

So far so good.

Ember best practices delivered straight to your inbox? Tell me where:

(One e-mail every month. No BS. Unsubscribe anytime!)

We now modify the template to display articles:

{{! app/templates/articles.hbs }}

{{#each model as |article|}}
  <li>{{ article.title }} <blockquote>{{ article.body}}</blockquote></li>
{{/each}}

Is your backend running? Let’s start our frontend app and check:

$ ember server

And this is what we (should!) see:

It works! But 5 articles only?

Our controller reveals exactly why ;)

Implementing client-side pagination

And now? How do we get all those links to, you know, browse through pages and all that?

Let’s go step by step.

We want to have controls for jumping to the previous and next page, our template will have to know which page is previous and which is next.

Our API responded with a links property, remember? That is exactly what we need – but how does the template access those links?

The Bad News is: it can’t.

Shocking – I know. Ember Data models still cannot access the links properties our backend outputs.

The Good News though is: we can adjust a few knobs and make pagination data accessible to our template! Every Ember Data model has a property called meta where arbitrary metadata can be stored.

First approach: Make the backend return links in a JSON API meta field (for example, with Rails we’d use rails-api-pagination). This is a pretty good solution IF changing the server API is an option.

But hey, we are hardcore Embereños. So bring it on baby, we’ll handle it client-side!

We already know how well serializers can massage our data so we will use our prowess to override a serializer.

$ ember g serializer application

As our route model hook calls this.store.query(), then normalizeQueryResponse is the place to deal with responses from that operation. All we want to do is make links accessible in our model’s metadata:

// app/serializers/application.js

export default DS.JSONAPISerializer.extend({

  normalizeQueryResponse(store, clazz, payload) {
    const result = this._super(...arguments);
    result.meta = result.meta || {};

    if (payload.links) {
      result.meta.pagination = this.createPageMeta(payload.links);
    }

    return result;
  }

});

The code says it all: meta.pagination will now hold the links which will be available in the template.

Here is a sample createPageMeta implementation:

// app/serializers/application.js

import DS from 'ember-data';

export default DS.JSONAPISerializer.extend({

  normalizeQueryResponse(store, clazz, payload) {
    // ...
  },

  createPageMeta(data) {

    let meta = {};

    Object.keys(data).forEach(type => {
      const link = data[type];
      meta[type] = {};
      let a = document.createElement('a');
      a.href = link;

      a.search.slice(1).split('&').forEach(pairs => {
        const [param, value] = pairs.split('=');

        if (param == 'page%5Bnumber%5D') {
          meta[type].number = parseInt(value);
        }
        if (param == 'page%5Bsize%5D') {
          meta[type].size = parseInt(value);
        }

      });
      a = null;
    });

    return meta;

  }

});

Adding a simple line to our template will reveal if this is viable:

{{! app/templates/articles.hbs }}

<h1>Last page: {{ model.meta.pagination.last.number }}!</h1>

{{#each model as |article|}}
  <li>{{ article.title }} <blockquote>{{ article.body}}</blockquote></li>
{{/each}}

That is correct!

Let’s put these links to good use and make our app functional. This is a very primitive implementation but should get us a head start:

{{! app/templates/articles.hbs }}

{{#each model as |article|}}
  <li>{{ article.title }} <blockquote>{{ article.body}}</blockquote></li>
{{/each}}

{{#each-in model.meta.pagination as |key value|}}
  {{link-to key "articles" (query-params page=value.number)}}
{{/each-in}}

Which renders:

Notice the cool little purple links down there! And mess around with ‘em.

This is as basic as the previous snippet.

{{! app/templates/articles.hbs }}

{{#each model as |article|}}
  <li>{{ article.title }} <blockquote>{{ article.body}}</blockquote></li>
{{/each}}

{{#each-in model.meta.pagination as |key value|}}
  {{link-to key "articles" (query-params page=value.number)}}
{{/each-in}}

<br>

{{#each count as |number|}}
  {{#if (eq number model.meta.pagination.self.number) }}
    {{ number }}
  {{else}}
    {{link-to number "articles" (query-params page=number)}}
  {{/if}}
{{/each}}

The eq helper will require the mega-hyper-handy Ember Truth Helpers.

And… what would count be? As we saw earlier, templates are backed by controllers. The count property simply is a generated array of page numbers:

// app/controllers/articles.js

export default Ember.Controller.extend({
  queryParams: ['page', 'size'],
  page: 1,
  size: 5,

  count: Ember.computed('model.meta.pagination.last.number', 'model.meta.pagination.self.number', function() {
    const total = this.get('model.meta.pagination.last.number') || this.get('model.meta.pagination.self.number');
    if (!total) return [];
    return new Array(total+1).join('x').split('').map((e,i) => i+1);
  })

});

Result:

That concludes our JSON API pagination chapter! I hope things make more sense now. Two exercises left for the reader:

Did it go well?

If you are looking for something like a “load more”, try ember-infinity or have a look at Building a UI Around an Ember Data App, specifically Showing all books via metadata.

I couldn’t make Ember CLI Pagination work with this set up. Were you more successful than me perhaps?

Let me know all of this in the comments below!

Enjoyed this article? Don't miss my next one!

Leave me your e-mail for content that will help you master Ember:

Do you want to master Ember fast?

Leave me your e-mail for helpful updates delivered straight to your inbox.

(A few e-mails per month. No BS. Unsubscribe anytime!)