PagedDataListSource – Consuming Data From Common Web APIs

This is my third post in a series discussing the data framework that is included as part of the open-source Cocoon framework. Previously I have introduced the framework, and detailed the SimpleDataListSource base implementation.

In this post I will describe the more advanced PagedDataListSource base implementation included as part of the framework. This is designed to allow simple integration of data that is retrieved from web APIs as a series of pages. This is a common approach to enable access to large data sets in an efficient manner. In general there will be some way to retrieve the number of items in the data set, and a way to retrieve a single ‘page’ of data (often these can be combined in a single API call).

Consider for example a hypothetical web call,

http://www.example.com/api/getEmployees?page=1

This might return the following XML response,

<PersonResult TotalCount="450" Page="1" PageSize="50">

    <Person Name="Bob"/>

    <Person Name="Dave"/>

    <Person Name="Amy"/>

    ...

</PersonResult>

We can immediately see that there are a total of 450 employees, however the API returns only the first 50 entries. To retrieve subsequent employees you make the same call with the relevant ‘page’ query parameter.

From the point of view of a modern Windows 8 Metro style application, the user does not want to see separate pages of data. Instead they expect a continuous scrolling grid of items, with subsequent data retrieved on demand as it is required to be displayed. It is this mismatch between the paging web API and the desired UI that the PagedDataListSource and the Cocoon data framework sets out to address.

Implementing A PagedDataListSource

The PagedDataListSource<T> class in marked as abstract so you must derive a domain specific custom class for the type of data that you wish to retrieve. There are then three methods that you should override,

• FetchCountAsync – This should retrieve the number of items in the data set


• FetchPageSizeAsync – This should return the number of items per page


• FetchPageAsync – This should return the items for a specified page

Note that each of these returns a Task<DataListPageResult<T>> as their response. DataListPageResult<T> is defined as,

public struct DataListPageResult<T>

{

    // *** Constructors ***

    public DataListPageResult(int? totalItemCount, int? itemsPerPage,

                              int? pageNumber, IList<T> page)

        : this()

    {

        this.TotalItemCount = totalItemCount;

        this.ItemsPerPage = itemsPerPage;

        this.PageNumber = pageNumber;

        this.Page = page;

    }

    // *** Properties ***

    public int? TotalItemCount { get; private set; }

    public int? ItemsPerPage { get; private set;}

    public int? PageNumber { get; private set; }

    public IList<T> Page { get; private set; }

}

This contains all the data that could be returned from any of the above methods. Whilst the requested information must always be present (e.g. a call to FetchCountAsync() must always return a ‘TotalItemCount’) any of the other values may be ‘null’ if they are not known.

The reason for this approach is that in many cases several of these items are returned from a single web API call – for example in the ‘getEmployees’ example above an initial call to GetFetchAsync() will not only return the number of items, but also the page size and the contents of the first page. Hence the implementation can often write FetchCountAsync() and FetchPageSizeAsync() as a simple call to FetchPageAsync(…).

So for our hypothetical ‘getEmployees’ web call we could write,

public class EmployeesDataListSource : PagedDataListSource<Person>

{

    // *** Overriden Base Methods ***

    protected override Task<DataListPageResult<Person>>
                           FetchCountAsync()

    {

        return FetchPageAsync(1);

    }

    protected override Task<DataListPageResult<Person>>
                           FetchPageSizeAsync()

    {

        return FetchPageAsync(1);

    }

    protected async override Task<DataListPageResult<Person>>
                                 FetchPageAsync(int pageNumber)

    {

        PersonResult result = await ExampleEmployeesApi.
                                         GetEmployees(pageNumber);

        return new DataListPageResult<FlickrPhoto>
                      (result.TotalCount, result.PageSize,
                       result.Page, result.Items);

    }

}

The Cocoon framework will then ensure that the correct pages are retrieved as required and that each web API call is only made once, with the results cached for future use.

Summary

Here I have discussed the PagedDataListSource<T> implementation of the Cocoon data framework. As usual the code is freely available for download from the Cocoon CodePlex site (to get the latest version go to the “Source Code” tab, select the first change set and use the “Download” link).

Next time I will show how you link these IDataListSource implementations with a DataList that can then be bound to your Windows 8 Metro style app’s UI. Following this I will show an end-to-end sample of how to the data framework to access a real web API and display it to the end user (Note: If you are really eager, the sample application is available from the latest code drop on the Cocoon CodePlex site).