Tuesday, January 31, 2012

SimpleDataListSource – Consuming Data From Simple Services

In my last post I introduced the data framework exposed as part of the Cocoon framework project. Whilst the same principles could be applied to any presentation layer, this framework is designed from the ground up to work great with Windows 8 Metro applications written in managed .Net languages. This includes support for several of the advanced features of WinRT based UIs, such as data virtualization (which I discussed in this post).

As I discussed, Cocoon splits the passing of data from a data source into the UI with two parts. The DataListSource (represented by the IDataListSource<T> interface) describes how the data is retrieved and maps well onto the typical web service API calls to do so. At the other end of the pipeline is the DataList whose responsibility it is to determine when to retrieve the data and to present it in a format that is easily bound to the UI.

Whilst this system is designed to be easily extensible, there are a number of typical use-cases that are provided out of the box with the Cocoon framework. In this post I will describe the SimpleDataListSource.

The SimpleDataListSource Class

The SimpleDataListSource<T>, as its name suggests, is an implementation of the IDataListSource<T> interface for use with simple web APIs that return all items in a list with a single API call.

As an example consider the Flickr API method flickr.photos.people.getList that returns all the people that are tagged in a given photo. Since the number of people is likely to be relatively small the API simply takes a photo ID and returns all the people in one go, and is an ideal candidate for the SimpleDataListSource<T>. This is in contrast to a method such as flickr.interestingness.getList that returns all the interesting photos for the current day. In this case they may be many hundred items so the Flickr API splits these into several pages that are returned one at a time.

In code form we could consider this as,

public class FlickrApi
    public Task<IList<PeopleTag>> GetPeopleInPhoto(string photoId)
    public Task<IList<Photo>> GetInterestingPhotos(int page)

Implementing A Simple Data Source

Since SimpleDataSource<T> is an abstract class you must first derive a custom class for a specific data type. The only method you then are required to implement is the FetchItemsAsync() method that will initiate the call to the web API to retrieve all the items.

For our Flickr photo tagging example,

public class TaggedPeopleDataListSource : SimpleDataSource<PeopleTag>
    // *** Fields ***
    private FlickrApi flickrApi;
    private string photoId;
    // *** Constructors ***
    public TaggedPeopleDataListSource(string photoId, FlickrApi flickrApi)
        this.flickrApi = flickrApi;
        this.photoId = photoId;
    // *** Overriden Methods ***
    protected override Task<IList<PeopleTag>> FetchItemsAsync()
        return flickrApi.GetPeopleInPhoto(string photoId);

Here we firstly derive from ‘SimpleDataListSource<PeopleTag>’ (since every item in the list is of type ‘PeopleTag’). In the constructor we take the photo ID to retrieve the tagged people, and a reference to the ‘FlickrApi’ implementation. In the FetchItemsAsync() method we simply call the API method and return the result.

The underlying SimpleDataListSource<T> implementation will ensure that the web API is only called once, and will cache the results in memory for future use.

As usual the code is freely available for download from the Cocoon CodePlex site (to get the latest version go to the “Source Code” tab, select the first change set and use the “Download” link).

Next Time

Whilst simple scenarios such as those supported by the SimpleDataListSource are relatively basic to implement without the support provided by the Cocoon framework, many web APIs return paged results that are much more difficult to consume. For a fast and responsive UI these should be retrieved asynchronously and on demand as the user scrolls through a long list of items.

Next time I will introduce the PagedDataListSource that allows you to support this with minimal code. After that I will introduce the final piece in the puzzle – the DataList that can be connected to any of these DataListSources for binding to the UI. Finally I will describe a simple end-to-end example of how to use the data framework to bind to a real web service.

Tuesday, January 17, 2012

Bridging the Data Divide–An Introduction to Cocoon Data Lists

When I first introduced the Cocoon framework one of the targets was to improve the ease at which data retrieval APIs (primarily web APIs) could be integrated into modern desktop applications. In this post I will introduce two concepts, the “data list” and the “data list source”, that are used in the Cocoon framework to simplify this process when writing Windows 8 Metro-style applications, in particular those implementing the MVVM design pattern.

The Data Divide

Let us first consider a typical web API that will return a list of information in response to a web request from a desktop application. In many cases there may be a large number of items in the list, so the API will split the results into several “pages”. For example, the hypothetical web call,

public Task<PersonResult> GetEmployeesAsync(int pageNumber) {...}

Might return the resulting data for pageNumber=1,

<PersonResult TotalCount="450" Page="1" PageSize="50">
    <Person Name="Bob"/>
    <Person Name="Dave"/>
    <Person Name="Amy"/>

Here, whilst there are a total of 450 items in the data set, only the first 50 are returned. If you then require any further items then additional calls must be made for the second, third, etc. pages as required.

In contrast let us consider a modern Windows 8 Metro-style UI. Here the user generally expects to see a single list of all items that can be scrolled through as desired. For performance reasons this list should take advantage of data virtualisation to retrieve data on demand, filling in the UI smoothly as the results are retrieved (see my last post on VirtualizingVector for more information).

It is this “data divide” between the state-less paged result sets of web APIs and the fluid continuous lists of desktop applications that often require much code to implement correctly. This is the problem that “data lists” and “data list sources” in the Cocoon framework are desired to solve.

“Data Lists” and “Data List Sources”

The approach taken by the Cocoon framework is to separate the two concerns of the web API calls (or any other data access call) and the resulting lists to display in the UI into separate components. These can then be developed separately using their own conventions, with the Cocoon framework bridging the divide.

In this model a web API call (or calls) that return information on a list of information is represented as part of a “data list source”. This understands how to query the source API to retrieve the list of data as required. In addition it is your application’s own representation of what it knows about the list of data – the length of the data set, and any items that have already been downloaded. This can be reused amongst different parts of your application and in some ways acts as a local cache of the data. Multiple data list sources will be available, for simple cases where all data is returned at once and for data that is returned by pages, as well as the option for custom implementations.

At the other end of the chain is the “data list”. This is the IList<T> implementation that is bound to the UI, either via the code behind or through a view model. Multiple data lists can be attached to the same data list source so that the application’s view of the data is consistent. Different data list implementations can determine how the data is retrieved for display; for example,

  • StaticDataList – This will retrieve all the information before display to the user
  • VirtualizingDataList – This will use data virtualization to only retrieve the data items that are currently visible to the user
  • IncrementalLoadingDataList – This will initially show a fixed number of items, but allow the user to expand the list if they wish to see more items
  • DynamicDataList – This will start with an empty list, with items being added as they are retrieved from the data list source

Between the data list and the data list source are a number of optional processing steps. For example these could constrain the list to only the first 50 items for preview, they could filter the items based on a search term, or could group the items to display as part of a grouped grid view.

The IDataListSource Interface

All of the above is orchestrated by the Cocoon.Data.IDataListSource<T> interface.

public interface IDataListSource<T>
    Task<int> GetCountAsync();
    Task<T> GetItemAsync(int index);

You can think of this as an asynchronous version of IList<T>, albeit a minimal version. There are two methods:

  • GetCountAsync() – Returns a task that results in the number of items in the list.

  • GetItemAsync(int index) – Returns a task that results in the item at the specified index.

Note that the GetItemAsync(…) method should throw an exception if ‘index’ is less than zero, however if ‘index’ is greater than the last item it should return default(T) (for reference types this will be null).


To summarize, the Cocoon framework supplies an infrastructure designed to bridge the gap between state-less web APIs and modern fluid Windows 8 Metro style applications. Over the next few blog posts (and associated code drops) I will provide a number of generic implementations of data lists and data list sources to make consuming lists of data from the web quick and easy.