Data Virtualization in Metro-style apps with ISupportPlaceholder

Update 8th April 2012 : Modified to reflect changes upon release of Windows 8 Consumer Preview - References to IVirtualizingVector replaced with ISupportPlaceholder.

In my last post I introduced an implementation of the IObservableVector<T> interface that allows Windows 8 Metro-style applications to automatically update data bound XAML items controls as the underlying data source changes. In this post I am going to delve one stage deeper by demonstrating an implementation of the ISupportPlaceholder interface that enables data virtualization

A Background to Data Virtualization

To start I will provide some background to data virtualization and explain why this is becoming essential for modern applications to improve performance with large data sets. In a traditional, non-data-virtualized application, lists of data must be fully loaded into memory before they can be displayed to the user. When this data is held in a local database or file system this may mean loading many hundred items into memory before being shown in the user interface. With modern cloud-connected applications this becomes more of an issue as they must all be downloaded across what is a relatively slow connection.
In web applications this problem has been solved by the concept of paging – only a small number of items are shown at any one time, and the user can move between the pages by following links. In client based applications however the user expects a more seamless experience – a single list that can be scrolled through as desired.
Data virtualization solves this problem by only downloading and storing in memory the data that is required, deferring the download of further items until they are scrolled into view. The user sees a single list as if they are viewing the total data set, however as they move through the list any newly visible items are obtained on demand.

Data Virtualization for Metro-Style Apps

In Windows 8, Metro style applications support virtualized lists through the ISupportPlaceholder interface,

public interface ISupportPlaceholder

{

    bool IsPlaceholder(object item);

}

At first sight it is not immediately obvious how this can be used to virtualize a list, however it provides the vital link between a virtualization aware data set and a data bound user interface. An ISupportPlaceholder implementation is responsible for providing any data virtualization logic,

• In general the list should behave as you would expect for an IObservableVector<T> implementation
• When the items are requested the list should return the actual item if this is available, or a placeholder item if it has yet to be downloaded
• The IsPlaceholder(…) method may then be called with each item returned in the list, and the ISupportPlaceholder implementation should return true if this is a placeholder item, otherwise false
• It is also the responsibility of the  ISupportPlaceholder implementation to download any requested items in the background and update the list as these are available.

In return the standard Metro-style items controls will show any items for which IsPlaceholder(…) returns true as a grey rectangle,

An ISupportPlaceholder Implementation

To assist with data virtualization in .Net based Metro-style applications a base implementation is provided as part of the freely available Cocoon framework. This is provided as the Cocoon.Data.VirtualizingVector abstract class, into which you simply provide your custom data retrieval logic. You will either be requested for the number of items in the list or a specific item, and once obtained should then call the UpdateCount(…) or UpdateItem(…) methods

public class PersonCollection : VirtualizingVector

{

    protected override void FetchCount()

    {

        // Perform custom code here to retrieve

        //     the number of items in the dataset

        UpdateCount(itemCount);

    }

    protected override void FetchItem(int index)

    {

        // Perform custom code here to retrieve

        //     the specified item from the dataset

         UpdateItem(index, retrievedItem);

    }

}

The VirtualizingVector class then takes care of implementing IObservableVector<T> and ISupportPlaceholder, returning of suitable placeholder objects for items as they are requested, and updating items with real data as it is made available.

Of course this approach would still consume large amounts of memory when displaying a list with a large number of placeholder items. The VirtualizingVector class overcomes this by using the provided VirtualizingList<T> implementation. Without going into detail, this behaves as you would expect for a list with a large number of elements, whilst allocating memory efficiently to store only those items that are not virtualized.

Summary

In conclusion, the Cocoon framework provides an ISupportPlaceholder implementation designed to enable data virtualization for Metro-style applications. The code is freely available for download from the Cocoon CodePlex site (to get the latest version go to the “Source Code” tab, select the first change set and use the “Download” link).
In my introduction to the Cocoon framework I stated that one of my aims was to bridge the divide between the stateless paging model of most web APIs and the “fast and fluid” interfaces of Metro-style applications. Now that I have provided the fundamentals, next time I will try to bridge this gap by introducing “data lists” and “data list sources”.