Data Virtualization in Metro-style apps with ISupportPlaceholder

Update 8th April 2012 : Modified to reflect changes upon release of Windows 8 Consumer Preview - References to IVirtualizingVector replaced with ISupportPlaceholder.

In my last post I introduced an implementation of the IObservableVector<T> interface that allows Windows 8 Metro-style applications to automatically update data bound XAML items controls as the underlying data source changes. In this post I am going to delve one stage deeper by demonstrating an implementation of the ISupportPlaceholder interface that enables data virtualization

A Background to Data Virtualization

To start I will provide some background to data virtualization and explain why this is becoming essential for modern applications to improve performance with large data sets. In a traditional, non-data-virtualized application, lists of data must be fully loaded into memory before they can be displayed to the user. When this data is held in a local database or file system this may mean loading many hundred items into memory before being shown in the user interface. With modern cloud-connected applications this becomes more of an issue as they must all be downloaded across what is a relatively slow connection.
In web applications this problem has been solved by the concept of paging – only a small number of items are shown at any one time, and the user can move between the pages by following links. In client based applications however the user expects a more seamless experience – a single list that can be scrolled through as desired.
Data virtualization solves this problem by only downloading and storing in memory the data that is required, deferring the download of further items until they are scrolled into view. The user sees a single list as if they are viewing the total data set, however as they move through the list any newly visible items are obtained on demand.

Data Virtualization for Metro-Style Apps

In Windows 8, Metro style applications support virtualized lists through the ISupportPlaceholder interface,

public interface ISupportPlaceholder

{

    bool IsPlaceholder(object item);

}

At first sight it is not immediately obvious how this can be used to virtualize a list, however it provides the vital link between a virtualization aware data set and a data bound user interface. An ISupportPlaceholder implementation is responsible for providing any data virtualization logic,

• In general the list should behave as you would expect for an IObservableVector<T> implementation
• When the items are requested the list should return the actual item if this is available, or a placeholder item if it has yet to be downloaded
• The IsPlaceholder(…) method may then be called with each item returned in the list, and the ISupportPlaceholder implementation should return true if this is a placeholder item, otherwise false
• It is also the responsibility of the  ISupportPlaceholder implementation to download any requested items in the background and update the list as these are available.

In return the standard Metro-style items controls will show any items for which IsPlaceholder(…) returns true as a grey rectangle,

An ISupportPlaceholder Implementation

To assist with data virtualization in .Net based Metro-style applications a base implementation is provided as part of the freely available Cocoon framework. This is provided as the Cocoon.Data.VirtualizingVector abstract class, into which you simply provide your custom data retrieval logic. You will either be requested for the number of items in the list or a specific item, and once obtained should then call the UpdateCount(…) or UpdateItem(…) methods

public class PersonCollection : VirtualizingVector

{

    protected override void FetchCount()

    {

        // Perform custom code here to retrieve

        //     the number of items in the dataset

        UpdateCount(itemCount);

    }

    protected override void FetchItem(int index)

    {

        // Perform custom code here to retrieve

        //     the specified item from the dataset

         UpdateItem(index, retrievedItem);

    }

}

The VirtualizingVector class then takes care of implementing IObservableVector<T> and ISupportPlaceholder, returning of suitable placeholder objects for items as they are requested, and updating items with real data as it is made available.

Of course this approach would still consume large amounts of memory when displaying a list with a large number of placeholder items. The VirtualizingVector class overcomes this by using the provided VirtualizingList<T> implementation. Without going into detail, this behaves as you would expect for a list with a large number of elements, whilst allocating memory efficiently to store only those items that are not virtualized.

Summary

In conclusion, the Cocoon framework provides an ISupportPlaceholder implementation designed to enable data virtualization for Metro-style applications. The code is freely available for download from the Cocoon CodePlex site (to get the latest version go to the “Source Code” tab, select the first change set and use the “Download” link).
In my introduction to the Cocoon framework I stated that one of my aims was to bridge the divide between the stateless paging model of most web APIs and the “fast and fluid” interfaces of Metro-style applications. Now that I have provided the fundamentals, next time I will try to bridge this gap by introducing “data lists” and “data list sources”.

ObservableVector as a replacement for ObservableCollection in Metro-style apps

For those of you who have had previous experience in making WPF and Silverlight applications you are probably familiar with the ObservableCollection<T> class. This represents a list of items (implementing IList<T>) whilst also raising the INotifyCollectionChanged.CollectionChanged event whenever items are added, removed or moved within the collection. When such collections are databound to the various list controls in WPF and Silverlight the UI will automatically update to reflect any changes.

ObservableCollection<T> and Metro-Style Apps

If you have followed the same technique when writing Windows 8 Metro-style apps you will have found that this no longer works. There are a couple of reasons for this behaviour,

1. In the Developer Preview version of Windows 8 the INotifyPropertyChanged interface is duplicated in both the System.ComponentModel and Windows.UI.Xaml.Data namespaces. Whilst ObservableCollection<T> implements the former, the new Metro XAML framework uses the latter.
2. Rather than INotifyCollectionChanged as implemented by ObservableCollection<T>, WinRT uses the IObservableVector<T> interface.

Observable Collections in Metro-Style Apps

Whilst this issue is likely to be addressed prior to release of Windows 8, for the time being there are a few workarounds.

1. The Windows 8 SDK Data Binding sample includes an ObservableVector class that takes an existing object that implements INotifyCollectionChanged, and wraps it in an IObservableVector<T> implementation.
2. Colin Eberhardt has posted a similar solution on his blog.
3. Avi Pilosof has taken a slightly different approach by deriving a new class from ObservableCollection<T>, implementing both INotifyCollectionChanged and IObservableVector<T>.

Whilst these solve the problem, they all involve wrapping the existing ObservableCollection<T> class, resulting in events being raised for both the old and new approaches. This is ideal for cases where you wish to write interoperable code for both existing and Metro applications, but is an added overhead when you are natively writing a new Windows 8 application.

A Metro-Designed Observable Collection Class

In response to this I have written a ground-up ObservableVector<T> class for use in Windows 8 Metro-style applications that does not have the overhead of an underlying ObservableCollection<T>. The code has been released as part of the freely available Cocoon framework. You can use this class by creating a new instance directly,

IList<Person> list = new ObservableVector<Person>();

list.Add(new Person("Andrew", "Wilkinson"));

or by deriving a custom class,

public class PersonCollection : ObservableVector<Person>

{

    ...

}

It is important to note however that in the current Windows Developer Preview there is a bug which means that databinding only works correctly to lists when the implement ObservableVector<object> rather than strongly typed collections. This is likely to change in the future.

In both cases the behaviour is designed to match that expected by WPF and Silverlight developers. There are a small number of subtle differences with respect to the previous ObservableCollection<T> class,

1. Since IObservableVector<T> does not have the same concept of moving items it does not implement the Move() method. Instead you should remove the item and then add it to its new place in the collection.


2. The constructor of the ObservableVector<T> class optionally accepts an IList<T> that is wrapped by the implementation rather than copied internally – the reason for this will become apparent in later posts.


3. Currently no attempts are made to stop VectorChanged event handlers from modifying the original collection.

Summary

In conclusion, the Cocoon framework provides an IObservableVector<T> implementation specifically designed from the ground-up for Metro applications. The code is freely available for download from the Cocoon CodePlex site (to get the latest version go to the “Source Code” tab, select the first change set and use the “Download” link).

Next time I will discuss data virtualisation and the IVirtualizingVector interface.