Add support for System.Linq.IQueryable

Jun 27, 2008 at 9:46 AM
Currently if you make any IQueryable AsBindable, then it casts it down to IEnumerable, and loses the ability to filter data 'upstream'.

e.g. using a database table as a query source, and filtering by a textbox value.  After the query is made AsBindable the filtering is done on the client side only.
Coordinator
Jun 27, 2008 at 10:13 AM
Hi Normanr,

I believe there's a good reason for this, but can you please post an example of a query that exhibits this behavior, and the behavior you would expect?
Oct 8, 2008 at 8:25 PM
Edited Oct 8, 2008 at 8:33 PM
Hey Paul,

Using an example from your latest blog post:

var query = contacts.Where(c => c.Name.Contains(textBox1.Text));
you might find that contacts is an IQueryable for a contacts table in a database. Now as a normal Non-AsBinable query, the DLINQ layer converts the Name.Contains to a SQL LIKE operator. This means that the database is responsible for doing the filtering. As soon as you AsBindable the query, it executes the database call.

So you might have a table of 10000 contacts. If the query is run via BindableLinq, then the database returns 10000 contacts once, and BindableLinq does the filtering each time the textbox changes. It would be nice to have the filtering passed to the database so that only a few items are returned from the database. Although once you get this far, you want some magic logic that says 'only search if at least 3 chars have been typed', so this should work too:

var query = contacts.Where(c => c.Name.Contains(textBox1.Text) && textBox1.Text.Length >= 3);
which is 100% untested, but I think works :-)

Cheers
Norman
Coordinator
Oct 8, 2008 at 11:07 PM
Hi Norman,

Thanks for posting. The missing "AsBindable" in that post was a bug on my behalf - it should absolutely call AsBindable before the Where. Without AsBindable, Bindable LINQ will never even look at the query. But if it did call AsBindable, then yes, what you have described is exactly what would happen.

I did experiment with Polling and a few other extensions, but as I think about where Bindable LINQ should be used, vs. where data access or web service calls are typically made, I can't imagine really combining the queries in this way. For example, in the query above, you'd probably want to access the database via a background thread, and do paging - but it references the value of a textbox, which will cause WPF to throw an exception if invoked from a background thread.

Data access is usually fairly general and abstracted, while searching for things is generally pretty contextual. I'd advocate more of a view model pattern here, where your viewmodel is recieving the text and kicks off the search (you manage the threading, and when searching is done, yourself), but you use Bindable LINQ to populate the result set. One way to leverage this might be to perform a short search in memory, while performing a larger search on the database. That way if you search for "peter", and then search for "p", your "peter" result sets appear immediately while the database results are streamed in.

I have prototypes of an early Integration Point pattern that makes it easy to hook Bindable LINQ - which is all about reactive streams of data - and web services/data access - which are all discreet operations. If you send me your email address (paul@paulstovell.com) I'll send you a copy of the prototype that does the kind of searching specified above.

Paul
Oct 9, 2008 at 10:33 AM
Mmm, I guess that if it's going to take longer than ~100ms, it needs to go into a background thread.  I went back and re-read you comments on dropping the Asynchronous operator and it does seem that doing your own db operation in the background via an ObservableCollection is a pretty good way to go.

Does the Integration Point pattern handle the magic that detects the dependancies on textbox1.Text? - i.e. either automagically or via DependsOn calls.  Is there an easy way to say: here's my query/expression, figure out what it's dependanices are (or force them via DependsOn), and call this delegate when those dependancies change.

Would attaching a query.CollectionChanged event work? (does depandancy detection fire the CollectionChanged event?)
Coordinator
Oct 9, 2008 at 4:13 PM
Hi Norman,

Since you'd be doing the operation yourself, you can control routing back to the UI thread, and controlling when the query is re-executed. This would be required because things like "wait 3 seconds for the user to stop typing" is a very UI-oriented thing (out of interest, I'd probably do this via an attached dependency property, and use Explicit bindings on a TextBox, and then have my ADP manage the timers and push the value when the timer ticked). 

Here's an example of a super basic "integration point":
private static ObservableCollection<WeakReference> _allKnownCustomers = new ObservableCollection<WeakReference>();

public static IBindableCollection<Customer> SearchForCustomers(string name)
{
     // Start a background thread to fetch stuff from the DB using Stored Procs, web services, LINQ to SQL, whatever, using the "name" property
     // When it completes, use the Dispatcher to route the fetched items to the _allKnownCustomers collection. It'll raise events, and the query below get them and update...

     // This code is just performing an in-memory query on the items you fetched last time
     return _allKnownCustomers.Select(wr => (Customer)wr.Target).Where(c => c.Name.Contains(name));
}
The key here is that you're returning a bindable collection, which means as items can be added as they are yielded form the background thread, and the UI will get them. If you wanted to extend it to the point of doing live queries as the user types in a text box, you'd be doing similar things but with a more stateful object containing the query, and you'd cancel the current background thread and start a new one each time.

There's a bunch of things you could customize with such an "integration point" framework:
  • How items are stored. Above I used a weak references, so the GC can clean them up when they aren't used. You might instead choose to use the Caching Application Block
  • The "scope" of the integration point and the items it manages (and tracks changes to). Per call, per some context, or maybe global. 
  • How you'll do conflict detection and resolution, and whether you'll only ever have one instance of any item at once (an IdentityMap pattern)
  • How you'll handle timeouts and drop outs in connectivity, via a Circuit Breaker pattern
  • How you'll receive updates. SQL query notifications, polling, or WCF duplex bindings
What's nice about this is that although many of these problems are tricky, it's easy to abstract them and build some common patterns around them. Then, you abstract the whole thing in an "integration point". That integration point can be a service shared throughout your application, so that from the 3 views where you show the status of a job on a server, all 3 of them are always in sync, and yet you only have one active duplex channel or query notifications connection managed by the integration point.

I'm hoping this could replace all external communication from my WPF applications. Conceptually, you treat external calls of any nature as discreet interactions, but the rest of your application is completely reactive.