[redland-dev] CRUD with RDF triplestore and RDBMS

[Jo Walsh]

hello,
On Wed, Dec 28, 2005 at 12:25:59PM +0100, Roman Bischoff wrote:
> Now I wonder how other developers cope with CRUD requests that involve both,
> the RDBMS and the redland RDF triplestore? 

i have an implementation sketch of something resembling this:
two redland stores; the first is the working model, the second store
models changes inflicted on the first one. 
  
When a statement goes in the store, a reification of it goes in the
meta-store, and this is annotated with creation time and the user who
created it (or the URI it came from). This was designed for
wiki-nature applications where one might want to roll back, or at
least examine, transaction history for a set of statements. 

But this is CRD or even CR rather than CRUD; a statement isn't updated, 
but is removed from the store and replaced by a new one which has the same 
subject-predicate pair, with a different object. When this happens, a new
reified statement is created in the meta-store, marked as
*superseding* the previous statement. Implicitly, the creator of the
superseding statement in the meta-store is also the deleter of the
original statement.

I'd add 'CONNECT' to your CR(UD) list; an operation which takes three 
existing nodes and combines them to form a new statement, 
should be a different kind of operation and logged as such.    
 
> Do you do create/update/delete as one request from the (web-) client, then
> splitting it on the server side in two requests -- or would you rather have
> two separate requests from the web client; one for the RDBMS and one for the
> triplestore?

I would send one request and split it. If you are worried about
transaction integrity, why take the risk of separating out the
requests from the client to the webserver? 

In my case the application uses a python object wrapper around 
redland's python API, which can be used with or without a meta-store
attached. The meta-store version just reimplements the interface of the 
plain-old-store. ( http://map.wirelesslondon.info/docs/rdfobj.html ) 

> Was the answer different, if the web client is an "ajaxian" client?

I don't see why it would be, unless the request never needs to touch
the RDBMS, in which case you don't have the original problem ;)

> Are you using transactions (unit of work) for creating, updating, deleting a
> Resource in RDBMS and its metadata in Redland triplestore? 
> How can you do such things as long as Redland doesn't support transactions?

Well, this is why i started making 'reflect' (the with-meta-store
version of rdfobj); to provide at least the logging and rollback
capacities of transactions to a redland-based application.

(Again, i know this could be done within one model and/or using
redland contexts, but having a parallel meta-metadata model made more
aesthetic sense; easier to query the reified statements, and leaves
the original model 'clean' for exploring via the commandline).

In theory one should be able to rollback sets of statements or an
entire model and reconstruct what *should have been* in the original
store at any given point in time. 

But what happens if a statement goes into the store but for whatever
reason the corresponding reified statement doesn't appear in the
metastore, or the transaction information is logged wrongly? 
One could keep *two* meta-stores, reconstruct the original metadata
from them both, and see if the resulting models are the same. If
they're not, which meta-store was wrong? Well, each meta-store could
itself be a reflecting store with two meta-meta-stores attached to it.
This way you would wind up with at least 50 statements for each
original statement in the main store. But that adds up to an awful lot
of data integrity. :) 

Pls bear in mind that my understanding of RDBMS transactions is
derived from the 5-minute wikipedia crash course, i have no
logical justification for any of this, and it is probably nowhere near
an answer to your actual question; but it's fun to think about! 
I don't have a running version of 'reflect' at the moment, but
this has ramped up my enthusiasm for finishing it off; thanks.


-jo