[redland-dev] librdf bdb indexes

[Dave Beckett]

Ben Ranker wrote on 14th Dec 2007:
> It looks like a couple of years ago another user wanted to add an s2po
> index and got some help here[1][2]. I'm in a similar situation: I'd
> really like s2po and o2sp. I'm starting to hack together some code.
> 
> My question is not about adding the new hashes, though. I think I can
> figure that out from the referenced threads. (Thanks,
> Dave-four-years-ago!) 
> 
> My question is about configuration. I want to be able to use a
> configuration option to specify which indexes to create. In trying to
> figure out how to do this, I noticed that librdf_storage_hashes_init()
> already checks for an "indexes" option and stores it in
> context->indexes. As far as I can tell, though, nothing ever actually
> uses context->indexes.
> 
> What is context->indexes in rdf_storage_hashes.c? Was it added as a step
> toward some user-selectable indexing system like the one I'm now working
> on? Was it intended for something else entirely?

It was intended to be user-selectable indexing but I never got the
design of what the value was complete.

> And more broadly, has anybody else already thought through the problem
> of flexible user-selectable bdb indexes and what its configuration
> should look like? I'd love to hear any thoughts you guys have on it.

I'm thinking now of something slightly different.  The tricky bit is
that if you give people the option of choosing different indexes, this
really needs to be persisted so that each time you open a storage, you
get the indexes created the first time.   This means that the best
way to do this would be to store the config in a file once it's
created.  Something like, when BDB named 'foo' is created, it writes
foo-config.ttl as well as foo-sp2o.db etc. for the actual data.
The config file would specify what actual indexes and other options
that were given (such as contexts=yes).

Anyway, if you don't want to do any of that, some kind of value of
the indexes could be used.  Probably something like s:po,s:o,sp:o
seems possible with constraints - alphabet is s,p,o,c, you can only
pick a letter once and only in the order spo{c} (c optional - only
allowed if contexts=yes).

Dave