[redland-dev] Storage Option question (Indexed but not optimized?)

[Dave Beckett]

Lou Sakey wrote:
> I have been reading about the different storage options available with 
> the Redland Libraries and came across several terms that I find confusing.
> 
> In the  mySQL and SQLite storage summaries:
> Indexed but not optimized.

That means I haven't spent any time doing detailed tests on the performance
characteristics of those stores and then consequent optimising.

Although that was kind of true for mysql when I wrote that some years ago,
we've spent some time working on this at Yahoo! (my employer) but at this
stage I am not announcing anything.

> In the  hashes storage summary:
> Indexed.

this is also not optimised in the same way but I understand the tradeoffs.

> I do understand the concept of a database index.  However I am not sure 
> what the “not optimized” is referring to.
> 
> Does this mean that using a Hashed Berkeley DB is both indexed and 
> optimized, and that “mySQL” is only indexed?  What are the application 
> ramifications to using a non-optimized storage option such as mySQL 
> storage?  Is the “optimized” referring to database read or write 
> performance,  memory utilization, or disk size requirements?

I recently posted to redland-dev an alternative sqlite storage layout
that goes for more tables rather than one wide table.  Again, I haven't
done detailed performance tests but if it worked better it might be worth
making available or replacing the existing schema.  They could be
alternatives since the storage configuration has parameters.

Some people have spent a lot of time optimising their storage
systems for triple stores such as Garlick or OpenLink.  The latter may
provide a complete redland storage backend at some point although the
patch I've seen so far was not ready to commit.

I suppose I should say that the existing redland 1.0.7 mysql storage
has been extensively tested at Yahoo! - it's very solid and we have
been using it in production continually for well over a year, even with
it's not-optimal state.

Dave