[redland-dev] MainMem vs DBMS

[Dave Beckett]

On Wed, 2005-08-17 at 12:17 -0400, Christopher Schmidt wrote:
> On Wed, Aug 17, 2005 at 05:23:55PM +0200, Irvine Keans wrote:
> > Hi there,
> >  
> > I've done some tests with redland and got some unexpected results. I hope that you can help me.
> >  
> > I tried to parse 3 ontologies. Once in main memory and once in a mySQL database system.
> >  
> > The unexpected thing is, that the parsing into the DBMS take only a part of the time that parsing into main memory needs. But why?
> 
> Probably because a large chunk of the work is in storing efficient
> memory stores so that they can be accessed, which, if you're using a
> database, is taken care of you by the backing store. The in-memory store
> is, in my experience, designed mostly to be used for small models (<5k
> statements) at which point it works relatively well, but anything beyond
> that and you're going to run into problems.
> 
> I could, of course, be wrong, but that's been my experience. dajobe can
> probably offer more technical advice on the topic.

That's correct.

http://librdf.org/docs/storage.html goes over the tradeoffs and
describes how to get a faster indexed in-memory store:

"The memory store is not suitable for large in-memory models since it
does not do any indexing. For that, use the hash indexed store with
hash-type of memory."

(this doc will be moving into the main redland reference document)

Dave