[redland-dev] Triple storage overhead

[Morten Frederiksen]

Hi,

On Wednesday 02 June 2004 10:22, Sébastien Pierre wrote:
> Do you have an average triple storage overhead, similar to the test I
> made here <http://wiki.type-z.org/index.php/Notes/RedlandStorageImpact> ?
> If not, I could try to run these benchmarks with your backend.
That would be very interesting.

I haven't done any explicit measures, but I do know that it depends very much 
on the graph structure, since an extra statement takes up very little space 
if all the nodes are already known.

> >>However, I would like to know if it is possible to
> >>migrate a storage from the BDB backend to your MySQL backend ? This
> >>would be interesting as I don't want to force people to install MwSQL to
> >>use my software.
> Perl would be ok as long as it does not rely on specific modules : I
> don't want to add another dependency. I would prefer a C program, or
> even better, a Python script (so I could incorporate it seemlessly in my
> app).
Perhaps I should do it in Python then, for the challenge...

> Anyway, if the algorithm is not complicated, I could simply port your
> Perl script to Python.
That may be an option as well, and the algorithm I have in mind for take one 
is indeed quite simple - simply use streams from the original to the copy, 
but that might not be as efficient as going directly to the storage 
(especially for the inserts, even though I have some bulk optimization 
options - I can now load 6.7M triples in just over 20 minutes).

> So, could you indicated me how to install/configure your MySQL storage ?
The current Redland 0.9.16 comes with the previous version (for which I have 
an upgrade bash script ready), but you need MySQL installed before trying to 
build Redland. From there, it should be automatic.

The "next" version of the MySQL storage engine should end up in CVS in about 
two weeks time.


Regards,
Morten