[redland-dev] Integrating a new storage implementation

[Andrew Reslan]

Lauri

Hi, thanks once again for the info, I scoured the storage sources and missed the role of the get_feature function in enabling context searches. Seems so obvious now ;-)

Cheers

Andy




On 3 Jul 2013, at 16:08, Lauri Aalto <laalto at gmail.com> wrote:

> 
> Hi,
> 
> when contexts are used, you'll need to implement the corresponding context callbacks.
> 
> First, your get_feature() needs to return a non-zero literal e.g. "1" for LIBRDF_MODEL_FEATURE_CONTEXTS URI to enable contexts support i.e. librdf_model_supports_contexts() to return true.
> 
> Then, find_statements_in_context() is essential for good performance queries --- the fallback is to do linear searches on context_serialize() results.
> 
> Lauri
> 
> 
> On Tue, Jul 2, 2013 at 10:10 AM, Andrew Reslan <andrew.reslan at mac.com> wrote:
> Lauri
> 
> Hi, I would like to thank you for your help and report that I have been running with my custom storage in test for a four weeks now and after a few minor bug fixes it has been running without issues.
> 
> I would now like to support named graphs via the SPARQL query interface, I am assuming that these map to contexts at the storage layer, I can't quite figure out how to support these properly, I have implemented the get-contexts callback, but I must need to support contexts in the find_statements somehow?
> 
> Any input appreciated.
> 
> Andy
> 
> 
> On 6 May 2013, at 07:50, Lauri Aalto <laalto at gmail.com> wrote:
> 
>> 
>> Hi Andrew,
>> 
>> as you probably already have figured out, writing a custom librdf storage module is really about writing some function callbacks declared in rdf_storage_module.h.
>> 
>> When I wrote my first custom rdf_storage module, I used rdf_storage_sqlite.c as a reference. Eventually everything was written from scratch. You're probably fine using any of the rdf_storage_*.c as reference.
>> 
>> Registering the new storage is straghtforward with
>> 
>>     REDLAND_API
>>     int librdf_storage_register_factory(librdf_world *world, const char *name, const char *label, void (*factory) (librdf_storage_factory*));
>> 
>> where factory is your function callback that populates librdf_storage_factory with supported functions' pointers. After this, calling librdf_new_storage() with your storage_name calls your init() callback to create a new storage instance.
>> 
>> (It's also possible to load storage modules dynamically if librdf is built without --disable-modular. See code flagged with LIBRDF_MODULAR for more information. The static registration mentioned above is simpler to get started with.)
>> 
>> For unsupported functions, just leave the corresponding function pointer in librdf_storage_factory NULL and librdf itself will, in most cases, either return an appropriate error code or obtain the results with other, possibly less optimized callbacks, e.g. get_arcs_in() using find_statements().
>> 
>> A read-only query-only storage strictly only needs the lifecycle functions:
>> 
>>     init()
>>     terminate()
>>     open()
>>     close()
>> 
>> and the query function
>> 
>>     find_statements().
>> 
>> But to avoid segfaults for unsupported functions where librdf indirects using the function pointer without checking for non-NULL, you also need:
>> 
>>     size() - return negative for unsupported
>>     contains_statement() - can use find_statements() and check for non-empty stream
>>     serialise() - can use find_statements() with all statement parts empty
>> 
>> 
>> Lauri
>> 
>> On Sun, May 5, 2013 at 10:24 PM, Andrew Reslan <andrew.reslan at mac.com> wrote:
>> I want to integrate a new storage implementation.
>> 
>> This will be a readonly store, data will be side loaded (initially), access to the data would be via a query API on the storage layer, data persistence/read/write details will be handled by the underlying storage framework.
>> 
>> Which of the existing storage implementations would provide the simplest template as a starting point for integration?
>> 
>> It would also be really useful to know which operations to implement for a query only store and the best way to implement "unsupported" operations so that clients fail gracefully if they try to write to the store.
>> 
>> It looks like I will need to register the new implementation with the framework, what is the mechanism for doing this?
>> 
>> Any pointers appreciated.
>> _______________________________________________
>> redland-dev mailing list
>> redland-dev at lists.librdf.org
>> http://lists.librdf.org/mailman/listinfo/redland-dev
>> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.librdf.org/pipermail/redland-dev/attachments/20130703/e2918615/attachment.html>