[redland-dev] Multi-Threading Redland

[Nicholas J Humfrey]

> Hello,
>
> Yesterday I released RedStore version 0.1:
> http://code.google.com/p/redstore/
>
> I decided to keep the first release simple, and it can only handle a
> single request at a time.
>
> Is Redland multithreadable? Are all the storage model
> multithreadable? Are there any examples of this?
>
> Should the librdf_world, librdf_storage and librdf_models structures
> be shared between threads?
>
>
> Thanks,
>
> nick.

I have spent some time with Google and the redland-dev mailing list
archives and it looks like thread support in Redland is limited and
largely unused/untested.

Options for RedStore:

0) Only process a single request at a time
   - RedStore's aim is to be lightweight and does this now
   - Pro: simple and uses mimum memory
   - Pro: no problems with locking
   - Pro: very fast for a single client
   - Pro: works with all storage models
   - Con: slow at responsing to multiple simultanious requests
1) Use select() for IO muliplexing.
   - Pro: very efficent, only uses a single process
   - Con: not sure this would work well with Redland due to lack of main
select() loop.
   - Con: can make codebase difficult to understand
2) Use fork() before responding to each request
   - Pro: very simple and mostly portable
   - Pro: doesn't use lots of memory while waiting for a request
   - Con: have to wait for new process to start even for a lightly loaded
system
   - Con: would not work for memory based storage
3) Have a pre-fork() pool of processes
   - Pro: faster response time than basic fork()
   - Con: lots of processes sitting in memory
   - Con: more complex
   - Con: still would not work for memory based storage
4) Use a threading based model
   - Pro: faster response time than basic fork()
   - Pro: works with memory based storage
   - Con: Redland codebase isn't thread optimised?
   - Con: may end up blocking lots while waiting for other threads
   - Con: maybe difficult to debug
   - Con: not well tested
5) Something else?


Does anyeone have any suggestions or opinions?

Thanks,

nick.