[redland-dev] Multi-Threading Redland

[Jay Woods]

On Sunday 14 February 2010 02:26:25 pm Nicholas J Humfrey wrote:
> I have spent some time with Google and the redland-dev mailing list
> archives and it looks like thread support in Redland is limited and
> largely unused/untested.
> 
> Options for RedStore:
> 
> 0) Only process a single request at a time
>    - RedStore's aim is to be lightweight and does this now
>    - Pro: simple and uses mimum memory
>    - Pro: no problems with locking
>    - Pro: very fast for a single client
>    - Pro: works with all storage models
>    - Con: slow at responsing to multiple simultanious requests
> 1) Use select() for IO muliplexing.
>    - Pro: very efficent, only uses a single process
>    - Con: not sure this would work well with Redland due to lack of
>  main select() loop.
>    - Con: can make codebase difficult to understand
> 2) Use fork() before responding to each request
>    - Pro: very simple and mostly portable
>    - Pro: doesn't use lots of memory while waiting for a request
>    - Con: have to wait for new process to start even for a lightly
>  loaded system
>    - Con: would not work for memory based storage
> 3) Have a pre-fork() pool of processes
>    - Pro: faster response time than basic fork()
>    - Con: lots of processes sitting in memory
>    - Con: more complex
>    - Con: still would not work for memory based storage

Another possibility is to pre-fork() once when the process is set up. 
Each time one of the pre-forks is used trigger another pre-fork. If the 
time between using the pre-fork is longer than the time to create a pre-
fork, there is always one available and ready to go.

> 4) Use a threading based model
>    - Pro: faster response time than basic fork()
>    - Pro: works with memory based storage
>    - Con: Redland codebase isn't thread optimised?
>    - Con: may end up blocking lots while waiting for other threads
>    - Con: maybe difficult to debug
>    - Con: not well tested
> 5) Something else?
> 
> 
> Does anyeone have any suggestions or opinions?
> 
> Thanks,
> 
> nick.
> 
> 
> 
> _______________________________________________
> redland-dev mailing list
> redland-dev at lists.librdf.org
> http://lists.librdf.org/mailman/listinfo/redland-dev
>