[redland-dev] C# binding encoding problem

[Dave Beckett]

On Tue, 21 Sep 2004 09:07:45 +0100, "Victor Lindesay" <victor at schemaweb.info> wrote:

> Hi Redlanders,
> 
> I have been taking my first steps into Redland using Chris Pointon's
> VC++ project and Cesar Lopez Nataren's C# binding. I have got Redland to
> compile and have made a simple C# app for loading RDF and making RDQL
> queries. 

That's great that you got so far with the win32 build.  Thanks again
to Chris for doing the VC++ project work.


> ... However I am having some problems with text encoding.
> 
> Firstly I had to make changes to the C# wrapper regarding passing
> strings between managed and unmanaged code. I have had to change all
> IntPtr pointer = Marshal.StringToHGlobalAuto(aString); lines to IntPtr
> pointer = Marshal.StringToHGlobalAnsi(aString); and string aString =
> Marshal.PtrToStringAuto(pointer); lines to string aString =
> Marshal.PtrToStringAnsi(pointer);.
> Otherwise any calls that pass strings to or return strings from librdf
> do not work.

Hmm.  All Redland strings are UTF-8 except where stated otherwise
(such as librdf_node_get_literal_value_as_latin1).  I wonder where
I should document that and other things?


> For example, without these changes, stepping through the C code I notice
> that if I create a "rdfxml" parser, the line:
> if(!strcmp(syntax_name, "rdfxml")) { ...
> in
> librdf_parser_raptor_constructor() fails.
> 
> However with the string marshalling changes I get various text encoding
> problems. For example when I parse my FOAF file [1], loading from URL
> with librdf_parser_parse_into_model(), and stream out the triples, all
> my buddies with latin-1 characters come out wrong. Leandro Mariano López
> comes out as Leandro Mariano López!

Ouch.  Yeah, that's bad.   I take care that Redland is 8-bit happy
with strings, so that such things should not happen.
(See some of the RSS1.0 feeds on http://librdf.org/rss/ )

> Also, if I pass my FOAF as a string to
> librdf_parser_parse_string_into_model(), it loads all triples up to the
> first with a latin-1 character then stops. This and all subsequent
> triples are not parsed.
> 
> Am I doing something wrong compiling Redland? Or is it something on the
> .Net side?
> 
> I am using .Net 1.1 on Windows 2003 and compiling Redland with VC++ 6.
> 
> [Please be gentle, I am a complete C newbie ;-)]

I think Edd already started to reply on that, as he's the C# expert.
I don't know if Cesar's around.

ObPlug: Buy Edd and Niel's book, it's great!
    Mono: A Developer's Notebook
    http://usefulinc.com/edd/books/mono-notebook

Dave