Bug 279

Summary: locale not supported in Perl
Product: Infrastructure Reporter: Saara Huhmarniemi <saara.huhmarniemi>
Component: victorioAssignee: Saara Huhmarniemi <saara.huhmarniemi>
Severity: normal CC: borre.gaup, sjur.n.moshagen, trond.trosterud
Priority: P2 - As soon as possible    
Version: unspecified   
Hardware: All   
OS: Linux   
Bug Depends on:    
Bug Blocks: 249, 283    

Description Saara Huhmarniemi 2006-04-22 12:33:04 CEST
I have had problems with Perl and utf8 for a long time now. The utf8 characters do not work as expected but need special attention in the scripts. E.g. when I define a varibale which contains any string and use a simple test to check whether it is utf8 or not:

my $w_class="a";
if (Encode::is_utf8($w_class)) { print "Ok\n"; }
else { print "not utf8\n"; }

It never is. It's difficult to use any functions related to character conversions for this reason. I generally find a way to circumvent the problem but it takes time. Now I noticed that "use locale" command does not work at all. The configuration variable seems not to be set:

[saara@cochise ~]$ perl -V:d_setlocal

The value should be 'define'. I'll write an email to Roy D. and ask him to fix it.
Comment 1 Saara Huhmarniemi 2006-04-24 10:42:19 CEST
Ok, the actual value was 'define', I just had a typo in my command (Thanks to Roy for pointing that out). Anyway, the problem still persist and I would like to know if someone else has better settings than me. A simple test for locale, write this to a file test.pl:

use POSIX 'locale_h';
setlocale(LC_CTYPE, "fi_FI.utf8") or warn "uh oh... $!";
use locale;
print setlocale(LC_CTYPE);
my $test = "test" . "ä";
$test =~ s/(\w+)/[$1]/;
print "\n$test\n";

Then run command in cochise: perl test.pl

If you have the locale support, the output is:
[testä] and if not,  [test]ä

Comment 2 Sjur Nørstebø Moshagen 2006-05-24 16:01:17 CEST
Here's my output:

~$perl test.pl 

It seems I don't have locale support. I haven't been using cochise/victorio in a long time, I almost exclusively work on my local machine, where things are working. But I think my cochise setup was working as it should - I will have a look soonish.
Comment 3 Saara Huhmarniemi 2006-10-23 10:58:16 CEST
I have given up with the locale support. That is not needed anyway, since  the scripts have to work for different languages and actually they have to be _independent_ on the locale. The other Perl-problems can be fixed by setting the environment variable PERL_UNICODE to empty string:
export PERL_UNICODE=""

This is not yet a default setting for everyone, but will be. To get the utf-8 code working, the use utf8 has to be used in the scripts. I'll do the needed changes.