UTF-8/encoding

[michaelof]

Hi Yuki,

I'm a confused :-)

Tried to narrow down:

This small script using an utf8 string with German umlauts

`use strict;
use warnings;

use Text::vCard::Precisely::V4;
use utf8;
use open ':std', IO => ':encoding(UTF-8)';

my $vc = Text::vCard::Precisely::V4->new();
my $fn = "Först Last";
printf '%vX', $fn;
print "\n";
$vc->fn( $fn );
`

results in this output and error:

46.F6.72.73.74.20.4C.61.73.74 Attribute (content) does not pass the type constraint because: The value you provided, Först Last, was not supported at constructor Text::vCard::Precisely::V4::Node::new (defined at /home/michael/perl5/lib/perl5/Text/vCard/Precisely/V4/Node.pm line 67) line 46 Text::vCard::Precisely::V4::Node::new('Text::vCard::Precisely::V4::Node', 'HASH(0x55f9434e2410)') called at /home/michael/perl5/lib/perl5/Text/vCard/Precisely/V4.pm line 324 Text::vCard::Precisely::V4::__ANON__('FM-vrst Last') called at /home/michael/perl5/lib/perl5/x86_64-linux-thread-multi/Moose/Meta/TypeCoercion.pm line 65 Moose::Meta::TypeCoercion::__ANON__('FM-vrst Last') called at accessor Text::vCard::Precisely::V4::fn (defined at /home/michael/perl5/lib/perl5/Text/vCard/Precisely/V4.pm line 350) line 5 Text::vCard::Precisely::V4::fn('Text::vCard::Precisely::V4=HASH(0x55f9434c03f0)', 'FM-vrst Last') called at vc.pl line 14

Any hints would be great,

• Michael

[worthmine]

thank you for reporting.

• Before setting values to $vc, you have to encode the strings if you set use utf8; at the beginning of your code.

because I made it with no caring for use utf8;

use utf8;
use Encode qw(encode_utf8);
my $fn = "Först Last";
$vc->fn( encode_utf8 $fn );

print encode_utf8 $vc->as_string();

# if you've already declared like this, 
#use open ':std', IO => ':encoding(UTF-8)';
#print $vc->as_string(); # just print it directly.

the above may help you temporary, but..

• Using utf8 may cause mojibake when your OS is parsing .vcf.

the last trying I did was a few years ago, so perhaps It may have changed, I hope so.

[michaelof]

Yuki, THANK YOU again for great help!

Works now, one little side effect to get solved: I "skipped" the encode_utf8 for vcard nodes where I'm 100% sure they will not and never contain German umlauts, e.g. phone numbers.
Having done this, e.g. for BDAY

$vc->fn( encode_utf8 "Först Last'); $vc->bday( '$notEncodedString');

DESTROYS (!) the encoding of the FN entry before!

Bad idea, lession learned :-)

[worthmine]

Thank you for reporting!

I successfully reproduced what you said.

use feature qw(say);
use strict;
use warnings;

use Encode qw(decode_utf8 encode_utf8);
use Text::vCard::Precisely;

my $vc = Text::vCard::Precisely->new( version => '4.0' );

use utf8;
my $notEncodedString = '1953-10-15T23:10:00Z';
$vc->fn( encode_utf8 'Först Last' );
$vc->bday($notEncodedString);

say encode_utf8 $vc->as_string(); # it works

{
    no utf8;
    my $notEncodedString = decode_utf8 '1953-10-15T23:10:00Z'; # there is no necessary to decode, but did
    $vc->fn('Först Last');
    $vc->bday($notEncodedString);

    say encode_utf8 $vc->as_string();
}

this code generates below:

BEGIN:VCARD
VERSION:4.0
FN:Först Last
BDAY:1953-10-15T23:10:00Z
END:VCARD

BEGIN:VCARD
VERSION:4.0
FN:Först Last
BDAY:1953-10-15T23:10:00Z
END:VCARD

I'll try to solve this.

[michaelof]

Yuki, general question:

Which way of working together in github would you as the owner of Text::vCard::Precisely prefer:

• Would you like to close issues yourself, if you think they're are fixed/solved/workforme etc.?
• or should they closed by the creators?

[worthmine]

No, this issue is not to be closed yet. I'll close it when the issue is fixed.

[worthmine]

It seems my test code was broken.
I have fixed it for #23 but not #22. however, #22 can't be reproduced.

the code like below:

use feature qw(say);
use strict;
use warnings;

use Encode qw(decode_utf8 encode_utf8);
use Text::vCard::Precisely;

my $vc = Text::vCard::Precisely->new( version => '3.0' );

use utf8;
my $notEncodedString = '1953-10-15T23:10:00Z';
$vc->fn( encode_utf8 'Först Last' );
$vc->bday($notEncodedString);

say encode_utf8 $vc->as_string();

{
    no utf8;
    my $notEncodedString = decode_utf8 '1953-10-15T23:10:00Z';
    $vc->fn('Först Last');
    $vc->bday($notEncodedString);

# say encode_utf8 $vc->as_string();
    say $vc->as_string();
}

works well.

Is it fixed?

[michaelof]

Tried again, after upgraded to 0.27 from cpan (0.28 not released I guess).
--> Works FINE for me, too, now. Error not thrown anymore.
Maybe fixed by you as a "side effect" on the way to 0.27?

[worthmine]

hmm, it's so funny but now I can close it.

thank you.

[michaelof]

Commenting here, as UTF-8 related issue.

Tried with this small script, slightly changed:

#!/usr/bin/perl
use strict;
use warnings;
use utf8;

use Encode qw(encode_utf8);


use Text::vCard::Precisely;    
use Text::vCard::Precisely::Multiple;
my $vc = Text::vCard::Precisely->new( version => '4.0' );
my $vcm = Text::vCard::Precisely::Multiple->new( version => '4.0' );

my $fn = "Först Last";
printf '%vX', $fn;
print "\n";
$vc->fn( encode_utf8 $fn);
$vcm->add_option($vc);


$vc->as_file( "22_single_as_file.vcf");

my $str = $vc->as_string();
open my $out, '>', '22_single_as_string.vcf' or die $!;
print $out  $str;
close $out;

$vcm->as_file('22_multiple_as_file.vcf');

$str = $vcm->as_string();
open $out, '>', '22_multiple_as_string.vcf' or die $!;
print $out  $str;
close $out;

Current test result:

• Now as_string works, both for single and multiple.
• But as_file does not work anymore, mojibake

Tested with use utf8; and without: No difference, as_string fine, as_file mojibake
Test with

$vc->fn( $fn );

and

$vc->fn( encode_utf8 $fn);

also no difference: : No difference, as_string fine, as_file mojibake

Tested with my "real" script, to be 1000% sure :), does also create mojibake with new test version, currently using multiple->as_file

Summary:
Really good news is, that currently as_string without the need for any encode, would make it simpler.
I'm curiuos why as_file now does not work anymore :)

[worthmine]

I will see my source tomorrow