Sanitize special characters

Archived
1

Hi All,

How can I santize special characters using brighscript? I tried something like

print Asc("ä")

and it printed “0” instead of the actual ASCII value, which is “132”

The 2D api is skipping the special character, and leaving a space instead of rendering it. However, imagecanvas renders the special character just fine.

So 2 questions:

  1. How to sanitize special character?
  2. How to make 2D render special character?

Thanks!

2

Make sure your .brs source code file is UTF-8 encoded if you are embedding non-7-bit ASCII characters, e.g. in string constants.

Otherwise, characters should generally be specified as Unicode code points.

132 (&h84) does not appear to be a printable Unicode codepoint. If you want ‘ä’ that would be 228 (&hE4).

If you are getting different results depending on the drawing method it sounds like there are other issues.

3

Make sure your .brs source code file is UTF-8 encoded if you are embedding non-7-bit ASCII characters, e.g. in string constants.

I tried Asc(“ä”) directly on Roku debugger console, and it returns 0 instead of the ASCII value.

4

I don’t think you can do it from the debugger. If you put the statement in a UTF-8 encoded source file as KC suggested it will print “228”.

-JT

5

“sonnykr” wrote:

Make sure your .brs source code file is UTF-8 encoded if you are embedding non-7-bit ASCII characters, e.g. in string constants.

I tried Asc(“ä”) directly on Roku debugger console, and it returns 0 instead of the ASCII value.

The console only supports 7-bit ASCII.

6

oh! I see.. thank you RokuKc and renojim! That solves my first issue.

Still unsure why roScreen does not render special chars

7

Telnet console is a crank indeed, re i18n. worth(example) = lots * words, so:

BrightScript Debugger> s = chr(228): ? s, len(s), asc(s)
   ?    1               228
8

“sonnykr” wrote:
oh! I see.. thank you RokuKc and renojim! That solves my first issue.

Still unsure why roScreen does not render special chars

What font are you using?

9

I am using a custom font from my client. Wondering whether its a font issue! but how come it works fine on imageCanvas? It render the special char very well on canvas. I’m kinda confused.

10

If there are special tricks and or specific fonts to getting this working on an roScreen I’d love to learn more! I’ve tried many different fonts thinking that could be it, but have never been successful getting it to work. In my case I’m getting stuff out of ParseJSON. I’ve eventually just resorted to normalizing these characters on the backend as best as I can.

“RokuKC” wrote:

“sonnykr” wrote:
oh! I see.. thank you RokuKc and renojim! That solves my first issue.

Still unsure why roScreen does not render special chars

What font are you using?

11

You may want to post your code and perhaps we can see the problem. This works for me and displays Unicode character U+E5 (“small a with ring above”). Of course the font you’re using must contain the characters you’re trying to print.


function main() as Void
    reg = CreateObject("roFontRegistry")
    reg.Register("pkg:/myfont.ttf")
    font = reg.GetFont("myfont", 20, false, false) 

    screen = CreateObject("roScreen", true)
    screen.SetMessagePort(CreateObject("roMessagePort"))
    screen.Clear(&h800000FF)
    s = "test:" + Chr(&hE5)
    screen.DrawText(s, 100, 100, &h00FFFFFF, font)
    screen.SwapBuffers()

    while true
    end while
end function

–Mark

12

“gabek” wrote:
…
In my case I’m getting stuff out of ParseJSON. I’ve eventually just resorted to normalizing these characters on the backend as best as I can.

Where are you getting the data string that you pass into ParseJSON from, and do you have an example you can share?

Assuming the data comes from a web service http response, is the content UTF-8 encoded and tagged?

I’m curious what “normalizing these characters on the backend” refers to…
does that refer to using \u escape sequences in the JSON for non-ASCII characters, or replacing them with 7-bit ASCII, or ?

I’m also curious if “tried many fonts” includes the default font, or if possibly there are some issues with specific custom fonts formats.

Thanks.

13

You may want to post your code and perhaps we can see the problem. This works for me and displays Unicode character U+E5 (“small a with ring above”). Of course the font you’re using must contain the characters you’re trying to print.

I tried ur sample snippet (actually, this is exactly how I draw my text too..) with 2 different fonts. One of them rendered the special character without any problem. However, the second one failed to render. Then I used a font editor and saw that there are no font equivalent to those special characters inside the font file itself. I strongly think that its the font issue.

Thanks guys!