I am trying to capture timed metadata from an HLS live stream. Using IsTimedMetaData() on the roVideoPlayerEvent, the info available is formatted like this, with either a PRIV of GEOB key:
In Javascript, we use String.fromCharCode.apply(null, new Uint8Array(buf)) to translate the arraybuffer to a string. The string is a set of name/value pairs formatted like a URL querystring.
Is there a technique in Brightscript to achieve the same result?
GEOB is still giving me headaches. It ends up all our important data is provided as a GEOB frame. However, using the same code, I never get anything more than “?application/json” as the value:
? "+++++ TIMED METADATA +++++"
data = msg.GetInfo()
?data
if data.GEOB <> invalid
ba = CreateObject("roByteArray")
ba.FromHExString(data.GEOB)
strg = ba.ToAsciiString()
? "GEOB"
?strg
end if
I think that’s because your GEOB thingy contains null characters. As far as BrightScript is concerned, as soon as it finds a null character in a string, it treats it as the end of the string.
Before you convert to a string you’ll have to go through the byte array extracting each null-terminated piece of string data, then convert each of those to a string.
As Belltown says, the hex encoded string represents the binary data of the GEOB frame.
You would have to decode that into components yourself, e.g. using roByteArray.
http://id3.org/id3v2.4.0-frames 4.15. General encapsulated object =>
Text encoding $xx
MIME type $00
Filename $00 (00)
Content description $00 (00)
Encapsulated object
Thanks! You guys rock!
I am now officially out of my depth. Any pointers on how to decompose the byte array into its components?
…off to StackOverflow…
“RokuKC” wrote:
As Belltown says, the hex encoded string represents the binary data of the GEOB frame.
You would have to decode that into components yourself, e.g. using roByteArray.
Any chance the un-support of \0 in strings can be fixed? As far as i can tell the \0 arcane quirk is not even documented.
I can imagine the (d)effect stemming from ye olde C but string libraries don’t suffer from that since… forever. Even easier if rolling-your-own strings - which seems is what the Co. has done, because UTF-8. Shirley, i am serious - i remember the end of last century writing a script engine for a most unusual hardware^ where we had just barebones C to boot - so as part of it i had to write a simple string library supporting \0. It was no BFD.
As a side effect, that will fix the embarrassment of len(str) being O(n)… <wink-wink, nudge-nudge>
(^) Tandem NonStop - a fault-tolerant system used by banks/stock exchanges with no single point of failure (in some configurations having 3 CPUs and 3 banks of RAM working in tandem on the same thing, with the bus hardware arbitrating majority vote every cycle and isolating defective ones) - all components were hot-swappable, no restart; yes - RAM and CPU were replaceable with no down time :twisted:
Any chance the un-support of \0 in strings can be fixed? As far as i can tell the \0 arcane quirk is not even documented.
I think it is unlikely that Roku would add support for embedded chr(0) characters in BrightScript strings.
In my opinion there isn’t a valid use case for it. If you need to work with binary data, roByteArray is your best bet.
“RokuKC” wrote:
I think it is unlikely that Roku would add support for embedded chr(0) characters in BrightScript strings.
Hm, let me correct myself - i implied the only way to allow for U+0000 is by re-implementing String type with length_counter field. I was wrong - turns out there is a wickedly clever way to represent \0 in UTF-8 as 0xC0 0x80 as to avoid ever using the dreaded 0x00 octet. See Modified UTF-8.
However, if i try to do that through roByteArray, bizarre things happen:
Brightscript Debugger> ba = createObject("roByteArray"): ba.fromHexString("c080")
Brightscript Debugger> s = ba.toAsciiString(): ? len(s), s, asc(s)
1 ?? 63
What’s going on here? Seems like a bug - len() is correct but the rest is whacked?
“RokuKC” wrote:
I think it is unlikely that Roku would add support for embedded chr(0) characters in BrightScript strings.
Hm, let me correct myself - i implied the only way to allow for U+0000 is by re-implementing String type with length_counter field. I was wrong - turns out there is a wickedly clever way to represent \0 in UTF-8 as 0xC0 0x80 as to avoid ever using the dreaded 0x00 octet. See Modified UTF-8.
However, if i try to do that through roByteArray, bizarre things happen:
Brightscript Debugger> ba = createObject("roByteArray"): ba.fromHexString("c080")
Brightscript Debugger> s = ba.toAsciiString(): ? len(s), s, asc(s)
1 ?? 63
What’s going on here? Seems like a bug - len() is correct but the rest is whacked?
C080, or any 2-byte sequence starting with c0 or c1, is just not a valid UTF8 sequence. The official standard doesn’t allow for “overlong” encodings (representing a character using a 2-byte encoding when that character can be encoded in a single byte). BrightScript is just implementing the official standard.
Regarding your earlier point: “As far as i can tell the \0 arcane quirk is not even documented.” – it’s in the roByteArray description in the Component Reference.
“belltown” wrote:
C080, or any 2-byte sequence starting with c0 or c1, is just not a valid UTF8 sequence. The official standard doesn’t allow for “overlong” encodings (representing a character using a 2-byte encoding when that character can be encoded in a single byte). BrightScript is just implementing the official standard.Oh, please! Don’t tell me BrightScript is “holier-than-thou Java, Android and TCL” in implementing UTF-8 standard - that would be ridiculous stance to take. :roll:
It’s not out of purity that B/S does not use overlong NUL for internal repesentation.
Regarding your earlier point: “As far as i can tell the \0 arcane quirk is not even documented.” – it’s in the roByteArray description in the Component Reference.“Beware of the leopard!” - you are looking in the wrong place. NUL is legitimate character in ASCII, Unicode and BASIC. Imagine you never knew C in your life. What’s the expected outcome from the following?
for i = 0 to 10: ? len(chr(i)), : next
PS. RokuKC is right that NUL support in B/S strings is largely an academic concern… but shouldn’t me and you as academic luminaries be able to get our opinions in a row?
Specifically to how this thread started - with binary data - UTF-8 strings indeed are not a way to handle that. Cue \0xFE and \0xFF octets which could never ever be part of valid UTF-8
Belltown is correct, in that at least some of the BrightScript processing enforces valid UTF-8, and would reject the C080 sequence.
You can expect enforcement to get more strict over time.
Regardless of what someone might expect, BrightScript doesn’t define a character type, only a string type.
Having Chr(0) returning empty string is ‘as designed’.
Having Chr(0) returning empty string is ‘as designed’.
BASIC defines chr() as always returning a 1-character string, unless an out-of-range exception.
The deviation from this in a single case to return an empty string adds a 3rd outcome, a singularity, a quirk. An undocumented one at that.
Can we please not label the quirks “as designed” retro-actively? Feels sarcastic
“As implemented” or “won’t fix” (“as you were”) would be more genuine
“RokuKC” wrote:
BrightScript is not BASIC.
BrightScript is a dialect of BASIC, extending it as a scripting language (dynamic types), i.e. in general direction of VBScript and VBA.
I would like to think “because RokuAnthony wrote it” is not the sole reason we are using B/S on Roku - but rather^ because it’s easy to pick up by people coming from VB background. TRS-80 “Level II BASIC” - for which AJW wrote a simulator back in the days - is a BASIC, right? Besides the genealogical connection, it’s easy to trace the BASIC roots in the B/S core functionality, incl. idiosyncrasies like the dummy argument in the pos() and upTime() functions.
To that extent, what i said was it would be desirable - >>> if practical <<< - to maintain the well-known functionality, which in the case of CHR(x) is to return a 1-char string for x in the ASCII range [0, 127].
The BrightScript Chr() behavior of returning empty string for 0 and other non-valid codepoints is intentional and was done with forethought.
Wait, we might have a misunderstanding here. In what sense is \0 a “non-valid codepoint”? It is valid ASCII, valid Unicode, valid UTF-8, valid in JSON (see the easy-to-read spec)… i think even in HTML (which points to Unicode definition of control characters, which points to “C0 control codes”). The only one that takes exception to that is XML, can’t have \0 there - which is fine, as long as you do XML only.
I’m not sure why you are attributing sarcasm or non-genuineness to my attempt to provide information.
My apologies if i offended you, i was just objecting to what i perceived to be an euphemism. To clarify the terms, if some behavior stems from underlying implementation (i.e. comes “bottom up”), that is “as implemented” and not “as designed”. Was i wrong in assuming chr(0) = “” comes from shoring up the fact that B/S strings are currently implemented internally as ASCIIZ?
(^) reverse Hanlon’s (“assume benevolence or forethought…”) or me being a Pollyanna?
“RokuKC” wrote:
BrightScript does not support embedded NUL characters in strings.
Ok, so if the choice is clear, shouldn’t that un-support be documented?
For String and roString, i imagine - otherwise there are way many places where marshalling to/from another representation happens to re-iterate it - besides roByteArray and chr() i can outright think of ReadAsciiFile(), parseJSON(), roUrlTransfer.getToString(), roUrlEvent …
