Previous, I have code to add up weights of things, and the current total is 8.6, then this:
? "weight is " weight
A = Str(weight)
? "A is " A
but in the console it prints:
weight is 8.6
A is 8.599999
But when I test just setting beforehand:
weight = 8.6
Then both lines print “8.6” just fine
What the heck is going on??? :?
A couple extra notes:
Same results on my Roku 2 XS 3100X and Roku HD N1100
The values that were added up to make 8.6 are:
1, 2, 1, 0.1, 0.2, 0.2, 0.1, 1, 2, 1
So there is no fancy division or anything like that, should be exact and straightforward.
“Komag” wrote:
The values that were added up to make 8.6 are:
1, 2, 1, 0.1, 0.2, 0.2, 0.1, 1, 2, 1
So there is no fancy division or anything like that, should be exact and straightforward.
0.1 is not exactly representable in a floating point number, which internally uses a binary representation.
https://docs.python.org/2/tutorial/floatingpoint.html is an article I stumbled across which has some straightforward explanations.
The reason that Str and ? show different results is they happen to use slightly different values for the display precision.
In one case the extra precision shows a more accurate value, whereas the other rounds the small epsilon to show the expected value.
Hmm, okay. I guess I can work around it 
“RokuKC” wrote:
The reason that Str and ? show different results is they happen to use slightly different values for the display precision.
This obviously leads to confusion… Maybe it’s worth defaulting both methods to the same precision, and adding an optional parameter to str() to define precision?
It’s not just Str and ? that have issues, but Int() as well:
BrightScript Debugger> a=12345.9996
BrightScript Debugger> ?Int(a)
12346
BrightScript Debugger> a=12345.9995
BrightScript Debugger> ?Int(a)
12345
BrightScript Debugger>
You’d think that Int() would always round down, but I guess that 12345.9996 is stored internally as 123456 as a Float, which is only guaranteed to have 7 digits of precision (10 if it were a Double). Whatever precision you use in this case, there will be some point at which a number will be rounded up when it should be rounded down.
I’m guessing fix() will have this issue too, which gives me the opportunity to ask what is the difference between fix() and int() and why isn’t there one that rounds up? I can’t possibly be the first one who needed that.
“belltown” wrote:
You’d think that Int() would always round down
Apparently so did Roku (http://sdkdocs.roku.com/display/sdkdoc/BrightScript+Language+Reference#BrightScriptLanguageReference-99IntxasFloatasInteger 
Returns an integer representation of the argument, using the largest whole number that is not greater than the argument
Whatever nonsense caused int(12345.9996) to be rounded up needs to be resolved.
“squirreltown” wrote:
I’m guessing fix() will have this issue too, which gives me the opportunity to ask what is the difference between fix() and int()
According to the docs on the fix function (which are suspect at this point…):
Returns a truncated representation of the argument. All digits to the right of the decimal point are simply chopped off, so the resultant value is an integer. For non-negative X, FIX(X)=lNT(X). For negative values of X, FIX(X)=INT(X)+1. For example, FIX(2.2) returns 2, and FIX(-2.2) returns -2.
So, the difference should be that int(-2.5) will return -3 since -2 would be greater than -2.5, while fix(-2.5) would return -2 (decimal truncated).
“squirreltown” wrote:
and why isn’t there one that rounds up? I can’t possibly be the first one who needed that.
The cint() function will round up or down depending on the value ( cint( 2.1 ) → 2, cint( 2.8 ) → 3. If you need a function which is guaranteed to always round up (like ceil() in other languages), you’ll need to implement it yourself
A ceiling function was posted here a while ago: viewtopic.php?f=34&t=51230&p=347768
Whatever nonsense caused int(12345.9996) to be rounded up needs to be resolved.
belltown explained this. The problem isn’t in the int function, it’s in the value that’s being passed to it. 12345.9996 looks like it’s different than 12346 but it’s not. They both compile to the same float value, since there are only about 7 digits of precision in a float.
–Mark
Rek-thank you, must have been a squirrel !! when I was reading that part of the docs.
Mark - ignoring the “about” in your statement, does that mean that 1000000.5 is the same as 1000000?
So it’s 7 total digits, regardless of which side of the decimal?
“squirreltown” wrote:
Rek-thank you, must have been a squirrel !! when I was reading that part of the docs.
Mark - ignoring the “about” in your statement, does that mean that 1000000.5 is the same as 1000000?
So it’s 7 total digits, regardless of which side of the decimal?
That’s generally how floating-point numbers work; it doesn’t matter where the decimal point is.
And “about” means “about”. If you want an explanation (and a large squirrel-headache) for the underlying reason why it’s “about” and not “exactly”, then read these:
http://en.wikipedia.org/wiki/Floating_point
http://en.wikipedia.org/wiki/Single-precision_floating-point_format
