I ported a j2me midp 1.0 png encoder(http://www.chrfr.de/software/midp_png.html - by Christian Fröschlin) to run on the roku. It takes an RGBA ByteArray and returns a PNG ByteArray. It uses the LibRokuDev library.
It still needs some profiling and performance improvements as it is not very speedy. but I tested it with loading a jpg, using GetByteArray from the loaded bitmap, passed it to toPNG(width,height,pixels) and was able to display the png as a bitmap. Current code loads a static ByteArray of the Roku forums logo and encodes it and displays it on the roku screen.
I started looking into writing a roByteArray as a PNG and was at the stage of needing a deflate encoder (not needed for compression, just wrapping a raw encoding).
Hopefully the speed will increase soon. Let us know if that happens.
Thanks for the time and effort to put this together!
Quick profiling, I know the headers aren’t going to mean much without my specific code, but below is the breakdown of milliseconds of each section. Some are subsets of others. Letters A,B, etc are in toPNG, tC is toChunk, CDC is CreateDataChunk.
Significant time is in updateCRC (tc5), and a significant portion of that is rdRightShift and rdXOR, mostly rdRightShift.
Is there any way to improve the following function for a logic right shift? Looks like the main use in updateCRC is a shift by 8, so an optimization using count = 8 might suffice.
' ********************************************
' * Right logical (non-sign-extending) shift *
' ********************************************
function rdRightShift(num as integer, count = 1 as integer) as integer
mult = 2 ^ count
summand = 1
total = 0
for i = count to 31
if (num and summand * mult)
total = total + summand
end if
summand = summand * 2
end for
return total
end function
Pulling the XOR inline saves about 3 seconds out of the 20 spent on updateCRC
for n = 0 to lastn
index = ((c and not buf[n]) or (not c and buf[n])) and &hFF
'index = rdXOR(c, buf[n]) and &hFF
shiftedc = rdRightShift(c, :smiling_face_with_sunglasses:
c = ((shiftedc and not t[index]) or (not shiftedc and t[index]))
'c = rdXOR(t[index], shiftedc)
end for
Start Convert
Time tC1: 1
Time tC2: 0
Time tC3: 0
Time tC4: 0
Time xor1: 0
Time shift 1
Time xor2: 0
Time xor1: 0
Time shift 0
Time xor2: 0
Time tC5: 5
Time tC6: 1
Time A: 8
Time CDC1: 1594
raw.count = 261924
Time tC1: 1
Time tC2: 0
Time tC3: 0
Time tC4: 4
Time xor1: 0
Time shift 0
Time xor2: 0
Time xor1: 3299
Time shift 14592
Time xor2: 2869
Time tC5: 20766
Time tC6: 3
Time B: 27484
Time tC1: 0
Time tC2: 0
Time tC3: 0
Time tC4: 0
Time xor1: 1
Time shift 0
Time xor2: 1
Time xor1: 0
Time shift 0
Time xor2: 0
Time tC5: 4
Time tC6: 0
Time C: 6
Time 7
PNG Count = 262032
That’s all I have time for. I will review again when I actually need this function.
updateCRC: function(crc as integer, buf as object) as integer
c% = crc
for n = 0 to buf.count() - 1
index% = ((c% and not buf[n]) or (not c% and buf[n])) and &hFF
shiftedc% = (c% and &hFFFFFF00)/256
shiftedc% = shiftedc% and &hFFFFFF
c% = ((shiftedc% and not m.CRCTABLE[index%]) or (not shiftedc% and m.CRCTABLE[index%]))
end for
return c%
end function
By the way, the PNG encoding is about 4x faster on a Roku 2 over a Roku 1.
It would be nice if there was a built-in function to save a region to a PNG or JPEG.
I was wondering how much time could be saved by removing the boxing/unboxing (using the %), so thanks for that! I modified slightly to remove the AA lookup and the repeated calls to Count (though, not sure if Brightscript evaluates the “TO” on a for loop repeatedly or not, some languages do some don’t).
GPF, the frequency of negative numbers seems about 30-50%, and I would suspect that testing “< 0” rather than the function call of Sgn() would be much better performance.
The updateCRC function is now about 6x faster. This is code:
updateCRC: function(crc as integer, buf as object) as integer
c% = crc
lastn = buf.count() - 1
t = m.CRCTABLE
for n = 0 to lastn
index% = ((c% and not buf[n]) or (not c% and buf[n])) and &hFF
shiftedc% = (c% and &hFFFFFF00)/256
shiftedc% = shiftedc% and &hFFFFFF
c% = ((shiftedc% and not t[index%]) or (not shiftedc% and t[index%]))
end for
return c%
end function
Setting Alpha to 252
Done Setting Alpha
Start Convert
Time tC1: 0
Time tC2: 0
Time tC3: 1
Time tC4: 0
Time tC5: 2
Time tC6: 1
Time A: 5
Time CDC1: 1604
raw.count = 261924
Time tC1: 0
Time tC2: 0
Time tC3: 0
Time tC4: 4
Time tC5: 3295
Time tC6: 3
Time B: 10186
Time tC1: 0
Time tC2: 0
Time tC3: 0
Time tC4: 0
Time tC5: 2
Time tC6: 0
Time C: 4
Time 8
PNG Count = 262032
CDC1 sped up by a few tenths of a second (about 30%) in pngRoku.brs
raw = CreateObject(“roByteArray”) ’ highest index of raw will be dest%+4*width
'raw.setresize(4*(width*(eheight-sheight)) + (eheight-sheight), false)
'print "4sheightwidth = "+(4sheightwidth).toStr()
end% = eheight -1
for y = sheight to end%
raw[dest%] = 0 ’ No filter
dest%=dest%+1
mult% = 4sheightwidth
wend% = dest%+4*width-1
source% = source% + mult%
for dest% = dest% to wend%
raw[dest%] = pixels[source%] 'red
'dest%=dest%+1
source%=source%+1
end for
end for
print "Time CDC1: ";timer.TotalMilliseconds() : timer.Mark()
Timer B is timing in pngRoku.brs:
data = createDataChunk(width, startheight,endheight, pixels,true)',false)
Timer tc5 is timing in pngRoku.brs in toChunk():
crc = rdCRC().updateCRC(crc,bnid)
crc = rdCRC().updateCRC(crc,traw)
My experience is that indexing into a roByteArray is fairly quick and probably does not need to be reviewed for further optimization unless you can get rid of the array entirely. What DOES make sense is arranging it so that it doesn’t need to reallocated more than once. Reallocating an array could cause a copy if the new size doesn’t fit in the heap space available after the old array.
In Zlibdeflate.brs inside writeUncompressDeflateBlock, there’s more rdRightShifts by 8. Replacing those, we get good results.
About a 3x overall improvement. I think I would need another order of magnitude to be real useful. Looking for 0.2 to MAYBE 1 second total operation on a 256x256 block. I wish there were an >> and >>> operators for right shift logic and right shift arithmetic.
Done Setting Alpha
Start Convert
Time tC1: 0
Time tC2: 0
Time tC3: 1
Time tC4: 0
Time tC5: 2
Time tC6: 1
Time A: 5
Time CDC1: 1039
raw.count = 261924
Time tC1: 0
Time tC2: 1
Time tC3: 0
Time tC4: 4
Time tC5: 2886
Time tC6: 2
Time B: 8631
Time tC1: 1
Time tC2: 0
Time tC3: 0
Time tC4: 0
Time tC5: 2
Time tC6: 0
Time C: 5
Time 8
PNG Count = 262032
I haven’t looked in detail, but if a lot of data is being copied unnecessarily, then that might be a further improvement. Another option to avoid copying is to pass in source and destination indexes and do the copying while writing in the first place. I’m guessing maybe 50% to 70% potential improvement, maybe more. Good work so far. I’m done on this today.
I still don’t understand how this would be useful - unless the Roku is sending the result png file back to an external server? Wouldn’t it be much quicker to supply an image in a format the roku can already use to display - or was the point of this simply an excercise in which case can you have it convert a PNG file to say, an TIF file for me - just to prove it can be done?
“destruk” wrote:
I still don’t understand how this would be useful - unless the Roku is sending the result png file back to an external server? Wouldn’t it be much quicker to supply an image in a format the roku can already use to display - or was the point of this simply an excercise in which case can you have it convert a PNG file to say, an TIF file for me - just to prove it can be done?
Performance aside, one of the primary benefits of something like this would be dynamic image creation for use on the built-in screens. As an example, consider a sports channel that has 30 teams. It would be a horribly tedious and space consuming task to create separate images for every single possible matchup. If you could dynamically build a matchup poster based on the individual team logos, then it could provide much more flexibility.
Take it even further, and say you were developing a game and wanted to provide functionality to allow players to upload screenshots of their high scores to various social media sites… or maybe provide the ability to design an avatar in game, and upload it to an associated website… or even capture screens for upload to the server in particular error scenarios…
I can think of countless uses for the ability to save bitmaps to PNGs/JPGs…
Forget I even commented on this. Nobody gets me, but me.
There are hundreds of ways to do the same thing for your situations - a little bit of thought would have -
sports teams -
One image with every team logo on it and use roRegion to select the logos you need to match up - while not having to convert the image at all, ever, not once. - Much faster, easier to manage, easier to code.
Scores - send the score to the server and using PHP’s built-in image manipulation and creation features to create the end result and convert using any of the free graphics libraries to spit or or even email the image while conserving the roku user’s bandwidth - in ANY format without needing to be limited to PNG.
Just because the Roku could balance the USA’s budget over a period of 6 months doesn’t mean it’s the best way to do it, but if that puts a feather in your cap and on your resume, that’s just totally AWESOME.
By using the correct tool for the job, with the most efficiency available in doing so, you’ll save a lot of time and generally end up with a better result.
Does that at all make sense to you guys, or are we still in the “good job, here is a cookie and a silver star on your name” phase?
“destruk” wrote:
Forget I even commented on this. Nobody gets me, but me.
There are hundreds of ways to do the same thing for your situations - a little bit of thought would have -
sports teams -
One image with every team logo on it and use roRegion to select the logos you need to match up - while not having to convert the image at all, ever, not once. - Much faster, easier to manage, easier to code.
I think you missed the “on the built-in screens” part. You can’t use an roRegion as a poster on the roGridScreen or roPosterScreen or roSpringboardScreen.
You also missed the “performance aside” bit. I was arguing the benefits of being able to create PNGs and JPGs dynamically on the box, preferably natively, not specifically using this library. Constant round trips to your server not only uses up unnecessary bandwidth, it also adds latency to the channel, and extra burden on the server. When you’re paying for all of those resources, that’s a big deal. Why wouldn’t you prefer to use local resources instead, if possible?
None of your other comments address the screenshots, avatars, and error captures scenarios I presented either. There are also other applications, like being able to save the composited state of a screen when transitioning between multiple custom screens, or saving composited background elements, so they don’t have to all be individually redrawn on every pass. Like I said, I can think of countless different examples of where this functionality would be useful.
“destruk” wrote:
Does that at all make sense to you guys, or are we still in the “good job, here is a cookie and a silver star on your name” phase?
You said you didn’t understand how it could be useful, and I presented a number of scenarios in which it would. If none of those apply to you or foster ideas of other ways you could potentially use it, then so be it, but I think it’s incredibly short-sighted of you to think that just because you haven’t run across a situation where you’d need such functionality that it’s completely useless. I personally have worked on a few projects where I didn’t have the luxury of having a backend server that I could put code on to dynamically generate images on the fly for me, in which case something like this (again, performance aside) would have come in very handy. You haven’t, and apparently think you never will, so good for you.. here’s your feather and your silver star…
“TheEndless” wrote:
I personally have worked on a few projects where I didn’t have the luxury of having a backend server that I could put code on to dynamically generate images on the fly for me, in which case something like this (again, performance aside) would have come in very handy.
+1
It just seems like an entirely too much amount of work to do something which could easily be incorporated into a core component like - copytojpg, or copytopng to convert bitmap data roku or a server already has direct access to to begin with. Much like why the loopback IP address doesn’t work for ECP commands - that’s shortsighted, so I’m simply stepping in Roku’s shortsighted lead
“destruk” wrote:
It just seems like an entirely too much amount of work to do something which could easily be incorporated into a core component like - copytojpg, or copytopng to convert bitmap data roku or a server already has direct access to to begin with.
You won’t get any arguments there. It’s definitely something that would be best to be implemented in the core. The fact that someone actually took the time to port a PNG library to Brightscript might be enough to convince Roku it’s necessary. It eventually happened with the JSON libraries we all put together, so maybe…
I’m getting warm fuzzies about all this. It’s awesome that rdPNG has been of use since we put it out.
As requested originally (years ago), shift and xor operators/functions from Roku would help a lot.
The optimizations found are really interesting. Especially the typed variables. We did preliminary tests around the time we wrote the libraries, and found that for general use types variables were actually slightly slower in some cases that their untyped counterparts. That may have been optimized away in later firmware versions, or the cost of boxing may have made a difference in this case (obviously there are other benefits to typing variables).
We seriously considered writing a zlib implemetation originally, but found a workaround for our needs (palettized PNGs, where we just need to set the colors to create different colored PNGs), and we were trying to meet the Roku channel contest deadline for KidPaint (no win, damn you renojim!
rdPNG was actually created as a way to easily and programmatically access and modify PNG pallete entries. You can actually see the end result in KidPaint, which is itself open sourced and on github (https://github.com/rokudev/kidpaint). It’s horribly dated, especially since there’s a real roScreen object to use now (in my day, we had to use multiple roImageCanvas object and weird timing quirks to achieve animation, and we did it up-hill both ways while walking to school… )
Also, if anyone’s interesting in commit bits to the librokudev repo, that can definitely be arranged… Otherwise, we accept pull requests.
“destruk” wrote:
It just seems like an entirely too much amount of work to do something which could easily be incorporated into a core component like - copytojpg, or copytopng to convert bitmap data roku or a server already has direct access to to begin with.
You won’t get any arguments there. It’s definitely something that would be best to be implemented in the core. The fact that someone actually took the time to port a PNG library to Brightscript might be enough to convince Roku it’s necessary. It eventually happened with the JSON libraries we all put together, so maybe…
I agree, would be very nice to have. A way to access pixels directly would be even nicer. I originally was looking into it as a way to use a PNG like framebuffer for doing pixel plotting but it is slower then doing a DrawRect(x ,y , 1, 1, &hFF) etc. Also would be nice to have access to an audio buffer for building sound samples, multithreading Even better would be to just have homebrew access to the 3D roku sdk .
I have done some more work on the encoder and checked my changes into github. I have combined all the loops into one function and added/modified the adler32/crc32 function to take a byte at a time. https://github.com/GPF/pngEncodeRoku
function IDATChunk(width as integer,height as integer,pixels as object) as object
chunk = CreateObject("roByteArray")
chunk.push(0)'placeholder for idat size
chunk.push(0)'placeholder for idat size
chunk.push(0)'placeholder for idat size
chunk.push(0)'placeholder for idat size
n=ADLER32calc()
c=rdCRC()
y=0
x=0
f=0
BLOCK_SIZE = 32000
zlength=BLOCK_SIZE
id=asc("I"):chunk.push(id):c.updateOneCRC(id)
id=asc("D"):chunk.push(id):c.updateOneCRC(id)
id=asc("A"):chunk.push(id):c.updateOneCRC(id)
id=asc("T"):chunk.push(id):c.updateOneCRC(id)
'Zlib Header
id=8:chunk.push(id):c.updateOneCRC(id):f=f+1 ' CM = 8, CMINFO = 0
id=(31 - ( (8*2^8) MOD 31 ) ) MOD 31:chunk.push(id):c.updateOneCRC(id):f=f+1 ' FCHECK (FDICT/FLEVEL=0)
for i = 0 to pixels.count()-1
if ((y+x) MOD (BLOCK_SIZE)) =0 then
if (( pixels.count()-y) < BLOCK_SIZE) then
zlength=(pixels.count()-y)+(height-x)
id=1:chunk.push(id):c.updateOneCRC(id):f=f+1 ' Final flag, Compression type
print "y= "+y.toStr()
else
id=0:chunk.push(id):c.updateOneCRC(id):f=f+1 ' Final flag, Compression type
print "lasty= "+y.toStr()
endif
print "zlength= "+zlength.toStr()
id=(zlength and &HFF):chunk.push(id):c.updateOneCRC(id):f=f+1 ' Length LSB
id=((zlength and &HFF00)/256):chunk.push(id):c.updateOneCRC(id):f=f+1 ' Length MSB
id=( (NOT zlength) and &HFF):chunk.push(id):c.updateOneCRC(id):f=f+1 ' Length 1st complement LSB
id=( ( (NOT zlength) and &HFF00)/256) ' Length 1st complement MSB
chunk.push(id):c.updateOneCRC(id):f=f+1
endif
if(y MOD (width*4)) =0 then
id=0:chunk.push(id):c.updateOneCRC(id):n.UpdateAdler(id):f=f+1 'no Filter
x=x+1
endif
id=pixels[i]:chunk.push(id):
c.updateOneCRC(id):n.UpdateAdler(id):f=f+1
y=y+1
endfor
ad=n.TotalAdler()
temp = CreateObject("roByteArray")
temp = rdINTtoBA(ad)
id=temp[0]:chunk.push(id):c.updateOneCRC(id):f=f+1
id=temp[1]:chunk.push(id):c.updateOneCRC(id):f=f+1
id=temp[2]:chunk.push(id):c.updateOneCRC(id):f=f+1
id=temp[3]:chunk.push(id):c.updateOneCRC(id):f=f+1
datsize=rdINTtoBA(f) 'idat size
chunk[0]=datsize[0]
chunk[1]=datsize[1]
chunk[2]=datsize[2]
chunk[3]=datsize[3]
chunk.append(rdINTtoBA(not (c.TotalOneCRC() ) ))
return chunk
end function
function ADLER32calc() as object
this = {
'Member vars
s1%:1
s2%:0
'Methods
ResetAdler: function () as integer
m.s1%=1
m.s2%=0
end function
UpdateAdler: function (abs% as integer)
m.s1% = (m.s1% + abs%) MOD 65521
m.s2% = (m.s2% + m.s1%) MOD 65521
end function
TotalAdler: function () as integer
return (m.s2%*65536) + m.s1%
end function
}
return this
end function
updateOneCRC: function(buf as integer)
crc=m.TheCRC
t = m.CRCTABLE
index = ((crc and not buf) or (not crc and buf)) and &hFF
shiftedc = ((crc and &hFFFFFF00)/256) and &hFFFFFF
m.TheCRC= ((shiftedc and not t[index]) or (not shiftedc and t[index]))
end function
TotalOneCRC: function() as integer
return m.TheCRC
end function
7
)