Reply To: benchmark test: HaWe brickbench – for C# / Mono?

[Helmut Wunder]

Vlad,
thank you very much for your input and for your cooperation – I was afraid you might perceive my post as sort of rude, I apologize if you might have had this impression – but it was only meant to avoid obsolete data to misrepresent the real comparison at any later date which would be unfair to the other tests (not sure if I could express my intention correctly). Of course you are free to publish your own results which is in your own responsibility – and I certainly would appreciate if you’d publish them singularily without showing different platform data!

Indeed some changes meanwhile have been made in the RobotC VM which lead to a quicker runtime performance.
The code and the test have been written and performed by Xander Soldaat, and he wrote me that especially the sort algorithm is meanwhile replaced in an internal build by an intrinsic test with the speed comparable to Linux executables.
As soon as the new RC build has been released he surely will publish the updated results.
If you like, you certainly may perform the RC test and publish them, too (with a date time and a RC version stamp), and I then will gladly always update the newest results in the comparison chart. As I don’t have RC, I can’t do the tests by myself.
BTW, there are additional issues about the leJOS benchmarks and I’m waiting for Andy Shaw’s fixes, too, to include them as well, just post them please directly here in the forum again if you wish!

About Mono/C#:
Now I also understand the loop thing in your table. It appears to be something similar to the Java JIT (just-in-time) compiler. Actually it’s complicated to unite all those runtime data for each VM and each loop in my spreadsheet table – finally it would be bursting at the seams. Maybe just loops #1 + #3 would give a representative cross-section about this, I’ll add them ASAP.

Just to outline what the rest of the benchs is about (of course I know that you won’t know all the details without exception):
I would appreciate to get to know the matrix and the sort tests, possibly also the text_out. If there is no graphic lib, we’ll have to skip that thing (nxtOSEK also provides no graphic lib, but Martin Aumair wrote and published his own lib so that it could be performed).
About memory: is there a memavail() function where you can read this SRAM mem during runtime? Is it up to 64MByte like gpp C or even more, maybe up to all built-in 264 MByte?
Is the code size limited in either way?
About BT or daisy-chaining or multi-robot-communication and all that: Do you have an idea if direct I/O control is possible? Is there something written in the documentation? (The Lego FW provides 1+3 for USB daisy chaining and 1+7 for raw BT comm).
The multi-dim array thing is about how to manipulate multi-dim arrays, e.g. after beeing passed to functions. If you wish to compare the gpp C code to the RobotC or the nxtOSEK code you’ll recognize what I mean:
by gpp C, you can access both rows and columns individually/seperately, in other cases this is not possible, one has to multiply the rows by number of columns to access the cell in a virtual linearized array.
The first feature is the better one, of course, and this is what this point is about.
The last things about int, long, and float already have been answered in our German forum (you are welcome to join it!): http://www.mindstormsforum.de/viewtopic.php?f=71&t=8095&p=65299#p65297

The last line is about API completeness:
– can motors be commanded to run certain degrees by certain pwm? Is there a PID control to approach encoder targets?
– can all sensors of Lego EV3 and NXT kits be polled? Are there general i2c functions to poll 3rd party devices, too (even chained ones by reading/writing different single addr and data registers)?
– Screen API is complete AFAIK for writing text to any (x,y) position, graphics are not available as already mentioned.

Anyway, thanks again for your interest, your input and thanks for sharing your code! 🙂