benchmark test: HaWe brickbench – for C# / Mono?

HomeForumsMonoBrick EV3 Firmwarebenchmark test: HaWe brickbench – for C# / Mono?

Tagged: ,

This topic contains 43 replies, has 4 voices, and was last updated by Author Image Helmut Wunder 4 years, 6 months ago.

Viewing 14 posts - 31 through 44 (of 44 total)
  • Author
    Posts
  • #5276
    Author Image
    Anders Søborg
    Key Master

    Hi there

    Here is a screenshot of my first run. Is this what you need?

    /Anders

    Attachments:
    You must be logged in to view attached files.
    #5278
    Author Image
    Helmut Wunder
    Participant

    thank you very much for your input!

    to my surprise these new data differ a lot from previous test results (range shown in braces for the 1st to the 5th run):
    your new data by 1st run show more or less identical values which had been performed previously not until far later runs, partially even faster than already the gpp C results for native executables which is actually more than surprising (even considering that for gpp C the Lego VM is still running simultaneously) – so this is now a much better performance for the start than before :

    
    no. new (old 1st - 5th)
    0     1   ( 15 - 1)
    1    25   ( 35 - 26)
    2   233   (212^- 155^) // renewed
    3    12   ( 19 - 13)
    4    40   (229 - 35 )
    5   150   (299 - 179)
    6   219   (395 - 268)
    7   158   (475 - 185)

    are you using a different compiler than before? in case yes: which version?
    for completion of the list I’m also very curious about how the values would be for e.g., the 3rd and the 5th run additionally!

    Thank you for your efforts!
    Helmut

    #5279
    Author Image
    Tcm0
    Participant

    Anders, what did you do to your EV3? :O

    #5280
    Author Image
    Helmut Wunder
    Participant

    maybe pimped by a BBB ? :P

    #5281
    Author Image
    Anders Søborg
    Key Master

    Hi

    I simply AOT compiled the program. But I think that the image that I showed in my last post was from the second run. Here are images from four runs.

    Attachments:
    You must be logged in to view attached files.
    #5286
    Author Image
    Anders Søborg
    Key Master

    Hi and here is the image from the fifth run…

    /Anders

    Attachments:
    You must be logged in to view attached files.
    #5288
    Author Image
    Helmut Wunder
    Participant

    ok – I’ll conclude:

    no. new min/max (old 1st - 5th)
    0     1 -   5   ( 15 - 1)
    1    25 -  67   ( 35 - 26)
    2   227 - 245   (212^- 155^) // renewed
    3    12 -  18   ( 19 - 13)
    4    40 -  98   (229 - 35 )
    5   150 - 176   (299 - 179)
    6   219 - 333   (395 - 268)
    7   158 - 302   (475 - 185)

    would this be correct ? (I#m a little confused, admittedly ;) )

    #5289
    Author Image
    Anders Søborg
    Key Master

    Hi again

    I did a run without AOT compiling the program. As expected the first run is slow while the remaining runs are very fast. See attached images

    Attachments:
    You must be logged in to view attached files.
    #5294
    Author Image
    Anders Søborg
    Key Master

    Hi and here is the image from the fifth run without AOT…

    Attachments:
    You must be logged in to view attached files.
    #5296
    Author Image
    Helmut Wunder
    Participant

    what is AOT and what is not-AOT ?
    what is the common way ?

    #5297
    Author Image
    Anders Søborg
    Key Master

    Hi there

    AOT is described here.

    what is the common way ?

    Hmmm hard to tell. Pick the one that rates best :-)

    /Anders

    #5298
    Author Image
    Helmut Wunder
    Participant

    do I understand correctly:
    AOT is precompiled,
    and instead it would be like JIT for leJOS ?

    so if it was precompiled – there shoudn’t be any remarkable runtime differences – correct ?

    FTM, this would be the benchmarks from best to worst case (either if AOT or JIT, and by either run:

    no. new min/max (old min/max)
    0     1 -  16   (  1 -  15)
    1    25 -  67   ( 26 -  35)
    2   227 - 245   (155^ -212^) // renewed
    3    12 -  18   ( 13 -  19)
    4    40 -  98   ( 35 - 229 )
    5   150 - 176   (179 - 299)
    6   219 - 333   (268 - 395)
    7   158 - 302   (185 - 475)

    would you say this is correct ?
    (tbh, a printed list would be a little more clearly arranged for evalutaion than just photos 8-) )

    #5299
    Author Image
    Anders Søborg
    Key Master

    Hi Helmut

    AOT is precompiled,
    and instead it would be like JIT for leJOS ?

    Correct.

    so if it was precompiled – there shoudn’t be any remarkable runtime differences – correct ?

    Actually it is not as simple as that.

    Not AOT compiled

    If it is not AOT compiled then you would see a high performance penalty the first time you run the test, this is actually also the case in my tests. The 2nd time you run the test the code was been JIT compiled and it will execute as fast as possible. This is also what you can see from the results.

    AOT compiled
    If the code has been AOT compiled there is no need for JIT compiling the code and you will not get a performance penalty on the first run. However since it is pre-compiled it is not optimized in any way (compared to JIT compile), so that is why the second AOT run is slightly slower than the second run of the Non AOT run.

    So the conclusion is that if you can live with a performance penalty the first time some code is executed then don’t bother AOT compiling the code otherwise AOT compile the code.

    would you say this is correct ?

    The table seems to be correct – but did you use AOT or JIT

    /Anders

    #5300
    Author Image
    Helmut Wunder
    Participant

    thank you for your explanation, now I see clear 8-)
    - I took the best and the worst runs of 1-5, of either way –
    just observed:
    1 current value had to be corrected, i.e. (4) for matrices.

    The HaWe Brickbench Benchmark comparison Tables now had been updated!
    (some questions marks remaining for daisy chaining, raw BT and raw Wifi etc.
    if you should encounter issues, mistakes or bugs: please feel free to report, inputs are highly appreciated!):

    http://www.mindstormsforum.de/viewtopic.php?f=71&t=8095&p=64772#p64772

    thank you Anders for your input and your participation !
    Over all really a great job! 8-)

Viewing 14 posts - 31 through 44 (of 44 total)

You must be logged in to reply to this topic.

Posted in

Make a donation