{"id":2185,"date":"2013-11-22T14:49:29","date_gmt":"2013-11-22T02:49:29","guid":{"rendered":"https:\/\/www.deltics.co.nz\/blog\/?p=2185"},"modified":"2013-11-22T14:49:29","modified_gmt":"2013-11-22T02:49:29","slug":"smoketest-performance-case-visualisations","status":"publish","type":"post","link":"https:\/\/www.deltics.co.nz\/blog\/posts\/2185\/","title":{"rendered":"Smoketest – Performance Case Visualisations"},"content":{"rendered":"[Estimated Reading Time: <\/span> 2<\/span> minutes]<\/span><\/span>

This post is a peek behind the curtain of the next major update to Smoketest<\/strong> which I hope to have completed shortly: Performance Case visualisations<\/strong>.<\/p>\n

<\/p>\n

Smoketest<\/strong> has always had two types of test case that you could implement by deriving from two distinct base classes:<\/p>\n

TTestCase<\/strong> is the base class for correctness testing.<\/p>\n

TPerformanceCase<\/strong> is the base class for performance testing.<\/p>\n

The performance cases were what I used some years ago to compare the string handling performance of the RTL in different versions of Delphi<\/a> around the time that the switch was made to Unicode.<\/p>\n

I noted at the time that I was compiling the results manually to obtain the comparison data, but the raw data for those comparisons was coming from a Smoketest<\/strong> project that I put together to perform the actual tests.<\/p>\n

I had always intended to implement improved data capture and visualisations and I am now finally getting around to it.<\/p>\n

It is still very much a work in progress at this stage, but I have capture and comparisons implemented for running the same test project compiled with different Delphi versions and comparing the results in the Smoketest<\/strong> GUI itself:<\/p>\n

\"Preliminary<\/a>
Preliminary Performance Data Visualisation<\/figcaption><\/figure>\n

You can’t run all the different Delphi versions of a set of tests from within the same EXE of course, so there are some mechanics behind this which will need to be explained later on.<\/p>\n

There is also the facility to compare results between two different test cases in the same project:<\/p>\n

\"Case<\/a>
Case Comparison<\/figcaption><\/figure>\n

As might be more noticeable in this example, currently all test results are “normalised” against the worst result in each case<\/em>. i.e. the fact that the bars for the first three WIDE results (blue) in the above test are the exact same length as each other and as the green ANSI bars in the remaining five tests, does not mean that the performance is equal.<\/p>\n

What it does tell us is that in the first three cases the WIDE test yielded the worst result but in the remaining cases it was the ANSI test that did least well.<\/p>\n

I also have some other comparisons in mind but these may come later. It’s still early days yet and I shall blog about how performance cases and visualisations work in Smoketest<\/strong> in more detail when this work is nearer completion.<\/p>\n","protected":false},"excerpt":{"rendered":"

[Estimated Reading Time: <\/span> 2<\/span> minutes]<\/span><\/span> This post is a peek behind the curtain of the next major update to Smoketest which I hope to have completed shortly: Performance Case visualisations.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":[]},"categories":[4],"tags":[292,262,51],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p1TKYv-zf","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":2144,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2144\/","url_meta":{"origin":2185,"position":0},"title":"Smoketest 1.0 – Release and Be Damned!","date":"14 Nov 2013","format":false,"excerpt":"As I have been promising for some time (quite literally 5 years (!), I am ashamed to admit) I am finally unclenching and releasing the Smoketest framework into the wild, ready or not. The code is published and will continue to be updated in a github repository. Documentation is still\u2026","rel":"","context":"In "Delphi"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/Screen-Shot-2013-11-14-at-19.51.57-.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":375,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/375\/","url_meta":{"origin":2185,"position":1},"title":"Delphi 2009 – StringPerformance Redux","date":"22 Sep 2008","format":false,"excerpt":"It looks like I may have jumped the gun with my conclusions from the previous exercise to benchmark string performance in Delphi 2009.\u00a0 Following a useful exchange in the comments with Kryvich I corrected a small discrepancy in the tests and made some changes to the performance testing subsystem within\u2026","rel":"","context":"In "Delphi"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/delphi2009-stringperformance-chart.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":2193,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2193\/","url_meta":{"origin":2185,"position":2},"title":"DUnit Compatibility in Smoketest","date":"25 Nov 2013","format":false,"excerpt":"To address some odd concerns about differences between DUnit and Smoketest, I thought it would be useful to demonstrate how it is entirely within the gift of a Smoketest user to create their own \"comfort\" layer, to make using Smoketest more similar to the DUnit framework if they wish (though\u2026","rel":"","context":"In "Delphi"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2095,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2095\/","url_meta":{"origin":2185,"position":3},"title":"Extending Smoketest (Part 1) – An Inspector Calls","date":"05 Nov 2013","format":false,"excerpt":"In the soon to be released Smoketest framework it is sometimes useful to create new test types to supplement the tests built-in to the framework. In this and the next post I will walk through the process of implementing and registering a custom test with the Smoketest framework. In a\u2026","rel":"","context":"In "Delphi"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2956,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2956\/","url_meta":{"origin":2185,"position":4},"title":"Introducing: Smoketest 2.0","date":"24 Sep 2019","format":false,"excerpt":"Meet the new Smoketest. Nothing like the old Smoketest. A new, ultra-lightweight unit testing framework with xUnit 2.x result output (and the ability to support other output formats as needed).","rel":"","context":"In "automation"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/vape_2.jpg?fit=1200%2C307&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":2164,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2164\/","url_meta":{"origin":2185,"position":5},"title":"Smoketest – Some Differences With DUnit","date":"19 Nov 2013","format":false,"excerpt":"Writing tests in Smoketest is intended to enable a test developer to write tests in a way that describe themselves, without requiring the test developer to add this \"narrative\" themselves. To see this in action, I thought I would compare some simple DUnit tests with the equivalent using the Smoketest\u2026","rel":"","context":"In "Delphi"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/2185"}],"collection":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/comments?post=2185"}],"version-history":[{"count":4,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/2185\/revisions"}],"predecessor-version":[{"id":2192,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/2185\/revisions\/2192"}],"wp:attachment":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/media?parent=2185"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/categories?post=2185"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/tags?post=2185"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}