{"id":307,"date":"2008-09-10T21:28:23","date_gmt":"2008-09-10T09:28:23","guid":{"rendered":"https:\/\/www.deltics.co.nz\/blog\/?p=307"},"modified":"2008-09-10T21:28:23","modified_gmt":"2008-09-10T09:28:23","slug":"patching-a-class-vmt","status":"publish","type":"post","link":"https:\/\/www.deltics.co.nz\/blog\/posts\/307\/","title":{"rendered":"Patching A Class VMT"},"content":{"rendered":"[Estimated Reading Time: <\/span> 6<\/span> minutes]<\/span><\/span>

Caution: <\/strong>The contents of this post may cause dizziness or nausea.\u00a0 Take only as prescribed and if symptoms persist seek professional advice.<\/em><\/p>\n

Recently I found myself needing to do something I had never done before – create an instance of an arbitrary class derived from some base class and destroy it, and do so without invoking any<\/span><\/em> constructor or destructor code that the derived class(es) may have introduced. This is the difficult bit.<\/p>\n

Impossible?\u00a0 Don’t be silly, this is Delphi<\/span> we’re talking about….<\/p>\n

<\/p>\n

Method In The Madness<\/h3>\n

To hopefully explain (do I mean “justify”?) this madness, a bit of background to the initial problem may be useful.<\/p>\n

The need that I had for this insanity stemmed (stum?) from a perfectly reasonable limitation imposed on code in class methods – they cannot reference normal instance methods.\u00a0 By which I don’t mean call them – they cannot even reference them (directly at least).<\/p>\n

This is because a method reference, as previously discovered in the Multicast Events series, is actually a pair<\/em> of pointers – a pointer to an instance and<\/span> the method entry point.\u00a0 A class method – by definition – does not have an instance reference so a valid TMethod<\/strong> reference cannot be formed by the compiler.<\/p>\n

So this will not compile:<\/p>\n

  type\r\n    TVariantMethod = function: Variant of object;\r\n    TObjectMethod = procedure(var aObject: TObject) of object;\r\n\r\n    TFoo = class(TBase)\r\n    private\r\n      function FuncA: Variant;\r\n      procedure ProcA(var aObject: TObject);\r\n    protected\r\n      class procedure AddName(const aName: String;\r\n                              aMethod: TVariantMethod); overload;\r\n      class procedure AddName(const aName: String;\r\n                              aMethod: TObjectMethod); overload;\r\n      class procedure RegisterMethods; virtual;\r\n    end;\r\n\r\n    class procedure TFoo.RegisterMethods;\r\n    begin\r\n      AddName('Func', FuncA); \/\/ Illegal reference to instance method\r\n      AddName('Proc', ProcA);  \/\/ Illegal reference to instance method\r\n    end;<\/pre>\n
In this case however, whilst I had a need to refer<\/span> to a method, I was not actually concerned with any instance – a TMethod<\/strong> with a valid code pointer and a NIL instance would have been fine (later on when the methods were needing to be called for a specific known instance the underlying TMethod record Data field would be fixed up dynamically as required).<\/p>\n
But as I say, this code simply will not compile – a class procedure cannot reference an instance method.  Period.<\/p>\n
So why not extract and pass a simple procedure or function pointer?  i.e. a CODE pointer, without any DATA.<\/p>\n
Firstly of course, a pointer to a “first class” function cannot be passed where a method reference (function of object) is expected – they are very different things.  Ditto procedures of course.<\/p>\n
Which left me with two possible options:<\/p>\n
1. Untyped pointers
\n2. Temporary objects<\/p>\n
Casting Type Safety To The Wind<\/h3>\n
An untyped pointer approach could have worked but is inherently unsafe (patching the VMT isn’t?! [Ed]) and results in a more cumbersome implementation overall:<\/p>\n
  type\r\n    TFoo = class(TBase)\r\n      :\r\n    protected\r\n      class procedure AddFunctionName(const aName: String;\r\n                                      aMethodAddr: Pointer);\r\n      class procedure AddObjectName(const aName: String;\r\n                                    aMethodAddr: Pointer);\r\n      class procedure RegisterMethods; virtual;\r\n    end;\r\n\r\n    class procedure TFoo.RegisterMethods;\r\n    begin\r\n      AddFunctionName('Func', @TFoo.FuncA);\r\n      AddObjectName('Proc', @TFoo.ProcA);\r\n    end;<\/pre>\n
In this approach the type safety of the two different method types is lost completely.  This means not only that separate methods are now required to register the two different types of method, but also a mistake could very easily be made and not picked up by the compiler.<\/p>\n
i.e. a pointer to a function returning a variant could be passed to the AddObjectName()<\/strong> method, with dire consequences at runtime when the function returning a Variant<\/strong> is called as if it were a procedure yielding a TObject<\/strong> reference in a var parameter.<\/p>\n
In practice there were scores of classes registering many methods each – the potential for errors to creep in was too great.<\/p>\n
Temporary Insanity<\/h3>\n
The temporary object approach\u00a0should be straight forward though:<\/p>\n
Convert class procedures to regular instance methods and at the point at which code would normally invoke the class method instead create an instance, call the necessary methods, then dispose (i.e. Free<\/strong>) the temporary object:<\/p>\n
  type\r\n    TFoo = class(TBase)\r\n      :\r\n    protected\r\n      procedure AddName(const aName: String;\r\n                   aMethod: TVariantMethod); overload;\r\n      procedure AddName(const aName: String;\r\n                   aMethod: TObjectMethod); overload;\r\n      procedure RegisterMethods; virtual;\r\n    end;\r\n\r\n    procedure TFoo.RegisterMethods;\r\n    begin\r\n      AddName('Func', FuncA);\r\n      AddName('Proc', ProcA);\r\n    end;\r\n\r\n  \/\/ Class registration code:\r\n  procedure RegisterClass(const aClass: TFooClass);\r\n  var\r\n    temp: TFoo;\r\n  begin\r\n    temp := aClass.Create;\r\n    try\r\n      temp.RegisterMethods;\r\n    finally\r\n      temp.Free;\r\n    end;\r\n  end;<\/pre>\n
Type safety is restored and all is well.<\/p>\n
Just one problem.  Whilst in theory this should not have presented a problem, in practice things were not quite so simple.<\/p>\n
First, in the actual real world case, TFoo<\/strong> was the base class of an existing complex and extensive hierarchy of classes, many of which might be considered “legacy” code and were not – shall we say – meticulously designed.  Numerous constructors were introduced at various levels in the hierarchy and of course a variety of destructors were involved.  Some classes relied on AfterConstruction<\/strong> and BeforeDestruction<\/strong> overrides, and a lot of code in the constructor and destructor chains made assumptions about what was or had been initialised, and in many cases destructors were very sensitive to the correct constructor having been used etc etc.<\/p>\n
All in all, a bit of a problem, the result of which was that many classes could not be instantiated using a common, “default” constructor in this throw-away manner – numerous errors occured, especially in the destructor chains of some classes.<\/p>\n
Wrapping the Free<\/strong> in a try..except<\/strong> could have dealt with the obvious symptoms, but the risk of side effects was too great.<\/p>\n
Introducing a default constructor that could be reliably used was no problem.  The much bigger problem was avoiding any AfterConstruction code and the destructor chain without modifying every single piece of code in these possible methods to only execute if handling a “genuine” (i.e. not a temporary) object – a horrible intrusion of root behaviours into the hierarchy derived classes (that would also have to be reflected in any new classes or future changes to existing ones).<\/p>\n
It was at this point that I remembered some of the secrets contained within the VMT (Virtual Method Table).<\/p>\n
Here’s One I Prepared Earlier<\/h3>\n
It just so happened that in developing my SmokeTest testing framework I had implemented an exposure of the VMT, drawing on the work of Ray Lischner in “Delphi In A Nutshell”<\/a> (every Delphi developer should have at least one copy of this on their book shelf) and the blog of Hallvard Vassbotn.<\/p>\n
At the time I was only interested in the published method information accessible via the VMT, but the VMT contains more than this, as Hallvard Vassbotn explains very nicely<\/a>.<\/p>\n
The key is the fact that the entry points for ALL the problem areas affecting my temporary object approach are contained in the VMT – AfterConstruction<\/strong>, BeforeDestruction<\/strong> and the Destroy<\/strong> method itself, are all there in the VMT.<\/p>\n
If the VMT contains these entry points then if I modify the VMT, could I not in fact “patch” these fundamental class behaviours at runtime? \u00a0As long as I restored the VMT to it’s original state once I was done with it no-one would be any the wiser….<\/p>\n
\r\n  constructor TFoo.CreateTemporary;\r\n  begin\r\n    \/\/ NO-OP\r\n  end;\r\n\r\n  \r\n  procedure RegisterClass(const aClass: TFooClass);\r\n  var\r\n    vmt: PVirtualMethodTable;\r\n    oldAfterConstruction: PAfterConstruction;\r\n    oldBeforeDestruction: PBeforeDestruction;\r\n    oldDestroy: PDestroy;\r\n    temp: TFoo;\r\n  begin\r\n    vmt := GetVirtualMethodTable(aClass);\r\n\r\n    oldAfterConstruction := vmt.AfterConstruction;\r\n    oldBeforeDestruction := vmt.BeforeDestruction;\r\n    oldDestroy := vmt.Destroy;\r\n\r\n    vmt.AfterConstruction := NIL;\r\n    vmt.BeforeDestruction := NIL;\r\n    vmt.Destroy := NIL;\r\n    try\r\n      temp := aClass.CreateTemporary;\r\n      try\r\n        temp.RegisterMethods;\r\n      finally\r\n        temp.Free;\r\n      end;\r\n    finally\r\n      vmt.AfterConstruction := oldAfterConstruction;\r\n      vmt.BeforeDestruction := oldBeforeDestruction;\r\n      vmt.Destroy := oldDestroy;\r\n    end;\r\n  end;<\/pre>\n
The pointer types referenced in this code and the virtual method table record type are as described in Hallvard Vossbotn’s article.<\/p>\n
This compiles just fine but will blow up at runtime with an access violation on the first attempt to modify a member of the vmt<\/strong> record:<\/p>\n
  vmt.AfterConstruction := NIL; \/\/ BOOM! <\/pre>\n
How is that possible?  This can’t be an invalid memory address and can’t be outside our process.  Why would reading from this address be OK, but writing cause an access violation?<\/p>\n
This had me scratching my head for a while until I stumbled across what looks like some GExperts code in the Koders database<\/a>.  This shows how to patch a user defined virtual method (and also how to change the parent class of a class!).<\/p>\n
The key of course is the toggling of protection flags on the memory occupied by the VMT record.  There is also a subtle requirement that I initially missed.  The AfterConstruction<\/strong>, BeforeDestruction<\/strong> and Destroy<\/strong> pointers cannot in fact be NIL – the runtime will automatically call the methods that these fields point to, so they have to point to something; some no-op\/null stubs are required.<\/p>\n
So one last refinement and we’re done:<\/p>\n
  \/\/ NOTE: These are regular procedures, NOT \"of object\" !!\r\n  procedure NullAfterConstruction(aObject: TObject);\r\n  begin end;\r\n\r\n  procedure NullBeforeDestruction(aObject: TObject); \r\n  begin end;\r\n\r\n  procedure NullDestroy(aObject: TObject; aOutermost: SmallInt); \r\n  begin end;\r\n\r\n\r\n  procedure RegisterClass(const aClass: TFooClass);\r\n  var\r\n    vmt: PVirtualMethodTable;\r\n    oldAfterConstruction: PAfterConstruction;\r\n    oldBeforeDestruction: PBeforeDestruction;\r\n    oldDestroy: PDestroy;\r\n    oldProtect: DWord;\r\n    temp: TFoo;\r\n  begin\r\n    vmt := GetVirtualMethodTable(aClass);\r\n    oldAfterConstruction := vmt.AfterConstruction;\r\n    oldBeforeDestruction := vmt.BeforeDestruction;\r\n    oldDestroy := vmt.Destroy;\r\n\r\n    if NOT VirtualProtect( vmt, sizeof(TVirtualMethodTable). PAGE_READWRITE, oldProtect) then\r\n      RaiseLastOSError;\r\n\r\n    vmt.AfterConstruction := NullAfterConstruction;\r\n    vmt.BeforeDestruction := NullBeforeDestruction;\r\n    vmt.Destroy := NullDestroy;\r\n    try\r\n      temp := aClass.CreateTemporary;\r\n      try\r\n        temp.RegisterMethods;\r\n      finally\r\n        temp.Free;\r\n      end;\r\n    finally\r\n      vmt.AfterConstruction := oldAfterConstruction;\r\n      vmt.BeforeDestruction := oldBeforeDestruction;\r\n      vmt.Destroy := oldDestroy;\r\n\r\n      if NOT VirtualProtect( vmt, sizeof(TVirtualMethodTable). oldProtect, oldProtect) then\r\n        RaiseLastOSError;\r\n    end;\r\n  end;<\/pre>\n
And that’s it.  This creates a TFoo<\/strong> using a known, reliable no-op constructor and effectively suppresses all possible derived construction and destructor behaviour – that’s fine because we know that the constructor we introduced is a no-op that requires no clean up.<\/p>\n
One thing to be VERY<\/span><\/strong> careful of however is that this technique is absolutely NOT<\/em> thread-safe and cannot, as far as I can see, easily be made so.<\/p>\n
Epilogue \/ Footnote<\/h3>\n
After devising this solution it was subsequently decided that “fixing” the destructors in the class hierarchy was a desirable change in and of itself, and this in turn would allow a temporary object to be created without having to patch the VMT after all.<\/p>\n
The fixing of destructors in this case simply involved making them resilient to only partially initialised objects.  So a destructor containing code similar to this:<\/p>\n
\r\n  destructor TSomeFooClass.Destroy;\r\n  begin\r\n    fList.Clear;\r\n    fList.Free;\r\n    inherited;\r\n  end;\r\n<\/pre>\n
would be changed to:<\/p>\n
\r\n  destructor TSomeFooClass.Destroy;\r\n  begin\r\n    if Assigned(fList) then\r\n    begin\r\n      fList.Clear;\r\n      FreeAndNIL(fList);\r\n    end;\r\n\r\n    inherited;\r\n  end;\r\n<\/pre>\n
Which to my mind is the correct way to implement a destructor anyway (and yes I know lists do not ordinarily need to be explicitly Clear’d before Free’ing, but you get the idea).<\/p>\n
Thankfully, having had the fun of figuring out the VMT patch (a few minutes work all told) the rather less rewarding task of fixing all the destructors fell to someone else.<\/p>\n
\ud83d\ude42<\/p>\n","protected":false},"excerpt":{"rendered":"
[Estimated Reading Time: <\/span> 6<\/span> minutes]<\/span><\/span> Caution: The contents of this post may cause dizziness or nausea.\u00a0 Take only as prescribed and if symptoms persist seek professional advice. Recently I found myself needing to do something I had never done before – create an instance of an arbitrary class derived from some base class and destroy it, and do so without […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":[]},"categories":[4],"tags":[292,57,58,56],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p1TKYv-4X","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":338,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/338\/","url_meta":{"origin":307,"position":0},"title":"Delphi 2009 – A Heads-Up for Low-Level Coders","date":"13 Sep 2008","format":false,"excerpt":"Prompted by a conversation with some colleagues where-in we collectively speculated about the implementation details of a generic class and what impact - if any - this might have on performance vs a \"traditional\" polymorphic equivalent, I threw together a quick performance test case in my Smoketest framework, and as\u2026","rel":"","context":"In "Delphi"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":961,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/961\/","url_meta":{"origin":307,"position":1},"title":"Thinking Creatively with QueryInterface()","date":"02 Aug 2012","format":false,"excerpt":"I soon hope to be releasing \"Smoketest\", a testing framework that I have developed over the past few years. It has actually been in production use for most of that time (albeit by my own good self) but also continues to develop and evolve. \u00a0On the occasions when I have\u2026","rel":"","context":"In "Delphi"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1833,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/1833\/","url_meta":{"origin":307,"position":2},"title":"Importing an Android Class For Use in Delphi","date":"03 Oct 2013","format":false,"excerpt":"In a previous post I noted the absence of the BatteryManager class in the AndroidAPI.JNI units. This class contains some constants useful when reading battery information. I showed how to use a suitably massaged literal in place of these missing constants, but in response to observations from Paul and Brian\u2026","rel":"","context":"In "Android"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2169,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2169\/","url_meta":{"origin":307,"position":3},"title":"Smoketest – Set Me Up \/ Tear Me Down","date":"20 Nov 2013","format":false,"excerpt":"In a previous post I demonstrated how the default \"pretty name\" for a Smoketest test case (derived from the test case classname) can be over-ridden by a test developer by implementing a specific interface (INameCase) on the test case class itself. There are some other interfaces that can be implemented\u2026","rel":"","context":"In "Delphi"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1834,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/1834\/","url_meta":{"origin":307,"position":4},"title":"To init() or Not to Init(), That is the Correction","date":"03 Oct 2013","format":false,"excerpt":"The short answer is: Yes, init() In a previous post I lamented the fact [cough - Ed] that some Java class imports appeared to support the underlying Java constructors via an init() method, where others did not. This wasn't a \"fact\" at all. It was a misunderstanding based on an\u2026","rel":"","context":"In "Android"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2095,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2095\/","url_meta":{"origin":307,"position":5},"title":"Extending Smoketest (Part 1) – An Inspector Calls","date":"05 Nov 2013","format":false,"excerpt":"In the soon to be released Smoketest framework it is sometimes useful to create new test types to supplement the tests built-in to the framework. In this and the next post I will walk through the process of implementing and registering a custom test with the Smoketest framework. In a\u2026","rel":"","context":"In "Delphi"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/307"}],"collection":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/comments?post=307"}],"version-history":[{"count":13,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/307\/revisions"}],"predecessor-version":[{"id":320,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/307\/revisions\/320"}],"wp:attachment":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/media?parent=307"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/categories?post=307"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/tags?post=307"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}