Caution: The contents of this post may cause dizziness or nausea. Take only as prescribed and if symptoms persist seek professional advice.
Recently I found myself needing to do something I had never done before – create an instance of an arbitrary class derived from some base class and destroy it, and do so without invoking any constructor or destructor code that the derived class(es) may have introduced. This is the difficult bit.
Impossible? Don’t be silly, this is Delphi we’re talking about….
Method In The Madness
To hopefully explain (do I mean “justify”?) this madness, a bit of background to the initial problem may be useful.
The need that I had for this insanity stemmed (stum?) from a perfectly reasonable limitation imposed on code in class methods – they cannot reference normal instance methods. By which I don’t mean call them – they cannot even reference them (directly at least).
This is because a method reference, as previously discovered in the Multicast Events series, is actually a pair of pointers – a pointer to an instance and the method entry point. A class method – by definition – does not have an instance reference so a valid TMethod reference cannot be formed by the compiler.
So this will not compile:
type TVariantMethod = function: Variant of object; TObjectMethod = procedure(var aObject: TObject) of object; TFoo = class(TBase) private function FuncA: Variant; procedure ProcA(var aObject: TObject); protected class procedure AddName(const aName: String; aMethod: TVariantMethod); overload; class procedure AddName(const aName: String; aMethod: TObjectMethod); overload; class procedure RegisterMethods; virtual; end; class procedure TFoo.RegisterMethods; begin AddName('Func', FuncA); // Illegal reference to instance method AddName('Proc', ProcA); // Illegal reference to instance method end;
In this case however, whilst I had a need to refer to a method, I was not actually concerned with any instance – a TMethod with a valid code pointer and a NIL instance would have been fine (later on when the methods were needing to be called for a specific known instance the underlying TMethod record Data field would be fixed up dynamically as required).
But as I say, this code simply will not compile – a class procedure cannot reference an instance method. Period.
So why not extract and pass a simple procedure or function pointer? i.e. a CODE pointer, without any DATA.
Firstly of course, a pointer to a “first class” function cannot be passed where a method reference (function of object) is expected – they are very different things. Ditto procedures of course.
Which left me with two possible options:
1. Untyped pointers
2. Temporary objects
Casting Type Safety To The Wind
An untyped pointer approach could have worked but is inherently unsafe (patching the VMT isn’t?! [Ed]) and results in a more cumbersome implementation overall:
type TFoo = class(TBase) : protected class procedure AddFunctionName(const aName: String; aMethodAddr: Pointer); class procedure AddObjectName(const aName: String; aMethodAddr: Pointer); class procedure RegisterMethods; virtual; end; class procedure TFoo.RegisterMethods; begin AddFunctionName('Func', @TFoo.FuncA); AddObjectName('Proc', @TFoo.ProcA); end;
In this approach the type safety of the two different method types is lost completely. This means not only that separate methods are now required to register the two different types of method, but also a mistake could very easily be made and not picked up by the compiler.
i.e. a pointer to a function returning a variant could be passed to the AddObjectName() method, with dire consequences at runtime when the function returning a Variant is called as if it were a procedure yielding a TObject reference in a var parameter.
In practice there were scores of classes registering many methods each – the potential for errors to creep in was too great.
The temporary object approach should be straight forward though:
Convert class procedures to regular instance methods and at the point at which code would normally invoke the class method instead create an instance, call the necessary methods, then dispose (i.e. Free) the temporary object:
type TFoo = class(TBase) : protected procedure AddName(const aName: String; aMethod: TVariantMethod); overload; procedure AddName(const aName: String; aMethod: TObjectMethod); overload; procedure RegisterMethods; virtual; end; procedure TFoo.RegisterMethods; begin AddName('Func', FuncA); AddName('Proc', ProcA); end; // Class registration code: procedure RegisterClass(const aClass: TFooClass); var temp: TFoo; begin temp := aClass.Create; try temp.RegisterMethods; finally temp.Free; end; end;
Type safety is restored and all is well.
Just one problem. Whilst in theory this should not have presented a problem, in practice things were not quite so simple.
First, in the actual real world case, TFoo was the base class of an existing complex and extensive hierarchy of classes, many of which might be considered “legacy” code and were not – shall we say – meticulously designed. Numerous constructors were introduced at various levels in the hierarchy and of course a variety of destructors were involved. Some classes relied on AfterConstruction and BeforeDestruction overrides, and a lot of code in the constructor and destructor chains made assumptions about what was or had been initialised, and in many cases destructors were very sensitive to the correct constructor having been used etc etc.
All in all, a bit of a problem, the result of which was that many classes could not be instantiated using a common, “default” constructor in this throw-away manner – numerous errors occured, especially in the destructor chains of some classes.
Wrapping the Free in a try..except could have dealt with the obvious symptoms, but the risk of side effects was too great.
Introducing a default constructor that could be reliably used was no problem. The much bigger problem was avoiding any AfterConstruction code and the destructor chain without modifying every single piece of code in these possible methods to only execute if handling a “genuine” (i.e. not a temporary) object – a horrible intrusion of root behaviours into the hierarchy derived classes (that would also have to be reflected in any new classes or future changes to existing ones).
It was at this point that I remembered some of the secrets contained within the VMT (Virtual Method Table).
Here’s One I Prepared Earlier
It just so happened that in developing my SmokeTest testing framework I had implemented an exposure of the VMT, drawing on the work of Ray Lischner in “Delphi In A Nutshell” (every Delphi developer should have at least one copy of this on their book shelf) and the blog of Hallvard Vassbotn.
At the time I was only interested in the published method information accessible via the VMT, but the VMT contains more than this, as Hallvard Vassbotn explains very nicely.
The key is the fact that the entry points for ALL the problem areas affecting my temporary object approach are contained in the VMT – AfterConstruction, BeforeDestruction and the Destroy method itself, are all there in the VMT.
If the VMT contains these entry points then if I modify the VMT, could I not in fact “patch” these fundamental class behaviours at runtime? As long as I restored the VMT to it’s original state once I was done with it no-one would be any the wiser….
constructor TFoo.CreateTemporary; begin // NO-OP end; procedure RegisterClass(const aClass: TFooClass); var vmt: PVirtualMethodTable; oldAfterConstruction: PAfterConstruction; oldBeforeDestruction: PBeforeDestruction; oldDestroy: PDestroy; temp: TFoo; begin vmt := GetVirtualMethodTable(aClass); oldAfterConstruction := vmt.AfterConstruction; oldBeforeDestruction := vmt.BeforeDestruction; oldDestroy := vmt.Destroy; vmt.AfterConstruction := NIL; vmt.BeforeDestruction := NIL; vmt.Destroy := NIL; try temp := aClass.CreateTemporary; try temp.RegisterMethods; finally temp.Free; end; finally vmt.AfterConstruction := oldAfterConstruction; vmt.BeforeDestruction := oldBeforeDestruction; vmt.Destroy := oldDestroy; end; end;
The pointer types referenced in this code and the virtual method table record type are as described in Hallvard Vossbotn’s article.
This compiles just fine but will blow up at runtime with an access violation on the first attempt to modify a member of the vmt record:
vmt.AfterConstruction := NIL; // BOOM!
How is that possible? This can’t be an invalid memory address and can’t be outside our process. Why would reading from this address be OK, but writing cause an access violation?
This had me scratching my head for a while until I stumbled across what looks like some GExperts code in the Koders database. This shows how to patch a user defined virtual method (and also how to change the parent class of a class!).
The key of course is the toggling of protection flags on the memory occupied by the VMT record. There is also a subtle requirement that I initially missed. The AfterConstruction, BeforeDestruction and Destroy pointers cannot in fact be NIL – the runtime will automatically call the methods that these fields point to, so they have to point to something; some no-op/null stubs are required.
So one last refinement and we’re done:
// NOTE: These are regular procedures, NOT "of object" !! procedure NullAfterConstruction(aObject: TObject); begin end; procedure NullBeforeDestruction(aObject: TObject); begin end; procedure NullDestroy(aObject: TObject; aOutermost: SmallInt); begin end; procedure RegisterClass(const aClass: TFooClass); var vmt: PVirtualMethodTable; oldAfterConstruction: PAfterConstruction; oldBeforeDestruction: PBeforeDestruction; oldDestroy: PDestroy; oldProtect: DWord; temp: TFoo; begin vmt := GetVirtualMethodTable(aClass); oldAfterConstruction := vmt.AfterConstruction; oldBeforeDestruction := vmt.BeforeDestruction; oldDestroy := vmt.Destroy; if NOT VirtualProtect( vmt, sizeof(TVirtualMethodTable). PAGE_READWRITE, oldProtect) then RaiseLastOSError; vmt.AfterConstruction := NullAfterConstruction; vmt.BeforeDestruction := NullBeforeDestruction; vmt.Destroy := NullDestroy; try temp := aClass.CreateTemporary; try temp.RegisterMethods; finally temp.Free; end; finally vmt.AfterConstruction := oldAfterConstruction; vmt.BeforeDestruction := oldBeforeDestruction; vmt.Destroy := oldDestroy; if NOT VirtualProtect( vmt, sizeof(TVirtualMethodTable). oldProtect, oldProtect) then RaiseLastOSError; end; end;
And that’s it. This creates a TFoo using a known, reliable no-op constructor and effectively suppresses all possible derived construction and destructor behaviour – that’s fine because we know that the constructor we introduced is a no-op that requires no clean up.
One thing to be VERY careful of however is that this technique is absolutely NOT thread-safe and cannot, as far as I can see, easily be made so.
Epilogue / Footnote
After devising this solution it was subsequently decided that “fixing” the destructors in the class hierarchy was a desirable change in and of itself, and this in turn would allow a temporary object to be created without having to patch the VMT after all.
The fixing of destructors in this case simply involved making them resilient to only partially initialised objects. So a destructor containing code similar to this:
destructor TSomeFooClass.Destroy; begin fList.Clear; fList.Free; inherited; end;
would be changed to:
destructor TSomeFooClass.Destroy; begin if Assigned(fList) then begin fList.Clear; FreeAndNIL(fList); end; inherited; end;
Which to my mind is the correct way to implement a destructor anyway (and yes I know lists do not ordinarily need to be explicitly Clear’d before Free’ing, but you get the idea).
Thankfully, having had the fun of figuring out the VMT patch (a few minutes work all told) the rather less rewarding task of fixing all the destructors fell to someone else.