Caution: The contents of this post may cause dizziness or nausea.  Take only as prescribed and if symptoms persist seek professional advice.

Recently I found myself needing to do something I had never done before – create an instance of an arbitrary class derived from some base class and destroy it, and do so without invoking any constructor or destructor code that the derived class(es) may have introduced. This is the difficult bit.

Impossible?  Don’t be silly, this is Delphi we’re talking about….

Method In The Madness

To hopefully explain (do I mean “justify”?) this madness, a bit of background to the initial problem may be useful.

The need that I had for this insanity stemmed (stum?) from a perfectly reasonable limitation imposed on code in class methods – they cannot reference normal instance methods.  By which I don’t mean call them – they cannot even reference them (directly at least).

This is because a method reference, as previously discovered in the Multicast Events series, is actually a pair of pointers – a pointer to an instance and the method entry point.  A class method – by definition – does not have an instance reference so a valid TMethod reference cannot be formed by the compiler.

So this will not compile:

  type
    TVariantMethod = function: Variant of object;
    TObjectMethod = procedure(var aObject: TObject) of object;

    TFoo = class(TBase)
    private
      function FuncA: Variant;
      procedure ProcA(var aObject: TObject);
    protected
      class procedure AddName(const aName: String;
                              aMethod: TVariantMethod); overload;
      class procedure AddName(const aName: String;
                              aMethod: TObjectMethod); overload;
      class procedure RegisterMethods; virtual;
    end;

    class procedure TFoo.RegisterMethods;
    begin
      AddName('Func', FuncA); // Illegal reference to instance method
      AddName('Proc', ProcA);  // Illegal reference to instance method
    end;

In this case however, whilst I had a need to refer to a method, I was not actually concerned with any instance – a TMethod with a valid code pointer and a NIL instance would have been fine (later on when the methods were needing to be called for a specific known instance the underlying TMethod record Data field would be fixed up dynamically as required).

But as I say, this code simply will not compile – a class procedure cannot reference an instance method. Period.

So why not extract and pass a simple procedure or function pointer? i.e. a CODE pointer, without any DATA.

Firstly of course, a pointer to a “first class” function cannot be passed where a method reference (function of object) is expected – they are very different things. Ditto procedures of course.

Which left me with two possible options:

1. Untyped pointers
2. Temporary objects

Casting Type Safety To The Wind

An untyped pointer approach could have worked but is inherently unsafe (patching the VMT isn’t?! [Ed]) and results in a more cumbersome implementation overall:

  type
    TFoo = class(TBase)
      :
    protected
      class procedure AddFunctionName(const aName: String;
                                      aMethodAddr: Pointer);
      class procedure AddObjectName(const aName: String;
                                    aMethodAddr: Pointer);
      class procedure RegisterMethods; virtual;
    end;

    class procedure TFoo.RegisterMethods;
    begin
      AddFunctionName('Func', @TFoo.FuncA);
      AddObjectName('Proc', @TFoo.ProcA);
    end;

In this approach the type safety of the two different method types is lost completely. This means not only that separate methods are now required to register the two different types of method, but also a mistake could very easily be made and not picked up by the compiler.

i.e. a pointer to a function returning a variant could be passed to the AddObjectName() method, with dire consequences at runtime when the function returning a Variant is called as if it were a procedure yielding a TObject reference in a var parameter.

In practice there were scores of classes registering many methods each – the potential for errors to creep in was too great.

Temporary Insanity

The temporary object approach should be straight forward though:

Convert class procedures to regular instance methods and at the point at which code would normally invoke the class method instead create an instance, call the necessary methods, then dispose (i.e. Free) the temporary object:

  type
    TFoo = class(TBase)
      :
    protected
      procedure AddName(const aName: String;
                   aMethod: TVariantMethod); overload;
      procedure AddName(const aName: String;
                   aMethod: TObjectMethod); overload;
      procedure RegisterMethods; virtual;
    end;

    procedure TFoo.RegisterMethods;
    begin
      AddName('Func', FuncA);
      AddName('Proc', ProcA);
    end;

  // Class registration code:
  procedure RegisterClass(const aClass: TFooClass);
  var
    temp: TFoo;
  begin
    temp := aClass.Create;
    try
      temp.RegisterMethods;
    finally
      temp.Free;
    end;
  end;

Type safety is restored and all is well.

Just one problem. Whilst in theory this should not have presented a problem, in practice things were not quite so simple.

First, in the actual real world case, TFoo was the base class of an existing complex and extensive hierarchy of classes, many of which might be considered “legacy” code and were not – shall we say – meticulously designed. Numerous constructors were introduced at various levels in the hierarchy and of course a variety of destructors were involved. Some classes relied on AfterConstruction and BeforeDestruction overrides, and a lot of code in the constructor and destructor chains made assumptions about what was or had been initialised, and in many cases destructors were very sensitive to the correct constructor having been used etc etc.

All in all, a bit of a problem, the result of which was that many classes could not be instantiated using a common, “default” constructor in this throw-away manner – numerous errors occured, especially in the destructor chains of some classes.

Wrapping the Free in a try..except could have dealt with the obvious symptoms, but the risk of side effects was too great.

Introducing a default constructor that could be reliably used was no problem. The much bigger problem was avoiding any AfterConstruction code and the destructor chain without modifying every single piece of code in these possible methods to only execute if handling a “genuine” (i.e. not a temporary) object – a horrible intrusion of root behaviours into the hierarchy derived classes (that would also have to be reflected in any new classes or future changes to existing ones).

It was at this point that I remembered some of the secrets contained within the VMT (Virtual Method Table).

Here’s One I Prepared Earlier

It just so happened that in developing my SmokeTest testing framework I had implemented an exposure of the VMT, drawing on the work of Ray Lischner in “Delphi In A Nutshell” (every Delphi developer should have at least one copy of this on their book shelf) and the blog of Hallvard Vassbotn.

At the time I was only interested in the published method information accessible via the VMT, but the VMT contains more than this, as Hallvard Vassbotn explains very nicely.

The key is the fact that the entry points for ALL the problem areas affecting my temporary object approach are contained in the VMT – AfterConstruction, BeforeDestruction and the Destroy method itself, are all there in the VMT.

If the VMT contains these entry points then if I modify the VMT, could I not in fact “patch” these fundamental class behaviours at runtime?  As long as I restored the VMT to it’s original state once I was done with it no-one would be any the wiser….

  constructor TFoo.CreateTemporary;
  begin
    // NO-OP
  end;

  
  procedure RegisterClass(const aClass: TFooClass);
  var
    vmt: PVirtualMethodTable;
    oldAfterConstruction: PAfterConstruction;
    oldBeforeDestruction: PBeforeDestruction;
    oldDestroy: PDestroy;
    temp: TFoo;
  begin
    vmt := GetVirtualMethodTable(aClass);

    oldAfterConstruction := vmt.AfterConstruction;
    oldBeforeDestruction := vmt.BeforeDestruction;
    oldDestroy := vmt.Destroy;

    vmt.AfterConstruction := NIL;
    vmt.BeforeDestruction := NIL;
    vmt.Destroy := NIL;
    try
      temp := aClass.CreateTemporary;
      try
        temp.RegisterMethods;
      finally
        temp.Free;
      end;
    finally
      vmt.AfterConstruction := oldAfterConstruction;
      vmt.BeforeDestruction := oldBeforeDestruction;
      vmt.Destroy := oldDestroy;
    end;
  end;

The pointer types referenced in this code and the virtual method table record type are as described in Hallvard Vossbotn’s article.

This compiles just fine but will blow up at runtime with an access violation on the first attempt to modify a member of the vmt record:

  vmt.AfterConstruction := NIL; // BOOM! 

How is that possible? This can’t be an invalid memory address and can’t be outside our process. Why would reading from this address be OK, but writing cause an access violation?

This had me scratching my head for a while until I stumbled across what looks like some GExperts code in the Koders database. This shows how to patch a user defined virtual method (and also how to change the parent class of a class!).

The key of course is the toggling of protection flags on the memory occupied by the VMT record. There is also a subtle requirement that I initially missed. The AfterConstruction, BeforeDestruction and Destroy pointers cannot in fact be NIL – the runtime will automatically call the methods that these fields point to, so they have to point to something; some no-op/null stubs are required.

So one last refinement and we’re done:

  // NOTE: These are regular procedures, NOT "of object" !!
  procedure NullAfterConstruction(aObject: TObject);
  begin end;

  procedure NullBeforeDestruction(aObject: TObject); 
  begin end;

  procedure NullDestroy(aObject: TObject; aOutermost: SmallInt); 
  begin end;


  procedure RegisterClass(const aClass: TFooClass);
  var
    vmt: PVirtualMethodTable;
    oldAfterConstruction: PAfterConstruction;
    oldBeforeDestruction: PBeforeDestruction;
    oldDestroy: PDestroy;
    oldProtect: DWord;
    temp: TFoo;
  begin
    vmt := GetVirtualMethodTable(aClass);
    oldAfterConstruction := vmt.AfterConstruction;
    oldBeforeDestruction := vmt.BeforeDestruction;
    oldDestroy := vmt.Destroy;

    if NOT VirtualProtect( vmt, sizeof(TVirtualMethodTable). PAGE_READWRITE, oldProtect) then
      RaiseLastOSError;

    vmt.AfterConstruction := NullAfterConstruction;
    vmt.BeforeDestruction := NullBeforeDestruction;
    vmt.Destroy := NullDestroy;
    try
      temp := aClass.CreateTemporary;
      try
        temp.RegisterMethods;
      finally
        temp.Free;
      end;
    finally
      vmt.AfterConstruction := oldAfterConstruction;
      vmt.BeforeDestruction := oldBeforeDestruction;
      vmt.Destroy := oldDestroy;

      if NOT VirtualProtect( vmt, sizeof(TVirtualMethodTable). oldProtect, oldProtect) then
        RaiseLastOSError;
    end;
  end;

And that’s it. This creates a TFoo using a known, reliable no-op constructor and effectively suppresses all possible derived construction and destructor behaviour – that’s fine because we know that the constructor we introduced is a no-op that requires no clean up.

One thing to be VERY careful of however is that this technique is absolutely NOT thread-safe and cannot, as far as I can see, easily be made so.

Epilogue / Footnote

After devising this solution it was subsequently decided that “fixing” the destructors in the class hierarchy was a desirable change in and of itself, and this in turn would allow a temporary object to be created without having to patch the VMT after all.

The fixing of destructors in this case simply involved making them resilient to only partially initialised objects. So a destructor containing code similar to this:

  destructor TSomeFooClass.Destroy;
  begin
    fList.Clear;
    fList.Free;
    inherited;
  end;

would be changed to:

  destructor TSomeFooClass.Destroy;
  begin
    if Assigned(fList) then
    begin
      fList.Clear;
      FreeAndNIL(fList);
    end;

    inherited;
  end;

Which to my mind is the correct way to implement a destructor anyway (and yes I know lists do not ordinarily need to be explicitly Clear’d before Free’ing, but you get the idea).

Thankfully, having had the fun of figuring out the VMT patch (a few minutes work all told) the rather less rewarding task of fixing all the destructors fell to someone else.

🙂

10 thoughts on “Patching A Class VMT

  1. How about abusing nil?

    class procedure TFoo.RegisterMethods;
    var
    foo: TFoo;
    begin
    foo := nil;
    AddName(‘Func’, foo.FuncA);
    AddName(‘Proc’, foo.ProcA);
    end;

  2. Good idea Uli! – in fact never mind NIL, just an uninitialised “foo” is enough to hoodwink the compiler since we aren’t using the data pointer at all (but does result in a pesky “uninitialised” warning – darn).

    Admittedly it’s a much less “dirty” hack, but does suffer a little from having to be liberally sprinkled throughout the entire class hierarchy and adding potentially confusing “noise” to the registration methods (in month/years hence some one will wonder what on earth is going on).

    The VMT patch did at least have the saving grace of being highly localised (in just the one method) and allowing the registration methods to be written most intuitively, without having to jump through hoops to coerce the compiler.

    (and, as it turned out of course, the patch ultimately wasn’t even necessary, nor did we need to strong-arm the compiler. just fix our own destructors!)

    In any event, it wouldn’t have been half as much fun as figuring out how to patch the VMT.

    🙂

  3. > In any event, it wouldn’t have been half as much fun as figuring out how to
    > patch the VMT.
    I know. 🙂

  4. The VMT patch makes the code unsafe when threads are in the game.

  5. Or you could just require the methods to be published and look them up by name when you need them using the TObject.MethodAddress

    Waaaaay simpler, and even threadsafe.

  6. @Andreas: I do make that point in the post. In our case the code involved is invoked as part of the initialization sequence of the application so even if threading is later introduced into the application as a whole, this approach would have been “safe” (for *our* case!) or easily made so (in the specific, not the general).

    But as I said, we ended up not having to use it anyway – it eventually proved to be only an interesting diversion.

    🙂

    @Xepol: That may have been how we would have done it if we were working with a clean slate.

    Unfortunately this was in the context of a long established and highly complex class hierarchy. Published visibility was already being used for other purposes in that hierarchy. Also the name by which a method was required to be known was not, and would not be, the same as the name of the method itself (and in some cases, the same method might be known by a number of different names).

    Also the scenario in the post is not 100% complete – a great deal more information was registered along with the name + method pair, information that could not be derived simply from RTTI, but that was not directly relevant to the problem so is not shown.

  7. Hi Jolyon, nice post!.

    Do you know a way to patch a property?, for example if I have this:

    published
    property FirstName: string read FFirstName write FFirstName;

    Is there a way of intercepting the setter’s “Before Write” and “After Write” of the property?

    Leonardo.

  8. Hi Leonardo,

    As far as I know there is no “Before Write” or “After Write” events for properties which are declared as directly accessing a member variable. Not in native code Delphi at least – in .NET I believe that properties always have an underlying getter and/or setter method.

    If the reader/writer are not virtual (which for property accessors they typically aren’t) then I do not think that they would be discoverable let alone modifiable in the VMT anyway.

    If they *were* virtual then I think you could “patch” the accessor methods yourself using a variation on the techniques described in this post.

    But if you have a case where the accessors are currently not virtual, and yet you are in a position to make them so, then simply adding your required code, or appropriate hooks to enable you to cleanly insert before/after change event handlers would be a much better idea than hacking around in the VMT, imho.

    Good luck.

  9. I’m wondering if you couldn’t use

    temp:= AClass.NewInstance();
    try
    temp.RegisterMethods();
    finally
    temp.FreeInstance();
    end;

    to achieve the same thing, without messing with the VMT table?

  10. You know what, I think you could. But where’s the fun in that?

    🙂

    Ultimately it was the technique that I found interesting, not necessarily the specific problem that it was initially devised to address (which it didn’t get used for in the end anyway! lol).

    You could, for example, use the technique to patch the NewInstance/FreeInstance VMT entries themselves, in order to redirect the memory management of an arbitrary class to some memory manager optimised for that class.

    e.g. if you know that you require a large number of instances of a class during some process, pre-allocate one large block of memory for all instances then serve them up by advancing a pointer into that block and freeing the memory all in one go when done.

    i.e. replacing potentially hundreds or thousands of discrete memory allocations and deallocations with one single allocation/deallocation pair.

    Actually, I think that would make an interesting research project to blog about and wouldn’t take long to knock up!

    Thanks for the inspiration. 🙂

Comments are closed.