Hooking Metal for rendering in C++

Metal is a rendering framework provided by Apple which is used on MacOS and iOS. We can hook metal to draw things on top of a game or any program using Metal for rendering.

To do this, we need to understand how Metal draws things to the screen. A MTLDevice is a GPU object that allows you to setup other Metal objects. If the device has one GPU, it will return the exact same value every time MTLCreateSystemDefaultDevice is called.

After we get a handle to the GPU, we can create a MTLCommandBuffer which stores encoded commands that the GPU will render when it is commited.

Another essential component to rendering with Metal is a renderPassDescriptor. A renderPassDescriptor object allows you to add attachments to each render pass such as colorAttachments, depthAttachments, and stencilAttachments, which tell the GPU how to render different objects.

As stated in the title, we will be hooking Metal in C++, but how? Metal-cpp is a framework provided by Apple to call Metal from a C++ context. It’s really just a wrapper around Obj-C Metal objects.

Hooking

In a perfect world, we could hook objc_msgSend and if ‘op’ is equal to the method for creating the commandBuffer and RenderPassDescriptor. For example,

MTL::Device* device = MTL::CreateSystemDefaultDevice();
MTL::RenderPassDescriptor* descriptor;
MTL::CommandBuffer* buffer;

void* hooked_objc_msgSend(void* self, const char* op, ...) {
    if (strstr(op, "commandBuffer") != nullptr) {
        buffer = reinterpret_cast<MTL::CommandBuffer*>(self);
    }
    if (strstr(op, "RenderPassDescriptor") != nullptr) {
        descriptor = reinterpret_cast<MTL::RenderPassDescriptor*>(self);
    }
    return original_objc_msgSend(self, op, ...);
}

Unfortunately, hooking objc_msgSend is hard.

Drawing

Written on October 2, 2022