hipFatbin - Create/Modify Fatbins from inputs

What you can do with this library:

• Create Fatbins from multiple inputs
• Add code object to a fatbin
• Remove code object from fatbins
• iterate over code objects in fatbin
• etc.

Expected Inputs

The input APIs are SPIRV and code object.

TODO: need to support compressed code objects.

Requirements

Only works on linux right now. Requires CXX20 compiler. If you want to run HIP tests, ROCm needs to be installed.

How to build

• git clone the repo
• cd hipFatbin && mkdir build && cd build
• cmake .. && make -j$(nproc)

Options

• To enable testing: -DHIPFATBIN_ENABLE_TESTING=ON
• To enable hip testing: -DHIPFATBIN_ENABLE_HIP_TESTING=ON, this requires rocm to be installed.

Examples

Create Fatbins from code objects

If you want to create fatbin from two code objects of different archs

// Assume you have two code objects
const void* code1{}, code2{};
size_t code1_size, code2_size;

// 1. Create Fatbin handle
hipFatbinHandle handle;
hipFatbinCreate(&handle, nullptr /* options */);

// 2. Add code objects to fatbin
hipFatbinAddCubin(handle, code1, code1_size, arch1 /* arch of code object */, "" /* identifier */);
hipFatbinAddCubin(handle, code2, code2_size, arch2 /* arch of code object */, "" /* identifier */);

// 3. Get size of final fatbin, this step creates fatbin
size_t fatbin_size{};
hipFatbinSize(handle, &fatbin_size);

// 4. Allocate buffer to get fatbin in that buffer
std::vector<unsigned char> fatbin(fatbin_size, 0);
hipFatbinGet(handle, fatbin.data());

// 5. Release memory
hipFatbinDestroy(handle);

Add code object to existing fatbin

If you already have a fatbin and you want to add one more arch to it

// Assume you have two code objects
const void* fatbin{}, code_object{};
size_t fatbin_size, code_object_size;

// 1. Create Fatbin handle
hipFatbinHandle handle;
hipFatbinCreate(&handle, nullptr /* options */);

// 2. Add Fatbin and code object
hipFatbinAddFatbin(handle, fatbin, fatbin_size);
hipFatbinAddCubin(handle, code_object, code_object_size, arch /* arch of code object */, "" /* identifier */);

// 3. Get final fatbin
size_t final_fatbin_size{};
hipFatbinSize(handle, &final_fatbin_size);

// 4. Allocate buffer to get fatbin in that buffer
std::vector<unsigned char> fatbin(final_fatbin_size, 0);
hipFatbinGet(handle, fatbin.data());

// 5. Release memory
hipFatbinDestroy(handle);

Remove code object from a fatbin

Remove certain arch code object from fatbin, make them smaller to load.

// Assume you have two code objects
const void* fatbin{};
size_t fatbin_size;

// 1. Create Fatbin handle
hipFatbinHandle handle;
hipFatbinCreate(&handle, nullptr /* options */);

// 2. Load Fatbin
hipFatbinAddFatbin(handle, fatbin, fatbin_size);

// 3. Remove certain arch, in this case gfx1030
hipFatbinRemoveArch(handle, "gfx1030"); 

// 4. Get final fatbin size
size_t final_fatbin_size{};
hipFatbinSize(handle, &final_fatbin_size);

// 4. Allocate buffer to get fatbin in that buffer
std::vector<unsigned char> fatbin(final_fatbin_size, 0);
hipFatbinGet(handle, fatbin.data());

// 5. Release memory
hipFatbinDestroy(handle);

What this library can not do

• It can not merge in two code objects, meaning if you have two code objects of gfx1030. We can not merge them into a single code object which now becomes part of fatbin.