It's been a while. Maybe you noticed! I've been working on various odds and ends, and my focus has shifted around a bit. I'll probably talk about this more in the future, but the short version is: Yes, I am still working on Geas but it may be taking a bit of a backseat for a while, and no, I'm not dead, nor have I just stopped doing anything worthwhile in my spare time.
I've been working on a project using LLVM lately. I got it compiling on Windows, I got it linked into my project, and I even got a little LLVM JIT function built and running. It takes in an int32 and returns that number + 1. Hooray.
Problems cropped up with the next step: Making it possible for the VM to call out to my external C functions. In all of the tutorials I could find that discussed doing this, it seems like they declared a function prototype for an external function, and then it just worked. I am here to tell you that, in my environment, developing using Visual C++ on Windows, it does not just work.
After a fair bit of beating my head against various walls and tearing out most of the hair that I have left, I finally figured out how to make it work. I want to record it here so that if anyone else runs into this problem, maybe Google will direct them here and I can help them out.
Here's the important C code...
C++ code GenericValue gv; std::vectorargs; FunctionType *ft = FunctionType::get(Type::getVoidTy(context), false); Function *g = Function::Create(ft, Function::ExternalLinkage, Twine("testCall"), module); gv = exec->runFunction(g, args);
... and what LLVM tells me was the content of the module ...
LLVM Module output ; ModuleID = 'egg' declare void @testCall()
... which looks fine. But that exec->runFunction call would crash and burn with the following error message:
LLVM ERROR: Tried to execute an unknown external function: void ()* testCall
It's probably worthwhile to note here that this same thing happened with the version of Kaleidoscope, the toy language from the LLVM tutorials, but only if I tried to import one of the functions defined in the Kaleidoscope code, like putchard() (or so it seemed). I could import say, sin() just fine.
Now, it looks like the reason this works for others is because they have libffi installed, a fancy bit of code that lets you introspect on your compiled code to find other function names at runtime. That sounds great, but let's just say that trying to get it to compile on Windows to work with Visual Studio was not great. So I figured that there must be some way to do this without using ffi, since it seemed to have conditional compile paths for if it wasn't there.
And there is! It's a good thing LLVM is open source — I didn't find the answer anywhere in documentation. I found it by delving through the source code, tracking down the error I was getting, and tracing execution around. Good times.
Anyway, the upshot is that I discovered two important things:
- LLVM actually inserts some common library functions (sin, printf, etc.) into your externals for you, no matter what. This is why Kaleidoscope was able to extern in things like sin() but not putchard().
- To manually insert a symbol into LLVM's interpreter, you need to use sys::DynamicLibrary::AddSymbol(), and you need to write a wrapper function.
I saw lots of mention of wrapper functions around but I couldn't see any sort of explanation. Basically, you have a function with a signature like
GenericValue lle_X_FUNCTIONNAME(FunctionType *ft, const std::vector&args)
with FUNCTIONNAME replaced appropriately (say, lle_X_testCall). In that function, as you can see, you get a FunctionType descriptor so you can tell which version of the function's been called, and a vector of arguments. You can do whatever you want with these; you're back in C at this point so the world is your oyster!
With your wrapper in place, you add it to the DynamicLibrary symbol table
sys::DynamicLibrary::AddSymbol("lle_X_FUNCTIONNAME", (void *)lle_X_FUNCTIONNAME);
and you're good to go. Make sure you've included "llvm/System/DynamicLibrary.h". When you call a function, it will search this symbol table, and find your function, and then call it. And you, if you're like me, will lean back, let out a long-suffering sigh, and whisper "finally... finally" to the ceiling.
Code is from my project, hasn't been tested in this form, so YMMV. #include "llvm/LLVMContext.h" #include "llvm/Module.h" #include "llvm/DerivedTypes.h" #include "llvm/Constants.h" #include "llvm/Instructions.h" #include "llvm/ModuleProvider.h" #include "llvm/Analysis/Verifier.h" #include "llvm/ExecutionEngine/ExecutionEngine.h" #include "llvm/ExecutionEngine/Interpreter.h" #include "llvm/ExecutionEngine/JIT.h" #include "llvm/ExecutionEngine/GenericValue.h" #include "llvm/Support/raw_ostream.h" #include "llvm/System/DynamicLibrary.h" using namespace llvm; GenericValue lle_X_testCall(FunctionType *ft, const std::vector&args) { Log::debug("testCall wrapper has executed."); GenericValue gv; gv.IntVal = 0; return gv; } int main(int argc, char **argv) { LLVMContext context; Module *module = new Module("egg", context); ExecutionEngine *exec = EngineBuilder(module).create(); FunctionType *ft = FunctionType::get(Type::getVoidTy(context), false); Function *g = Function::Create(ft, Function::ExternalLinkage, Twine("testCall"), module); sys::DynamicLibrary::AddSymbol("lle_X_testCall", (void *)lle_X_testCall); GenericValue gv; std::vector args; args.clear(); gv = exec->runFunction(g, args); }
Anyway, I've written this just after figuring this all out so I could make sure that something was out there. There are probably a lot of improvements to be made on the code above. Really I'm just hoping that if you're fighting with the same problem, this will point you in the right direction. Good luck!
