As someone who's working on a game engine that's designed to be highly data-driven, with only the core game engine itself in a systems programming language and a lot of calls out to scripts, this is very intriguing! C++20 doesn't seem like a good candidate for an actual scripting language, so I'd have to find a suitable scripting language that can compile to RISC-V, though, and of course requiring precompilation is an issue.
Yes, precompilation is not optional. I've used many languages in sandboxes over the years developing both this and other emulators.
I really like Nim the most. It's Python spiritually, but with types. Types just really really necessary, and it does have FFI support so you can forward arguments either from a wrapper or use {.cdecl} directly. It's still not a barneskirenn (as we say in Norway), but definitely the most fun I've had using another language in my various sandboxes.
Nelua and Zig are close seconds. Both are just so easy to work with C-based FFI. Nelua is probably not safe to use, as it's still a work in progress. At least last I checked. Zig is very much ready to use. Maybe not your cup of tea, though.
Golang has a complex run-time and I don't recommend using it. It's definitely possible though as my sandbox does have a full MMU. Just expect integration to be long and ardous. And the ABI changes potentially every version.
Rust is one of the easier ones. I didn't find it fun to work with though. It has by far the best inline assembly, but too much fighting with the compiler. I know that people love Rust, and it is fully supported in my emulator.
C/C++ has the benefit of having the ability to have their underlying functions overridden by native helper system calls. Eg. replacing memcpy() with a system call that has native performance. It's too long a long topic to talk about here, but Nim and Nelua also falls under this umbrella along with other languages that can compile to C/C++.
Kotlin is definitely possible to use. A bit hard to understand how the native stuff actually works, but I did manage to run a hello world program in several sandboxes with some effort. I'm not 100% sure but I think I managed to convert a C API header directly to something kotlin understands using a one-liner in the terminal. I would say it scores high just on that. Again, just a bit hard to understand how to talk to Kotlin from an external FFI looking in.
JavaScript is of course possible with both jitless v8 and QuickJS. v8 requires writing some C++ scaffolding + host API functions, and QuickJS requires the same in C. Either works.
And I think that's all the languages I have tried. If you think there's a missing language here, then I would love to try it!
This is a really interesting list, thank you for replying!
(the following are entirely undirected musings on using your technology for my game engine, mostly in case anyone finds them interesting or has something to suggest that I haven't thought of since I'm new at this)
At least for my use case C, Zig, Rust, and even probably Nim are out of the question because I'm not just using scripting for sandboxing capabilities and such, I'm also using it because I want people to be able to program games and write mods in a high-level language, so using a systems programming language as my scripting language kind of feels like it defeats part of the purpose and I might as well just use dynamic linking with a C ABI or something crazy like that (I'm new to this so excuse me if that's nonsense lol). JavaScript and Kotlin are intriguing though, because they aren't systems programming languages, so I'll have to think long and hard about those;
I've been considering C# for scripting lately (because I like it well enough, it's widespread, and known in the gaming world) and I wonder if you can get natively aot compiled Kotlin to work, if you could get natively aot compiled C# to work too... there would probably be similar complications due to the large and complex runtime, but I know you can strip it down so I wonder how that might play in. I also wonder what the performance would be like compared to just hosting the dotnet runtime, which is what I was intending to do before I saw your post. Maybe at some point I should set up a benchmark to compare! Although I have to say the pre-compilation thing is a bit of a deal-breaker for me, since I really want people to be able to put plain text things directly in the game folder and see those changes in the engine without having to do any kind of build process, which is something that is achievable with the regular dotnet runtime using a precompiled assembly bootstraps the rest of the sctipts using dynamic assembly compilation and loading to collect everything else in the script directory.
Hm, if you don't actually need a sandbox then I think just using the C# run-time makes a lot of sense. I also like C#, but I've never been in a situation to try the fairly new AOT support. Sounds like a good idea, though. C# is a very good language.
Do you mean compiling the .NET runtime (CoreCLR) itself or AOT compiling C# code? Because I recently tried AOT compiled C# and that builds quickly. And you don't generally need to compile the .NET runtime when embedding it because you can just load the builds they ship.
Simply running a DotNET dll. I just tried a basic "Hello World" program compiled to a CIL .dll and to RV64 machine code, on my x86 Linux machine (original ThreadRipper 2990WX) and then running it through different JIT/interpreters. Wall time:
DotNET: 0.061s
qemu-riscv64: 0.006s
Spike: 0.031s
Qemu is JIT with Linux syscall layer built in (in native code).
Spike is a RISC-V interpreter with Linux syscall layer provided by interpreted RISC-V "pk"
So the DotNET JIT has quite a high overhead for startup, or one-time code.
DotNET beats emulated RISC-V here, but the RISC-V emulator (with RISC-V code compiled with -O1) is pretty much as fast as a lazy person compiling C to native x86 gets.
Good to know. That solidifies my plan to embed the dotnet runtime in my engine and load JIT code, not compile C# to AOT and then run that through an emulation layer
