hlua is a Rust wrapper for the Lua programming language which aims to provide a safe interface with this language with minimal overhead. I have a spent a lot of time figuring out how to do that, so I have decided to write about some parts of its design.
Introduction to the Lua stack
Lua is a programming language, and one of its goals is to be easy to embed in another language. Executing some code is as easy as creating a new Lua environment with luaL_newstate and calling luaL_do_string. But executing Lua code in itself is not very interesting if it can’t interact with Rust.
Interacting with Rust usually means calling Rust functions from within Lua, but for this article we’ll just focus on reading and writing global variables.
Let’s say for example that we execute some Lua code that modifies a global variable:
// initializes the environment
let lua = luaL_newstate();// parse and execute some string
luaL_do_string(lua, “a = 5;”);
// this is pseudo-code ^ ; in reality we need to pass a CString
After luaL_do_string returns, the script has finished executing and the global variable a now contains 5. In order to read this value in Rust, the Lua library’s API provides us with a stack, and functions to manipulate this stack. In this situation, we need to tell Lua to push the value of the variable a on the stack, then we investigate the content of the stack, and then we pop the value.
// push the value of `a` on the stack
lua_getglobal(lua, “a”);// get the value ; `-1` means the “the top element”
let value = lua_tonumber(lua, -1);lua_pop(lua, 1);
Note that calling lua_tonumber supposes that the value is a number or of a type convertible to a number. In a real-life situation, we would need to check the type first.
This code looks pretty simple, doesn’t it? But how would we do this if a was an array?
To load a specific element of an array, we first need to load a handle to the array on the stack (like we did above), then push the key that we want to read, then ask Lua to load the corresponding value. For example, if we wanted to read the element 1 of a:
lua_getglobal(lua, “a”);
lua_pushnumber(lua, 1);
lua_gettable(lua, -2);
// ^ again, this supposes that `a` is indeed an array// the key has been poped
// the stack now contains the array and the valuelet value = lua_tonumber(lua, -1);lua_pop(lua, 2); // we pop both the value and the array
Another situation: in order to iterate over an array we need to push the null key and call lua_next, which pops the key and pushes a key-value combination. We then need to read the value, pop it, and call lua_next again to return the key right after the one that is still on the stack. If lua_next returns 0, we have finished iterating.
lua_getglobal(lua, “a”);
lua_pushnil(lua);while lua_next(lua, -2) != 0 {
// the stack now contains the array, the key and the value
let key = lua_tonumber(lua, -2)
let value = lua_tonumber(lua, -1);
lua_pop(lua, 1);
// the stack now contains the array and the key
// we are ready for the next iteration
}
As you can imagine, one of the challenges of writing clean Lua code is to manage the stack correctly. Some Lua functions push values, some others pop values, and the programmers must not make any mistake. To make these operations safer this where Rust comes into play.
The Rust wrapper
In order to wrap around all these operations in Rust, we are going to create a struct named Lua with a basic constructor and destructor:
pub struct Lua {
context: *mut lua_State,
}impl Lua {
pub fn new() -> Lua {
Lua {
context: luaL_newstate(), // skipping "unsafe" for clarity
}
}
}impl Drop for Lua {
fn drop(&mut self) {
lua_close(self.context);
}
}
As a start, one of the methods we could add to this struct is get_global_number, similar to our first example above:
impl Lua {
pub fn get_global_number(&self, name: &str) -> i32 {
lua_getglobal(self.context, name);
let value = lua_tonumber(self.context, -1);
lua_pop(self.context, 1); value
}
}
This naive approach has a problem: if the user shares a &Lua between multiple threads and some of them call get_global_number simultaneously, we have a race condition.
The best solution to this is to require a &mut self instead of &self, and the user can use a Mutex<Lua> if necessary. After all this function does indeed modify something: the stack. An alternative would have been to mark Lua as !Sync, but that would have been too restrictive.
Arrays
Thread-safety issues aside, the question that you probably now have in mind is: how to handle arrays elegantly? First, how do we load a single value out of an array? With the Lua API, we need to load the array with lua_getglobal, push the key, load the value, get the value, and then clean the stack.
Surely we could write a function named get_array similar to get_global_number that does all the necessary operations and then returns the value. But that would be inefficient, as multiple calls to get_array would result in unnecessary calls to lua_getglobal.
Instead we need to use two steps: make the user load the array, then make the user load an element from the array:
pub struct LuaTable<’a> {
lua: &’a mut Lua,
}impl Lua {
pub fn load_array(&mut self, name: &str) -> LuaTable {
lua_getglobal(self.context, name);
assert!(lua_istable(self.context, -1));
LuaTable { lua: self }
}
}impl<’a> LuaTable<’a> {
fn get_number(&mut self, key: i32) {
lua_pushnumber(self.lua.context, key);
let value = lua_gettable(self.lua.context, -2);
lua_pop(self.lua.context, 1);
}
}impl<’a> Drop for LuaTable<’a> {
fn drop(&mut self) {
lua_pop(self.lua.context, 1);
}
}
Usage:
let mut table_access = lua.load_array(“a”);
let value = table_access.get_number(1);
drop(table_access);The LuaTable object serves as a guard. As long as it is alive, the array is at the top of the stack. Methods on the LuaTable can thus directly read its content.
Similarly, to iterate over the table we can create a LuaTableIterator struct. As long as the LuaTableIterator is alive, we assume that the table and the next key are on the top of the stack. The code would be too long to show, but I think that you get the principle.
Being more generic
Before going forward, we need to make some changes to our wrapper. Having functions named get_global_number, load_array or get_number is not very elegant. Instead we’re going to write a trait named Read that represents types that can be loaded from a Lua environment.
Some of the implementations of this trait will need to keep the Lua object borrowed, so in order to have correct lifetimes we have to use a little trick where we add a lifetime parameter to the trait itself.
pub trait Read<’a> {
fn read(lua: &’a mut Lua, index: i32) -> Result<Self, ReadError>;
}impl<’a> Read<’a> for i32 {
fn read(lua: &’a mut Lua, index: i32) -> Result<i32, ReadError> {
if lua_isnumber(lua.context, index) {
let value = lua_tonumber(lua.context, index);
lua_pop(lua.context, 1);
Ok(value)
} else {
lua_pop(lua.context, 1);
Err(ReadError::WrongType)
}
}
}impl<’a> Read<’a> for LuaTable<’a> {
fn read(lua: &’a mut Lua, index: i32)
-> Result<LuaTable<’a>, ReadError>
{
if lua_istable(lua, index) {
Ok(LuaTable { lua: lua })
} else {
lua_pop(lua.context, 1);
Err(ReadError::WrongType)
}
}
}impl<’a> Read<’a> for i8 { … }
impl<’a> Read<’a> for i16 { … }
impl<’a> Read<’a> for String { … }
impl<’a> Read<’a> for bool { … }
…
The read functions reads the value at the given location. Sometimes it keeps the Lua object borrowed (with LuaTable), sometimes not (i32, bool, etc.). In all situations, it pops the value afterwards.
We can now replace get_global_number by a single get function:
impl Lua {
pub fn get<T>(&mut self, name: &str) -> Result<T, ReadError>
where T: Read
{
lua_getglobal(self.context, name);
Read::read(self.context, -1)
}
}This function can now be used for both numbers and tables:
let a: i32 = lua.get("a");
let b: LuaTable = lua.get("b");Arrays over arrays
But wait… this works for the get function of the Lua struct, but our LuaTable struct also needs a get function.
If we wanted to read an array inside an array, we would need to pass a &’a mut Lua to the inner LuaTable. Since we can only have one mutable borrow of Lua at a time, we would need to destroy the outer LuaTable first, and this is not what we want.
Instead we want the inner LuaTable to mutably borrow the outer LuaTable, so that we regain access to the outer table once the inner table is poped from the stack.
This means that we will need to be generic over the Lua context by creating a trait named AsMutLua. Objects that implement this trait represent an access to a Lua context.
pub trait AsMutLua {
unsafe fn as_mut_lua(&mut self) -> *mut lua_State;
}impl<'a> AsMutLua for &'a mut Lua {
unsafe fn as_mut_lua(&mut self) -> *mut lua_State {
self.context
}
}pub trait Read<T> where T: AsMutLua {
fn read(lua: T, index: i32) -> Result<Self, ReadError>;
}
Notice that instead of having Read<’a, T> we have Read<T>, where T is for example a &’a mut Lua. When you have the choice, it is usually more convenient this way. It will also serve us in a moment.
We can now update the rest of the code:
impl Lua {
pub fn get<’a, T>(&’a mut self, name: &str)
-> Result<T, ReadError>
where T: Read<&’a mut Lua>
{
let ctxt = self.context;
lua_getglobal(ctxt, name);
Read::read(self, -1)
}
}pub struct LuaTable<T> where T: AsMutLua {
lua: T,
}impl<'a, T> AsMutLua for &'a mut LuaTable<T> where T: AsMutLua {
unsafe fn as_mut_lua(&mut self) -> *mut lua_State {
self.lua.as_mut_lua()
}
}impl<T> LuaTable<T> where T: AsMutLua {
pub fn get<’a, U>(&’a mut self, key: i32)
-> Result<U, ReadError>
where T: Read<&’a mut LuaTable<T>>
{
let ctxt = self.lua.as_mut_lua();
lua_pushnumber(ctxt, key);
lua_gettable(ctxt, -2);
Read::read(self, -1)
}
}impl<T> Drop for LuaTable<T> where T: AsMutLua {
fn drop(&mut self) {
lua_pop(self.lua.as_mut_lua(), 1);
}
}
When the user calls lua.get(“table”), the returned type is a LuaTable<&’a mut Lua>. The Lua object continues to be borrowed, preventing anything else from messing with our stack. The user can then call table.get(“subtable”), which will return a LuaTable<&’a mut LuaTable<&’b mut Lua>>.
Push guard
But wait, there’s another problem with this whole design. When iterating over a table for example, we want to read the key but not pop it, despite what our Read trait does. In some situations we want to read-and-pop, and in other situations we only want to read.
We could pass a flag to the read function to specify whether to pop the value after it is no longer needed, but there is a better design: using a PushGuard, whose purpose is to pop the value on destruction.
We are going to de-couple reading and poping. If we want to just read an array without poping it, we are going to create a LuaTable<&mut Lua>. Instead if we want to read an array then pop it, we are going to create a LuaTable<PushGuard<&mut Lua>>.
pub struct PushGuard<T> where T: AsMutLua {
lua: T,
size: i32,
}impl<T> Drop for PushGuard<T> where T: AsMutLua {
fn drop(&mut self) {
if self.size != 0 {
lua_pop(self.lua.as_mut_lua(), self.size);
}
}
}impl<T> AsMutLua for PushGuard<T> where T: AsMutLua {
fn as_mut_lua(&mut self) -> *mut lua_State {
self.lua.as_mut_lua()
}
}
Our get function can now be written like this:
impl Lua {
pub fn get<’a, T>(&’a mut self, name: &str)
-> Result<T, ReadError>
where T: Read<PushGuard<&’a mut Lua>>
{
let ctxt = self.context;
lua_getglobal(ctxt, name);
let guard = PushGuard { lua: self, size: 1 };
Read::read(guard, -1)
}
}To sum things up, here is what happens when you call get and expect a table:
- We load the value that we want to examine on the stack.
- We create a PushGuard that corresponds to this value.
- We check that the value really is a table and build a LuaTable.
- Let’s say that we want to examine a sub-table. We load this sub-table on the stack over the outer table.
- We create a PushGuard that corresponds to this sub-table.
- We check that the sub-table really is a table a build a LuaTable.
- This LuaTable is destroyed. As it had ownership of the PushGuard, this PushGuard is destroyed too and the subtable is poped.
- The outer LuaTable is destroyed too. Again, the PushGuard is destroyed and the table is poped. The stack is clean again and the Lua is unborrowed.
Conclusion
With a clever usage of traits and generics, the Rust programming language allows you to write an abstraction over Lua that is almost free and remains safe at the same time, thanks to RAII.
This is just an overview of the design of hlua. I didn’t talk about things such as callbacks, writing userdata, writing/reading tuples, etc. which also make use of some semi-obscure features of Rust.
Unfortunately several things are still missing in the Rust language, like HKTs, before hlua can be made cleaner in all domains.
