Typechecker & requiring native modules

[petrihakkinen]

Hi! We have a bunch of native (C++) modules that are optionally required by scripts. How could I register them to the typechecker?

For example, in a script we could have something like this:

local foo = require "foo"
foo.do_something("xyzzy")

Type definition file for module 'foo':

-- This probably isn't right since 'foo' is not really a global...
declare foo: {
	do_something: (name: string) -> ()
}

I've tried injecting the type definition file into frontend.moduleResolver.modules map but when the script tries to require "foo", the typechecker raises this error: Cannot require module foo: Module is not a ModuleScript. It cannot be required.

I checked the code in TypeChecker::checkRequire and it leads me to think only Lua source code modules are supported. Am I out of luck?

[JohnnyMorganz]

it leads me to think only Lua source code modules are supported

I believe this is correct - definition files are mainly there to be loaded into the global scope. Not sure if they can be tied to a particular module.

Although, you may still be able to get somewhere. This is how I would think to do it:

When you create a Frontend, you specify a FileResolver: https://github.com/Roblox/luau/blob/master/Analysis/include/Luau/FileResolver.h

For this, there are 2 key methods:

• resolveModule
• readSource

resolveModule is given the expression passed to require(). In this case, an AstExprConstantString. You need to map this into a managed file name (by managed I mean just a representation that you understand/control, e.g. can be just foo or maybe @foo to highlight its builtin etc.)

readSource is given a managed file name and returns some Lua source code (which is why I also think Lua source code is only possible right now).

Here, if we are given the managed file name @foo, we can return some built-in source code like this:

return {} :: {
     do_something: (name: string) -> ()
}

(and you can also export any type defs as necessary)

Put together into something like

static const std::string kBuiltinFooModule = R"BUILTIN_SRC(
--!strict
export type Value = string
return {} :: {
     do_something: (name: string) -> ()
}
)BUILTIN_SRC";

struct BuiltinsFileResolver : Luau::FileResolver {
    std::optional<SourceCode> readSource(const ModuleName& name)
    {
        if (name == "@foo")
            return Luau::SourceCode{kBuiltinFooModule, Luau::SourceCode::Module};
        return std::nullopt
    }

    std::optional<ModuleInfo> resolveModule(const ModuleInfo* context, AstExpr* expr)
    {
        if (Luau::AstExprConstantString* expr = node->as<Luau::AstExprConstantString>())
        {
            if (expr->value.data == "foo") 
                return Luau::ModuleInfo{"@foo"};
        }
        return std::nullopt;
    }
}

Its no longer really using type definitions syntax, just general Luau code.
Maybe there is a better way, but this looks like it could work to me

[JohnnyMorganz]

Actually, I just had another look, and we may be able to still use type definitions here.
Instead of a FileResolver, it looks like there is a ModuleResolver which is more suited.

If we create our own Module, and use loadDefinitionFile to load the definitions into the module root scope, then the ModuleResolver resolves to our created module.

Haven't look too deeply though, maybe you might find something

[petrihakkinen]

Many thanks! I'll dig deeper and see what I find...

[petrihakkinen]

I couldn't get the second approach working for some reason (load definitions into the module scope). Load definitions passes without errors but the type checker generates type errors for every call to the module as if the functions wouldn't exist. I think what is missing is some glue that tells that the type definitions belong to the module table.

Anyway, the first approach (embedding type annotations after "return {} ::") does the trick. I'll go with that for now, but I'll probably revisit this down the road to see if there's a cleaner way.