wasm-component generate components for modules with host function imports

[CharlieTap]

Say I have an existing module like this:

(module
  (import "env" "host_function" (func $host_function (result i32)))
  (func $call_host_function (result i32) call $host_function)
  (export "call_host_function" (func $call_host_function))
)

Running the component cli I get:

wasm-tools component new host_function.wasm -o host_function_component.wasm
error: failed to encode a component from module

Caused by:
    0: failed to decode world from module
    1: module was not valid
    2: module requires an import interface named `env`

The vast majority of imports are host functions and are not provided by other modules, in these cases using the type signature of the import it should be possible to generate any worlds/interfaces needed (like env in this case)?

[alexcrichton]

The short answer is: no, it's not possible to turn this module into a component.

The longer answer is: maybe? Are you trying to take any old core module off the shelf and turn it into a component? That's almost certainly not going to work. Are you trying to take a module specifically built to be a component and turn it into a component? If so this probably means you have not configured your toolchain correctly.

[CharlieTap]

Maybe my understanding is off here, so as i currently understand the component model wraps module/s and exposes a high level interface, which is share nothing and predominantly exists to allow modules from different languages to interact.

Say I wanted to publish a library from a kotlin wasm project which is a component and can be consumed by a rust wasm project. I have an existing module that works in my kotlin ecosystem, it depends on host function (something random non wasi related). I feel like I should be able to generate a component that exposes that same module, with the same function imports as the module and publish it unchanged?

[alexcrichton]

I might recommend reading https://component-model.bytecodealliance.org/ as an introduction to the component model, there's a pretty big mismatch with what you're expecting and what's implemented today in the fundamentals. At a high level what you're describing works, but how you frame the problem matters quite a lot here and affects how things work. For example:

Say I wanted to publish a library from a kotlin wasm project which is a component

From the get-go this means that your kotlin wasm project is a component, not a core module. That means that the kotlin project was already developed to work as a component meaning that this problem described in this issue is already gone, aka there are no unknown "env" imports in the kotlin project. Instead the kotlin project will import everything it needs from the host through a component interface, not whatever "env" refers to most of the time. Using component interfaces may or may not match the existing ABI of how "env" imports work right now.

can be consumed by a rust wasm project

This is modeled as a component import in the Rust project. For example WIT is used to describe the Kotlin component's exports, and this same WIT is used to generate bindings in the Rust project to import the functions. Later a tool such as wac is used to bundle these two components into one.

I feel like I should be able to generate a component that exposes that same module, with the same function imports as the module and publish it unchanged?

You're getting into the topic of technical specifics here. At its heart, you're correct, a component has a core module inside of it. That core module is unlikely to be the literal exact same as the ones you're producing today though with "env" imports. The ABIs are different, import strings are different, etc. You could instead produce this new core module and use that with/without a component, but components are not suitable for "take any old module off the shelf and wrap it in a component", that's just not a design goal at this time.

[CharlieTap]

I realised I'm asking a couple of different things and one of them doesn't make any sense, lets stick with the one i think it still relevant 😅

You could instead produce this new core module and use that with/without a component, but components are not suitable for "take any old module off the shelf and wrap it in a component", that's just not a design goal at this time.

I guess this part is the bit that I expected, at least for modules that are language agnostic. In my head I imagine a future where we have repositories (like npm) which expose components but not modules. If I had a module which was language agnostic but required a host function, for example to get the time from wasi:clock. Would it be possible to tell the component new cli that all function imports it finds on the module are host functions which should be present on the component also (and do any wiring if needed)?

[pchickey]

In my head I imagine a future where we have repositories (like npm) which expose components but not modules.

We already support using OCI registries to distribute components, see https://github.com/bytecodealliance/wasm-pkg-tools

If I had a module which was language agnostic but required a host function, for example to get the time from wasi:clock. Would it be possible to tell the component new cli that all function imports it finds on the module are host functions which should be present on the component also (and do any wiring if needed)?

Yes, the whole idea of component new is to take a wasm module, which contains imports that just use the core wasm type system (i.e. the only atoms are i32, i64, f32, f64, and then lesser used types like externref), and wrap that module into a a component. The component has imports which can use the entire component model type system (e.g. in wasi:clocks/monotonic-clock it has instant and duration types, which are synonyms to i64, and also more sophisticated types like pollable, which is a resource type and gets represented as an i32 when passed to/from modules.

A core wasm module doesn't contain enough information in its import and export items to describe all of the types used by a component. So, the component new cli needs additional component type information. It searches for that information in custom sections that fit an expected naming pattern. For some toolchains, like Rust or C/C++, the wit-bindgen tool will use special toolchain support for custom sections in order to provide the appropriate component type information in a custom section in the module for you. When that is not the case, like if you wrote a module by hand in wat or produced it without wit-bindgen, the subcommand wasm-tools component embed <WIT> [INPUT] is provided to take all of the type information in a given wit, and embed it in a module, for consumption by wasm-tools component new.