Section 7.2: Ambiguous description of module resolution

[iyernaveenr]

In Chapter 7 of rustbook, Section 7.2 -> Modules Cheat Sheet -> Declaring modules:, the preference of the compiler for module declarations is clearly defined. However, when I tried experimenting with all the three ways, namely:
1 of 3) Inline, within curly brackets that replace the semicolon following mod garden
2 of 3) In the file src/garden.rs
3 of 3) In the file src/garden/mod.rs
I observed inconsistency in the compiler's behaviour when using multiple ways.

To check how these three ways might conflict with each other, I first tried 1 of 3) and 2 of 3) above. The behaviour of the compiler was to silently (as in, no error or warning) prefer 1 of 3) over 2 of 3), effectively ignoring 2 of 3). So, the compilation was successful.

However, when I tried 2 of 3) and 3 of 3), the compiler threw error[E0761], complaining about the module being found in multiple places.

Code (1 of 3) and 2 of 3))

restaurant$ tree
.
├── Cargo.lock
├── Cargo.toml
└── src
    ├── front_of_house
    │   └── hosting.rs
    ├── front_of_house.rs
    ├── lib.rs
    └── main.rs

2 directories, 6 files

restaurant$ cat Cargo.toml 
[package]
name = "restaurant"
version = "0.1.0"
edition = "2021"

[dependencies]

restaurant$ cat src/main.rs 
use restaurant::eat_at_restaurant;

fn main() {
    println!("{}:{}: Hello, world!", file!(), line!());
    eat_at_restaurant();
}

restaurant$ cat src/lib.rs 
mod front_of_house;

pub use crate::front_of_house::hosting;

pub fn eat_at_restaurant() {
    hosting::add_to_waitlist();

restaurant$ cat src/front_of_house.rs 
pub mod hosting {
    pub fn add_to_waitlist() {
        println!("{}:{}: I was called!", file!(), line!());
    }
}

restaurant$ cat src/front_of_house/hosting.rs 
pub fn add_to_waitlist() {
    println!("{}:{}: I was called!", file!(), line!());
}

restaurant$ cargo run
   Compiling restaurant v0.1.0 (/media/nibu/ext4_data/ext4_progprac/Basics/restaurant)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.16s
     Running `target/debug/restaurant`
src/main.rs:4: Hello, world!
src/front_of_house.rs:3: I was called!

Code (2 of 3) and 3 of 3))

restaurant$ tree
.
├── Cargo.lock
├── Cargo.toml
└── src
    ├── front_of_house
    │   ├── hosting
    │   │   └── mod.rs
    │   └── hosting.rs
    ├── front_of_house.rs
    ├── lib.rs
    └── main.rs

3 directories, 7 files

restaurant$ cat Cargo.toml 
[package]
name = "restaurant"
version = "0.1.0"
edition = "2021"

[dependencies]

restaurant$ cat src/main.rs 
use restaurant::eat_at_restaurant;

fn main() {
    println!("{}:{}: Hello, world!", file!(), line!());
    eat_at_restaurant();
}

restaurant$ cat src/lib.rs 
mod front_of_house;

pub use crate::front_of_house::hosting;

pub fn eat_at_restaurant() {
    hosting::add_to_waitlist();

restaurant$ cat src/front_of_house.rs 
pub mod hosting;

restaurant$ cat src/front_of_house/hosting.rs 
pub fn add_to_waitlist() {
    println!("{}:{}: I was called!", file!(), line!());
}

restaurant$ cat src/front_of_house/hosting/mod.rs 
pub fn add_to_waitlist() {
    println!("{}:{}: I was called!", file!(), line!());
}

restaurant$ cargo run
   Compiling restaurant v0.1.0 (/home/user/restaurant)
error[E0761]: file for module `hosting` found at both "src/front_of_house/hosting.rs" and "src/front_of_house/hosting/mod.rs"
 --> src/front_of_house.rs:1:1
  |
1 | pub mod hosting;
  | ^^^^^^^^^^^^^^^^
  |
  = help: delete or rename one of them to remove the ambiguity

error[E0425]: cannot find function `add_to_waitlist` in module `hosting`
 --> src/lib.rs:6:14
  |
6 |     hosting::add_to_waitlist();
  |              ^^^^^^^^^^^^^^^ not found in `hosting`

Some errors have detailed explanations: E0425, E0761.
For more information about an error, try `rustc --explain E0425`.
error: could not compile `restaurant` (lib) due to 2 previous errors

rustc --version --verbose:

rustc 1.84.0-nightly (32b17d56e 2024-10-28)
binary: rustc
commit-hash: 32b17d56eb02495f9865028e1f7271a3a48c0b9b
commit-date: 2024-10-28
host: x86_64-unknown-linux-gnu
release: 1.84.0-nightly
LLVM version: 19.1.1

cargo --version --verbose:

cargo 1.84.0-nightly (e75214ea4 2024-10-25)
release: 1.84.0-nightly
commit-hash: e75214ea4936d2f2c909a71a1237042cc0e14b07
commit-date: 2024-10-25
host: x86_64-unknown-linux-gnu
libgit2: 1.8.1 (sys:0.19.0 vendored)
libcurl: 8.9.0-DEV (sys:0.4.74+curl-8.9.0 vendored ssl:OpenSSL/3.0.2)
os: Ubuntu 22.4.0 (jammy) [64-bit]

I had originally raised this issue as an internal rust compiler issue due to the biased/inconsistent behaviour of the compiler. However, I was advised to open this issue here instead:
rust-lang/rust#139685

Thanks!

[StefanSalewski]

Are you sure that a directory structure like

    ├── front_of_house
    │   ├── hosting
    │   │   └── mod.rs
    │   └── hosting.rs

is valid and recommended in the book? My feeling was, that we can either have a mod.rs file in hosting folder, or a hostings.rs file, but not both. Are you referring to the original book, or to the one from Brown University?

[StefanSalewski]

effectively ignoring 2 of 3).

Well, if you replace the statement "mod my_module" with an inline module, it should not surprise that the file is ignored?

I think the only remaining question left is, if a combination of hosting/mod.rs with hosting.rs could make any sense, when both files have different content? But note, that the use of mod.rs files is legacy, which should not be used in new projects generally. And mixing both versions in a single crate is generally strongly dis-curated. I think this was told in some books. I think there was one special case mentioned somewhere, where mod.rs modules can make still some sense in modern Rust, but I can currently not remember details.

It is nice that you tested so carefully all variants, but still I would suggest closing this issue. Or tell us exactly where in our books the description is wrong.

[iyernaveenr]

Are you sure that a directory structure like

    ├── front_of_house
    │   ├── hosting
    │   │   └── mod.rs
    │   └── hosting.rs

is valid and recommended in the book? My feeling was, that we can either have a mod.rs file in hosting folder, or a hostings.rs file, but not both. Are you referring to the original book, or to the one from Brown University?

Original book. In my post, I had mentioned exactly where, quote:
"
In Chapter 7 of rustbook, Section 7.2 -> Modules Cheat Sheet -> Declaring modules:, the preference of the compiler for module declarations is clearly defined.
"
Here's the link for your convenience:
https://doc.rust-lang.org/book/ch07-02-defining-modules-to-control-scope-and-privacy.html

You can start from there and read on other sections in that chapter as well if you like. But I think that specific section in itself should suffice to make my case.

[iyernaveenr]

effectively ignoring 2 of 3).

Well, if you replace the statement "mod my_module" with an inline module, it should not surprise that the file is ignored?

I think the only remaining question left is, if a combination of hosting/mod.rs with hosting.rs could make any sense, when both files have different content? But note, that the use of mod.rs files is legacy, which should not be used in new projects generally. And mixing both versions in a single crate is generally strongly dis-curated. I think this was told in some books. I think there was one special case mentioned somewhere, where mod.rs modules can make still some sense in modern Rust, but I can currently not remember details.

It is nice that you tested so carefully all variants, but still I would suggest closing this issue. Or tell us exactly where in our books the description is wrong.

That special case you are referring to is under Section 11.3, subsection "Submodules in Integration Tests":
https://doc.rust-lang.org/book/ch11-03-test-organization.html#submodules-in-integration-tests
As described there, 3 of 3) seems to be the only way to prevent common from appearing in the test results.

Inlining is fine, but then the wording in the book is ambiguous w.r.t the interpretation by a compiler. Is the compiler supposed to error out in cases when multiple forms of declaring modules were used? Or, is the compiler supposed to silently prefer one over the other?
Whichever behavior is chosen, that should've been used consistently for any combination of module declarations used. But what was observed, as evidenced by the details I shared, is the inconsistent/biased behavior by the compiler.

I would personally prefer that the compiler error out. The reason for that is, imagine this - If I wanted to sneak in a malicious piece of code (say a backdoor, for example), all I have to do is to exploit this silent ignorance of the compiler to sneak in code. So, wherever a module is declared as per 1 of 3), I could simply add malicious code by replacing the semicolon with my code enclosed with curly braces. Maintainer of that code may not realize until it's too late, if at all, that their original code, which had resided in a file named after the module was never compiled into the final binary. This could be a serious security issue.

This is the reason I had originally posted this as an internal compiler issue, since I would've liked the compiler developers to handle this case as well. But then they suggested I post it here instead. I believe this is an issue that should not be closed, but instead be perused and if needed, escalated to handle the potential scenario where hackers could sneak in malicious code.

[iyernaveenr]

effectively ignoring 2 of 3).

Well, if you replace the statement "mod my_module" with an inline module, it should not surprise that the file is ignored?
I think the only remaining question left is, if a combination of hosting/mod.rs with hosting.rs could make any sense, when both files have different content? But note, that the use of mod.rs files is legacy, which should not be used in new projects generally. And mixing both versions in a single crate is generally strongly dis-curated. I think this was told in some books. I think there was one special case mentioned somewhere, where mod.rs modules can make still some sense in modern Rust, but I can currently not remember details.
It is nice that you tested so carefully all variants, but still I would suggest closing this issue. Or tell us exactly where in our books the description is wrong.

That special case you are referring to is under Section 11.3, subsection "Submodules in Integration Tests": https://doc.rust-lang.org/book/ch11-03-test-organization.html#submodules-in-integration-tests For the case mentioned in the book, 3 of 3) seems to be the only way to prevent common from appearing in the test results.

Inlining is fine, but then the wording in the book is ambiguous w.r.t the interpretation by a compiler. Is the compiler supposed to error out in cases when multiple forms of declaring modules were used? Or, is the compiler supposed to silently prefer one over the other? Whichever behavior is chosen, that should've been used consistently for any combination of module declarations used. But what was observed, as evidenced by the details I shared, is the inconsistent/biased behavior by the compiler.

I would personally prefer that the compiler error out. The reason for that is, imagine this - If I wanted to sneak in a malicious piece of code (say a backdoor, for example), all I have to do is to exploit this silent ignorance of the compiler to sneak in code. So, wherever a module is declared as per 1 of 3), I could simply add malicious code by replacing the semicolon with my code enclosed with curly braces. Maintainer of that code may not realize until it's too late, if at all, that their original code, which had resided in a file named after the module was never compiled into the final binary. This could be a serious security issue.

This is the reason I had originally posted this as an internal compiler issue, since I would've liked the compiler developers to handle this case as well. But then they suggested I post it here instead. I believe this is an issue that should not be closed, but instead be perused and if needed, escalated to handle the potential scenario where hackers could sneak in malicious code.

I've suggested in the link below that this issue be re-opened as an internal compiler issue instead:
rust-lang/rust#139685 (comment)
Thoughts?

[StefanSalewski]

Thanks for the clarifications.

The first reply to your original post in the Rust issue tracker was:

The compiler works perfectly fine here, the phrasing of the book is just ambiguous. These three options are simply mutually exclusive and the book doesn't make it clear.

And I think this is indeed the core of the problem. My understanding, when I read the official book two years ago was, that to make a module file visible, there is a "mod name;" instruction needed somewhere higher in the module hierarchy to use it. So if it is a security issue, when there are somewhere unused .rs files in the file hierarchy, might be an interesting question. I am currently revising the chapters of my own online book, and will see if I can make these points as clear as possible, thanks for your comments.

[iyernaveenr]

Thanks for the clarifications.

The first reply to your original post in the Rust issue tracker was:

The compiler works perfectly fine here, the phrasing of the book is just ambiguous. These three options are simply mutually exclusive and the book doesn't make it clear.

And I think this is indeed the core of the problem. My understanding, when I read the official book two years ago was, that to make a module file visible, there is a "mod name;" instruction needed somewhere higher in the module hierarchy to use it. So if it is a security issue, when there are somewhere unused .rs files in the file hierarchy, might be an interesting question. I am currently revising the chapters of my own online book, and will see if I can make these points as clear as possible, thanks for your comments.

Thanks. Due to lack of permission to re-open the issue I had originally opened as an internal compiler issue, I've opened a new issue, linking to the old issue:
rust-lang/rust#140085

Beyond the scope of this post - I'm curious why you'd want to author your own online book, when you could simply point out issues/mistakes in the existing rust book? The reason I ask is because if everyone started their own book like that, it'll lead to fragmentation, thereby, potentially creating more conflicting "facts", causing readers to search for the source of truth. Wouldn't having a single source of truth be preferable?

[StefanSalewski]

everyone started their own book

Yes, actually I am wondering myself about the hundreds of Rust books available at Amazon.

For me personally: In October 2023, when I read the official book, I created two dozen of serious issues at GitHub, but that time the maintainers have not been very interested in fixing issues, most of my ones have been closed after a few weeks just as "won't fix" or they stayed open for a year. With the new principal maintainer, things have improved, now indeed a lot issues have been fixed. But the official books has still its merits -- sections more addressed to true beginners in programming, or sections mostly useless, or very verbose. And I think these sections will stay forever. I think the official book deserves 3.5 stars from five, which is ok for a free book, but not really great. When you have looked at my book, you should have noticed that "Rust for C-Programmers" tries to be a concise Rust introduction, avoiding verbosity and longer explanations addressed to absolute beginners. And I think my book has a bit more actual content, it is more between the official book, which is more a tutorial, and "Programming Rust" by Jim Blandy, which might be already a bit heavy. I thought that such a book was still missing. Maybe such a book exists under the hundreds book at Amazon, but how could we find out?