Any build scripts can now use cargo::metadata=KEY=VALUE

[ranger-ross]

What does this PR try to resolve?

See #3544

• Adds an unstable -Zany-build-script-metadata feature
• Adds CARGO_DEP_<name>_<key> env vars as described in DEP_FOO_KEY-like system that can work without "links" #3544 (comment) when -Zany-build-script-metadata is passed

Unresolved questions

• Naming of the variable, see Any build scripts can now use cargo::metadata=KEY=VALUE #16436 (comment)
• Use of envify, see Any build scripts can now use cargo::metadata=KEY=VALUE #16436 (comment)

How to test and review this PR?

See the e2e tests included in this PR.
As for the changes, they are not too bad.

---

meta: I picked this up to learn a bit more about the build script and unit graph code in Cargo.

r? @epage

[epage]

For me, one unresolved question is what to name this env variable

• Technically, name_key is ambiguous. foo_bar_alice_bob could be foo_bar + alice_bob, foo + bar_alice_bob, etc and technically packages could conflict
• Whether we should envify the package name and key or not. iirc we do it in some places but not others. Overall, I would prefer we would never do it so it matches the name people see in Cargo.toml

[ranger-ross]

• Technically, name_key is ambiguous. foo_bar_alice_bob could be foo_bar + alice_bob, foo + bar_alice_bob, etc and technically packages could conflict

I suppose one idea would be to using another delimiter between the name and key.
Maybe double underscore? foo_bar__alice_bob would now be different than foo__bar_alice_bob.
Not sure if there are any other delimiters that are compatible across enough platforms. I think POSIX is already pretty restrictive with what can be used an env var name.

• Whether we should envify the package name and key or not. iirc we do it in some places but not others. Overall, I would prefer we would never do it so it matches the name people see in Cargo.toml

Given that env vars from cargo::metadata=KEY=VALUE are already envify'd in DEP_<links>_<key> I think it makes sense to keep that? I think it'd be more confusing if cargo::metadata=KEY=VALUE had multiple behaviors.

[epage]

Note that I highlighted these for being called out in the stabilization conversations and do not block this PR. I don't have concrete ideas to say exactly what should be done but want to raise these for a wider conversation.

[epage]

Calling out for resolving

[epage]

We should do whatever we do for links

[ranger-ross]

I spent some time testing this and as far as I can tell links only propagates env vars set with cargo::metadata if its a regular dependency.

I can see we explicitly filter out dev-dependencies in:

cargo/src/cargo/core/compiler/unit_dependencies.rs

Lines 987 to 990 in 577ab6f

	// Skip dependencies induced via dev-dependencies since
	// connections between `links` and build scripts only happens
	// via normal dependencies. Otherwise since dev-dependencies can
	// be cyclic we could have cyclic build-script executions.

and I believe that build-dependencies are implicitly not included when the unit graph is built.

I removed the FIXME comment, but lmk if you think there is something I missed

[epage]

We should probably have tests showing the behavior for both ways of accessing metadata. Here is what I used for experimenting:

#[cargo_test]
fn metadata_from_dep_kinds() {
    let set_metadata = r#"
fn main() {
    println!("cargo::metadata=key=value");
}
"#;

    let get_metadata = r#"
fn main() {
    println!("cargo::warning={:?}", std::env::var("DEP_FOO_KEY"));
}
"#;

    let p = project()
        .file(
            "Cargo.toml",
            r#"
                [workspace]
                resolver = "3"
                members = ["links", "n", "b", "d"]
            "#,
        )
        .file(
            "links/Cargo.toml",
            r#"
[package]
name = "links"
edition = "2024"
links = "foo"
"#,
        )
        .file("links/src/lib.rs", "")
        .file("links/build.rs", set_metadata)
        .file(
            "n/Cargo.toml",
            r#"
[package]
name = "n"
edition = "2024"

[dependencies]
links.path = "../links"
"#,
        )
        .file("n/src/lib.rs", "")
        .file("n/build.rs", get_metadata)
        .file(
            "b/Cargo.toml",
            r#"
[package]
name = "b"
edition = "2024"

[build-dependencies]
links.path = "../links"
"#,
        )
        .file("b/src/lib.rs", "")
        .file("b/build.rs", get_metadata)
        .file(
            "d/Cargo.toml",
            r#"
[package]
name = "d"
edition = "2024"

[dev-dependencies]
links.path = "../links"
"#,
        )
        .file("d/src/lib.rs", "")
        .file("d/build.rs", get_metadata)
        .build();

    p.cargo("check --all-targets")
        .with_stderr_data(
            str![[r#"
[COMPILING] links v0.0.0 ([ROOT]/foo/links)
[COMPILING] n v0.0.0 ([ROOT]/foo/n)
[COMPILING] b v0.0.0 ([ROOT]/foo/b)
[COMPILING] d v0.0.0 ([ROOT]/foo/d)
[WARNING] [email protected]: Ok("value")
[WARNING] [email protected]: Err(NotPresent)
[WARNING] [email protected]: Err(NotPresent)
[FINISHED] `dev` profile [unoptimized + debuginfo] target(s) in [ELAPSED]s

"#]]
            .unordered(),
        )
        .run();
}

just needs the new CARGO_DEP env variables added to it

[ranger-ross]

ahh thanks, using cargo::warning so we can leverage the snapbox testing infra is clever.

I added this test to the test commit and modified it to include CARGO_DEP_LINKS_KEY in the feat commit.

[ranger-ross]

note that while investigating #16436 (comment) I noticed that only checking the package name could lead to inconsistent behavior if a package has 2 versions of a package imported under different names.

To mitigate this I added checks for source_id and version. This is close to comparing package_id (Dependency does not have package_id)

Note that Dependency only has a version requirement so we use .matches(). I think in practice we should only have 1 possible matching version as Cargo would error while building the unit graph if the same dependencies. (ref)
correct me if I am wrong

[rustbot]

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.