Skip to content

macros: add "local" runtime flavor #7375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
12 changes: 11 additions & 1 deletion tests-integration/tests/macros_main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,13 @@ async fn spawning() -> usize {
join.await.unwrap()
}

#[cfg(tokio_unstable)]
#[tokio::main(flavor = "local")]
async fn local_main() -> usize {
let join = tokio::task::spawn_local(async { 1 });
join.await.unwrap()
}

#[test]
fn main_with_spawn() {
assert_eq!(1, spawning());
Expand All @@ -24,5 +31,8 @@ fn main_with_spawn() {
#[test]
fn shell() {
assert_eq!(1, basic_main());
assert_eq!(bool::default(), generic_fun::<bool>())
assert_eq!(bool::default(), generic_fun::<bool>());

#[cfg(tokio_unstable)]
assert_eq!(1, local_main());
}
35 changes: 27 additions & 8 deletions tokio-macros/src/entry.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,15 @@ type AttributeArgs = syn::punctuated::Punctuated<syn::Meta, syn::Token![,]>;
enum RuntimeFlavor {
CurrentThread,
Threaded,
Local,
}

impl RuntimeFlavor {
fn from_str(s: &str) -> Result<RuntimeFlavor, String> {
match s {
"current_thread" => Ok(RuntimeFlavor::CurrentThread),
"multi_thread" => Ok(RuntimeFlavor::Threaded),
"local" => Ok(RuntimeFlavor::Local),
"single_thread" => Err("The single threaded runtime flavor is called `current_thread`.".to_string()),
"basic_scheduler" => Err("The `basic_scheduler` runtime flavor has been renamed to `current_thread`.".to_string()),
"threaded_scheduler" => Err("The `threaded_scheduler` runtime flavor has been renamed to `multi_thread`.".to_string()),
Expand Down Expand Up @@ -177,15 +179,16 @@ impl Configuration {
use RuntimeFlavor as F;

let flavor = self.flavor.unwrap_or(self.default_flavor);

let worker_threads = match (flavor, self.worker_threads) {
(F::CurrentThread, Some((_, worker_threads_span))) => {
(F::CurrentThread | F::Local, Some((_, worker_threads_span))) => {
let msg = format!(
"The `worker_threads` option requires the `multi_thread` runtime flavor. Use `#[{}(flavor = \"multi_thread\")]`",
self.macro_name(),
);
return Err(syn::Error::new(worker_threads_span, msg));
}
(F::CurrentThread, None) => None,
(F::CurrentThread | F::Local, None) => None,
(F::Threaded, worker_threads) if self.rt_multi_thread_available => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add logic to rt_multi_thread_available to detect whether --cfg tokio_unstable is set and emit a hard error if it is not.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand this comment. Did you mean to add the logic to RuntimeFlavor::from_str?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error should be similar to this error:

let msg = if self.flavor.is_none() {
"The default runtime flavor is `multi_thread`, but the `rt-multi-thread` feature is disabled."
} else {
"The runtime flavor `multi_thread` requires the `rt-multi-thread` feature."
};

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you meant a0ca77e; if I've misinterpreted please feel free to suggest changes :)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I just saw your comment. Is 4363203 more what you had in mind?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think both are okay.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is harder than it appears because RUSTFLAGS is not passed to proc macros. I guess we have to emit a cfg check, so it actually can't live in either of these places.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

44a31b1 seems to work in my local tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't that also trigger other unrelated errors when it fails?

What we do today is essentially define the main macro two times to pass different values to the rt_multi_thread boolean, and have lib.rs do this:

#[cfg(feature = "rt-multi-thread")]
pub use tokio_macros::main;
#[cfg(not(feature = "rt-multi-thread"))]
pub use tokio_macros::main_rt as main;

This way, the macro knows based on the cfgs that apply in the main Tokio crate. We could use the same approach and have four macros.

worker_threads.map(|(val, _span)| val)
}
Expand All @@ -207,7 +210,7 @@ impl Configuration {
);
return Err(syn::Error::new(start_paused_span, msg));
}
(F::CurrentThread, Some((start_paused, _))) => Some(start_paused),
(F::CurrentThread | F::Local, Some((start_paused, _))) => Some(start_paused),
(_, None) => None,
};

Expand All @@ -219,7 +222,7 @@ impl Configuration {
);
return Err(syn::Error::new(unhandled_panic_span, msg));
}
(F::CurrentThread, Some((unhandled_panic, _))) => Some(unhandled_panic),
(F::CurrentThread | F::Local, Some((unhandled_panic, _))) => Some(unhandled_panic),
(_, None) => None,
};

Expand Down Expand Up @@ -408,13 +411,28 @@ fn parse_knobs(mut input: ItemFn, is_test: bool, config: FinalConfig) -> TokenSt
.unwrap_or_else(|| Ident::new("tokio", last_stmt_start_span).into_token_stream());

let mut rt = match config.flavor {
RuntimeFlavor::CurrentThread => quote_spanned! {last_stmt_start_span=>
#crate_path::runtime::Builder::new_current_thread()
},
RuntimeFlavor::CurrentThread | RuntimeFlavor::Local => {
quote_spanned! {last_stmt_start_span=>
#crate_path::runtime::Builder::new_current_thread()
}
}
RuntimeFlavor::Threaded => quote_spanned! {last_stmt_start_span=>
#crate_path::runtime::Builder::new_multi_thread()
},
};

let mut checks = vec![];

let build = if let RuntimeFlavor::Local = config.flavor {
checks.push(quote! {
#[cfg(not(tokio_unstable))]
compile_error!("The local runtime flavor is only available when `tokio_unstable` is set.");
});
quote_spanned! {last_stmt_start_span=> build_local(Default::default())}
} else {
quote_spanned! {last_stmt_start_span=> build()}
};

if let Some(v) = config.worker_threads {
rt = quote_spanned! {last_stmt_start_span=> #rt.worker_threads(#v) };
}
Expand All @@ -439,9 +457,10 @@ fn parse_knobs(mut input: ItemFn, is_test: bool, config: FinalConfig) -> TokenSt
let last_block = quote_spanned! {last_stmt_end_span=>
#[allow(clippy::expect_used, clippy::diverging_sub_expression, clippy::needless_return)]
{
#(#checks)*
return #rt
.enable_all()
.build()
.#build
.expect("Failed building the Runtime")
.block_on(#body_ident);
}
Expand Down
79 changes: 67 additions & 12 deletions tokio-macros/src/lib.rs
100644 → 100755
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The permission of this file is updated too.

Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,12 @@ use proc_macro::TokenStream;
/// Awaiting on other futures from the function provided here will not
/// perform as fast as those spawned as workers.
///
/// # Multi-threaded runtime
/// # Runtime flavors
///
/// The macro can be configured with a `flavor` parameter to select
/// different runtime configurations.
///
/// ## Multi-threaded
///
/// To use the multi-threaded runtime, the macro can be configured using
///
Expand All @@ -61,23 +66,37 @@ use proc_macro::TokenStream;
/// Note: The multi-threaded runtime requires the `rt-multi-thread` feature
/// flag.
///
/// # Current thread runtime
/// ## Current-thread
///
/// To use the single-threaded runtime known as the `current_thread` runtime,
/// the macro can be configured using
///
/// ```
/// ```rust
/// #[tokio::main(flavor = "current_thread")]
/// # async fn main() {}
/// ```
///
/// ## Function arguments:
/// ## Local
///
/// Arguments are allowed for any functions aside from `main` which is special
/// [Unstable API][unstable] only.
///
/// ## Usage
/// To use the [local runtime], the macro can be configured using
///
/// ### Using the multi-thread runtime
/// ```rust
/// # #[cfg(tokio_unstable)]
/// #[tokio::main(flavor = "local")]
/// # async fn main() {}
/// # #[cfg(not(tokio_unstable))]
/// # fn main() {}
/// ```
///
/// # Function arguments
///
/// Arguments are allowed for any functions, aside from `main` which is special.
///
/// # Usage
///
/// ## Using the multi-threaded runtime
///
/// ```rust
/// #[tokio::main]
Expand All @@ -100,7 +119,7 @@ use proc_macro::TokenStream;
/// }
/// ```
///
/// ### Using current thread runtime
/// ## Using the current-thread runtime
///
/// The basic scheduler is single-threaded.
///
Expand All @@ -125,7 +144,42 @@ use proc_macro::TokenStream;
/// }
/// ```
///
/// ### Set number of worker threads
/// ## Using the local runtime
///
/// Available in the [unstable API][unstable] only.
///
/// The [local runtime] is similar to the current-thread runtime but
/// supports [`task::spawn_local`](../tokio/task/fn.spawn_local.html).
///
/// ```rust
/// # #[cfg(tokio_unstable)]
/// #[tokio::main(flavor = "local")]
/// async fn main() {
/// println!("Hello world");
/// }
/// # #[cfg(not(tokio_unstable))]
/// # fn main() {}
/// ```
///
/// Equivalent code not using `#[tokio::main]`
///
/// ```rust
/// # #[cfg(tokio_unstable)]
/// fn main() {
/// tokio::runtime::Builder::new_current_thread()
/// .enable_all()
/// .build_local(tokio::runtime::LocalOptions::default())
/// .unwrap()
/// .block_on(async {
/// println!("Hello world");
/// })
/// }
/// # #[cfg(not(tokio_unstable))]
/// # fn main() {}
/// ```
///
///
/// ## Set number of worker threads
///
/// ```rust
/// #[tokio::main(worker_threads = 2)]
Expand All @@ -149,7 +203,7 @@ use proc_macro::TokenStream;
/// }
/// ```
///
/// ### Configure the runtime to start with time paused
/// ## Configure the runtime to start with time paused
///
/// ```rust
/// #[tokio::main(flavor = "current_thread", start_paused = true)]
Expand All @@ -175,7 +229,7 @@ use proc_macro::TokenStream;
///
/// Note that `start_paused` requires the `test-util` feature to be enabled.
///
/// ### Rename package
/// ## Rename package
///
/// ```rust
/// use tokio as tokio1;
Expand All @@ -202,7 +256,7 @@ use proc_macro::TokenStream;
/// }
/// ```
///
/// ### Configure unhandled panic behavior
/// ## Configure unhandled panic behavior
///
/// Available options are `shutdown_runtime` and `ignore`. For more details, see
/// [`Builder::unhandled_panic`].
Expand Down Expand Up @@ -247,6 +301,7 @@ use proc_macro::TokenStream;
///
/// [`Builder::unhandled_panic`]: ../tokio/runtime/struct.Builder.html#method.unhandled_panic
/// [unstable]: ../tokio/index.html#unstable-features
/// [local runtime]: ../tokio/runtime/struct.LocalRuntime.html
#[proc_macro_attribute]
pub fn main(args: TokenStream, item: TokenStream) -> TokenStream {
entry::main(args.into(), item.into(), true).into()
Expand Down