`test_embed.test_bpo20891` is racy under free-threading

[ZeroIntensity]

Bug report

Bug description:

I haven't seen this in CI, but when running regrtest locally, I noticed a crash on test_embed.test_bpo20891 regarding unlocking a mutex that wasn't locked.

The two offending pieces of code are here:

cpython/Programs/_testembed.c

Lines 412 to 417 in d5e75c0

	unsigned long thrd = PyThread_start_new_thread(bpo20891_thread, &lock);
	if (thrd == PYTHREAD_INVALID_THREAD_ID) {
	error("PyThread_start_new_thread failed!");
	return 1;
	}
	PyThread_acquire_lock(lock, WAIT_LOCK);

cpython/Programs/_testembed.c

Lines 382 to 395 in d5e75c0

	static void bpo20891_thread(void *lockp)
	{
	PyThread_type_lock lock = *((PyThread_type_lock*)lockp);
	PyGILState_STATE state = PyGILState_Ensure();
	if (!PyGILState_Check()) {
	error("PyGILState_Check failed!");
	abort();
	}
	PyGILState_Release(state);
	PyThread_release_lock(lock);
	}

The problem is that in rare cases, the created thread can hit the call to PyThread_release_lock before it's held by the main thread, which causes a fatal error. This isn't an issue on the GILful build, because PyGILState_Ensure will block until the main thread releases the GIL.

I think the best fix would be to use a PyEvent to signal to the main thread rather than a lock.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-137093: Fix race condition in test_embed.test_bpo20891 #137094