waitForProcess: does not exist (No child processes)

[dio4ev]

Hi all!

I hope that this is the right place to ask such a question.

I try to parallize parts of a program of mine, but get a strange runtime error and as far as I know it is related to something in the process package.
In my tool, I get some specific input, build a constraint from it which can be solved in 3 different ways (3 solvers; involving external tools).
Thus I would like to run these three different solvers on the constraint in parallel and return the result of the fastest one.
In simple words it looks as follows:
A solver s has a type s :: constraint -> IO [a] and I apply the constraint to all solvers to get a list of possible solutions [IO [a]]. After that I call async 0 on them and wait with waitAny for a result (if I find a correct result, i.e. a non-empty list is returned, then I kill all the other jobs with cancel). Within a solver external tools are called with readProcess and readProcessWithExitCode. The problem is that I get quite often the following runtime error waitForProcess: does not exist (No child processes).

• Is mixing async and process causing this?

• Is this related to something similar as 1?

• Does anybody has an idea what could cause this?

In the meantime I replaced the async package functions completely by forkIO etc., but still this exception occurs.
However, I never experienced this exception before I parallelized my program.

Thank you very much and cheers!

[Fuuzetsu]

It's a bit hard to help without seeing any code. You should be able to mix async and process, there's no fundamental limitation I think. #107 is probably not related.

I assume the code is private but if you could strip out the process invocation/waiting/cancel parts and post them, it'd be a lot easier to give help. Preferably the non-async version, just fewer moving blocks.

[dio4ev]

Sure I understand that, I will try to strip out some of the important parts:

After the input is parsed I call all elements of solvers on the input with the following parallelization:

ecParP :: Input -> Maybe Int -> Maybe Int -> IO [Result]
ecParP p i n = do
  var <- newEmptyMVar
  threads <- mapM (forkIO . mVarFun null var . run') solvers
  waitThreadSet [] success (S.fromList threads) var
  where (_, c) = getMain p
        run' solv = run solv c

        success Nothing = False
        success (Just ress) = any succ' ress
          where
            succ' (Failure _ _) = False
            succ' (Success _ _) = True

mVarFun :: (a -> Bool) -> MVar (ThreadId, Maybe a) -> IO a -> IO ()
mVarFun noResult var comp = do
  res <- comp
  id <- myThreadId
  if noResult res then
    putMVar var (id, Nothing)
  else
    putMVar var (id, Just res)

-- Wait for threads created with forkIO
waitThreadSet :: a -> (Maybe a -> Bool) -> S.Set ThreadId -> MVar (ThreadId, Maybe a) -> IO a
waitThreadSet def success threads var = go S.empty
  where
    go finished =
      if threads == finished then
        return def
      else do
        (id,res) <- takeMVar var
        if success res then do
          mapM_ killThread (threads S.\\ finished)
          return $ fromJust res
        else
          go (S.insert id finished)

Within a solver an external tool is e.g. called with:

readProcess "matlab" ["-nosplash", "-nodisplay", "-r", file] ""

I simplified the functions a bit and hope that this helps.

Cheers

[Fuuzetsu]

Doesn't look like it should be broken. Maybe waitForProcess should be checking ECHILD just like getProcessExitCode does? Perhaps someone else has to weigh in.

You should write down your process version and OS too I gues.

[dio4ev]

I found out that also readProcessWithExitCode is used in a local external tool I am using.

$ uname -a
Linux arch-work 5.15.13-arch1-1 #1 SMP PREEMPT Wed, 05 Jan 2022 16:20:59 +0000 x86_64 GNU/Linux

$ cabal info process

* process          (library)
    Synopsis:      Process libraries
    Versions available: 1.4.2.0, 1.4.3.0, 1.5.0.0, 1.6.10.0, 1.6.11.0, 1.6.12.0,
                        (1.6.13.0), 1.6.13.1, 1.6.13.2 (and 29 others)
    Versions installed: 1.6.13.2
    Homepage:      [ Not specified ]
    Bug reports:   https://github.com/haskell/process/issues
    Description:   This package contains libraries for dealing with system
                   processes.

                   The typed-process package is a more recent take on a process
                   API, which uses this package internally. It features better
                   binary support, easier concurrency, and a more composable
                   API. You can read more about it at
                   <https://github.com/fpco/typed-process/#readme>.
    Category:      System
    License:       BSD3
    Maintainer:    libraries@haskell.org
    Source repo:   https://github.com/haskell/process.git
    Dependencies:  base >=4.10 && <4.17, directory >=1.1 && <1.4,
                   filepath >=1.2 && <1.5, deepseq >=1.1 && <1.5,
                   Win32 >=2.4 && <2.13, unix >=2.5 && <2.8, base >=4 && <5,
                   bytestring, directory, process
    Documentation: ~/.ghcup/ghc/8.10.7/share/doc/ghc-8.10.7/html/libraries/process-1.6.13.2
    Cached:        No
    Modules:
        System.Cmd
        System.Process
        System.Process.Internals

[Fuuzetsu]

readProcess and readProcessWithExitCode end up going through same function in the end so I expect both to have the same behaviour fwiw.

[dio4ev]

Ahh yeah, then it was not necessary to mention that :)
But I still can't figure out what is going wrong here...

[robx]

Compare the discussion in fpco/typed-process#38 and #204. It does appear the process isn't async-safe.

[dio4ev]

@robx: Thanks.
Looks like there is no solution for this problem at the moment...

[robx]

@dio4ev did you see #231? I believe that fixes the issue, maybe you could verify it?

[robx]

Could we close this? The bug should be fixed in 1.6.14.0.

[snoyberg]

I believe so, closing.