Unable to run nomad `agent -dev` inside docker

[migueleliasweb]

Hi, I'm having issues running Nomad using this docker container.

I'm trying to run with these options:

$ docker run -it --net host -v /var/run/docker.sock:/var/run/docker.sock --privileged djenriquez/nomad agent -dev

Although the initialization works, when I try to install the example job it fails with these messages:

$ nomad job run example.nomad 
==> Monitoring evaluation "5fe255d4"
    Evaluation triggered by job "example"
    Allocation "1672dcc0" created: node "77289a29", group "cache"
    Evaluation within deployment: "088fcb7b"
    Allocation "1672dcc0" status changed: "pending" -> "failed"
    Evaluation status changed: "pending" -> "complete"
==> Evaluation "5fe255d4" finished with status "complete"

$ nomad status
ID       Type     Priority  Status  Submit Date
example  service  50        dead    2018-11-20T23:16:50+11:00

$ nomad status example
ID            = example
Name          = example
Submit Date   = 2018-11-20T23:16:50+11:00
Type          = service
Priority      = 50
Datacenters   = dc1
Status        = dead
Periodic      = false
Parameterized = false

Summary
Task Group  Queued  Starting  Running  Failed  Complete  Lost
cache       0       0         0        1       0         0

Latest Deployment
ID          = 088fcb7b
Status      = failed
Description = Failed due to unhealthy allocations

Deployed
Task Group  Desired  Placed  Healthy  Unhealthy
cache       1        1       0        1

Allocations
ID        Node ID   Task Group  Version  Desired  Status  Created  Modified
1672dcc0  77289a29  cache       0        run      failed  8s ago   7s ago

$ nomad status 1672dcc0
ID                  = 1672dcc0
Eval ID             = 5fe255d4
Name                = example.cache[0]
Node ID             = 77289a29
Job ID              = example
Job Version         = 0
Client Status       = failed
Client Description  = <none>
Desired Status      = run
Desired Description = <none>
Created             = 12s ago
Modified            = 11s ago
Deployment ID       = 088fcb7b
Deployment Health   = unhealthy

Task "redis" is "dead"
Task Resources
CPU      Memory   Disk     IOPS  Addresses
500 MHz  256 MiB  300 MiB  0     db: 127.0.0.1:30012

Task Events:
Started At     = N/A
Finished At    = N/A
Total Restarts = 0
Last Restart   = N/A

Recent Events:
Time                       Type             Description
2018-11-20T23:16:50+11:00  Alloc Unhealthy  Unhealthy because of failed task
2018-11-20T23:16:50+11:00  Not Restarting   Error was unrecoverable
2018-11-20T23:16:50+11:00  Driver Failure   failed to start task "redis" for alloc "1672dcc0-3006-c6df-a5f7-45d37589f069": Failed to create container: API error (400): {"message":"stat /tmp/plugin615200430: no such file or directory"}
2018-11-20T23:16:50+11:00  Task Setup       Building Task Directory
2018-11-20T23:16:50+11:00  Received         Task received by client

Have you seen this Driver Failure message before?

[ForsakenHarmony]

You have to map /tmp:/tmp

[migueleliasweb]

Hey @ForsakenHarmony, what will the volume change in this case?

The nomad process is already able write to /tmp inside the container (although it wouldn't maintain state in a restart, I know).

Cheers!

[ForsakenHarmony]

I think the docker daemon might write to /tmp on the host

[migueleliasweb]

You are right, it does write to the /tmp folder. But as it's running inside the container it would just do it to the containers /tmp, right?

I don't think it would know it's writing to the host or the container folder in this case.