We need a custom executor

[ve3jrw]

Discussion at https://groups.google.com/forum/#!topic/autobahnws/1WkyztKdoLw

I'm looking at using Autobahn-cpp in an embedded arm system with bonefish as the WAMP router.

I built the callee example, but was surprised to see it has a very large memory foot print (a number of processes have been removed from this listing):

Mem: 61968K used, 177892K free, 0K shrd, 748600K buff, 235092K cached
CPU: 1% usr 9% sys 0% nic 88% idle 0% io 0% irq 0% sirq
Load average: 0.19 0.28 0.31 1/126 2641
PID PPID USER STAT VSZ %MEM %CPU COMMAND
2622 1031 root S 54500 23% 0% ./callee -r map300 -p 8888
1025 1024 root S 8104 3% 0% ./bonefish --realm map300 --websocket-port 8080 --rawsocket-port 8888

This so I dug in a bit, and pmap shows 6 "stack" signatures (we have 8K stacks on this box):

2622: ./callee -r map300 -p 8888
00010000 336K r-x-- callee
00073000 4K r---- callee
00074000 4K rw--- callee
00075000 148K rw--- [ anon ]
b3a00000 4K ----- [ anon ]
b3a01000 8188K rw--- [ anon ]
b4200000 4K ----- [ anon ]
b4201000 8188K rw--- [ anon ]
b4a00000 4K ----- [ anon ]
b4a01000 8188K rw--- [ anon ]
b5200000 4K ----- [ anon ]
b5201000 8188K rw--- [ anon ]
b5a00000 4K ----- [ anon ]
b5a01000 8320K rw--- [ anon ]
b6221000 892K ----- [ anon ]
b63a8000 4K ----- [ anon ]
b63a9000 8188K rw--- [ anon ]
b6ba8000 412K r-x-- libm.so.6
b6c0f000 60K ----- libm.so.6
b6c1e000 4K r---- libm.so.6
b6c1f000 4K rw--- libm.so.6

but it only has one thread (from cat /proc/XXXX/status):

VmPMD: 0 kB
VmSwap: 0 kB
Threads: 1
SigQ: 0/1742
SigPnd: 0000000000000000
ShdPnd: 0000000000000000

Historically I've seen this when threads are not disconnected or joined. The stack memory gets orphaned and is not released until the program exits.

I removed the two RPC registrations and the stack count decreases by two. I suspect it is the boost future that is involved with the problem.

Same high number of stacks is seen with publisher and I suspect is common to all the other examples.

Have tried using the cppWamp library (forked from Autobahn) and it doesn't exhibit this problem, but it doesn't use boost futures.

[ecorm]

Have tried using the cppWamp library (forked from Autobahn) and it doesn't exhibit this problem, but it doesn't use boost futures.

I'd just like to point out for the record that CppWAMP was not forked from Autobahn|Cpp. It was developed independently from scratch.

[ve3jrw]

My mistake. I thought I had read that it was forked in one of the dialogs I read through.

[oberstet]

@ve3jrw it seems boost by default will run continuations in new threads #156 - This behavior is surprising/unexpected for me at least: I am using continuations etc exactly to get rid of threads. Anyway, call it my fault. I dont have time to dig into right now, but if this is all true, then we have to change all examples and make users aware of this trap.

[ecorm]

See #43

[oberstet]

ah, right. I tend to forget what I wrote before;) Anyway, @ecorm what do you think about #43 (comment) ?

[ecorm]

I was merely linking this issue to that one. In the experimental future-api branch in CppWAMP, I was able to solve the thread-spawning problem by writing my own custom executor. It was only tested with a toy test program. Feel free to borrow my custom executor idea for Autobahn|cpp.

[oberstet]

Ah, nice!

What disturbs me is that Boost/ASIO doesn't have an official solution (executor) for this. Others must have run into this also .. strange. This whole stuff (futures) seems to be very beta in C++/Boost/ASIO .. or am I just spoiled by Python? ;)

[ecorm]

Boost.Thread executors and continuations are indeed experimental, and are tagged as such in the documentation.

The 1.11.0 development release of the standalone Asio library has added Executor classes as proposed in N4370 and N4045.

The author of Asio proposes a different executor model than the experimental one in Boost.Thread (because of performance concerns). The Asio model also uses functions other than .then() for handler chaining.

Things have been progressing slowly on this front. I suspect that we might have to wait until the C++20 draft for this asynchronous stuff to solidify. :-(

[ve3jrw]

#156 is the same problem. Anyway, not an issue for us as we are using cppwamp to facilitate our product as this problem is a show stopper to use Autobahn. Thanks for the follow up and will continue to monitor and evaluate Autobahn C++ when the next requirement comes up and a fix has been made.