hub 3plumb | 2012-08-13 |
---|
hub - merge and split record streams
[hub]
[hub eof=ignore]
+-------+ X0 ---->| |----> Y0 X1 ---->| hub |----> Y1 X2 ---->| |----> Y2 +-------+
Pipes are point-to-point devices in plumb, any splitting or merging of data streams require a process that can accept multiple pipes. hub can accept any amount of input and output pipes and will copy data from any sink (input) pipes to all source (output pipes).Since the number of attached pipes can be modified on the fly, hub is sometimes created with only a single input and a single output. This lets the script to tap the pipe hub is sitting on, or to inject data into the flow. Another reason for using a single-in/single-out hub is to create a transparent copy process that can have process properties set.
eof handling
Normally when an eof is received on any of the pipes, eof will be delivered to all pipes and hub stops.If argument eof=ignore is passed, hub will ignore eof received on any of the pipes, while eof=ignore-on-source and eof=ignore-on-sink will cause ignoring eof delivered only on source or sink FDs (one-way averting of eof).
blocking/flow control
Default.buffering
None.
Multiple processes writing standard output (see also: eof handling):stdout=[hub] | env:1 {proc1} | stdout:* {proc2} | stdout:* [timer repeat=2 period=1] | stdout:*Multiple processes reading stdin (see also: eof handling):
env:0 | stdin=[hub] stdin:* | {proc1} stdin:* | {proc2}Multiple processes reading stdin, and a timer is also injected pretending timer events are from stdin so processes don't need to read multiple file descriptors:
env:0 | stdin=[hub] stdin:* | {proc1} stdin:* | {proc2} [timer repeat=0 period=1] | stdin:*Named hub "ipc" in the middle of a loop setup, to allow tapping/injection later:
env:0 | stdin=[hub] | {proc1} | ipc=[hub] | {proc2} | stdout=[hub] | env:1 # loop back data stdout:* | stdin:*hub can be used to get a pipeline sticky. Since only processes can have the sticky property, the following script would not work (and quit immediately after start) without the inserted hub:
# emulate cat env:0 | [hub],sticky=1 | env:1Since /dev/null is not fully portable, sometimes a discard virtual process is needed in the script; hub can do that as well. Take the above example: read all input without printing anything. A possible use is to detect eof on stdin, an event is generated by plumb core, which may be caught by another part of the script. The discard example:
# read and discard all input until eof on stdin env:0 | [hub],sticky=1In the following example multiple processes are writing stdout of plumb through a hub called stdout. Normally if any of the processes would emit an eof, that would immediately close the hub, back-promoting the eof to all other processes and also closing env/1. In many cases it's useful to ignore the eof on the output of some processes to prevent closing the hub: proc1 and proc3 will stop writing but will not cause hub to die, while an eof on proc2's stdout will cause everything to shut down.
stdout=[hub] | env/1 {proc1} | [hub eof=ignore-on-sink] | stdout/* {proc2} | stdout/* {proc3} | [hub eof=ignore-on-sink] | stdout/*The next example demonstrates the other way around: there is an stdout hub at the end of the tree and when proc1 sends out an eof it is shut down, and normally it would also shut down p2 and proc2 through eof-back-promotion. Since p2 is also writing env/2, we'd like to keep proc2 running until its own eof, while stdout should be closed by the eof from proc1. This is achieved by blocking eof-back-promotion between p2 and stdout. Note: stdout is closed when either proc1 or proc2 sends eof; proc1 is closed when proc2 sends eof.
stdout=[hub] | env/1 {proc1} | stdout/* {proc2} | p2=[hub] | [hub eof=ignore-on-source] | stdout/* p2/* | env/2If proc2 should be fully separated on eof-level, the following script shall be used. In this eof on proc1 will shut down stdout, but will not affect p2 or proc2, while an eof on proc2 will shut down p2 but will not affect stdout or proc1.
stdout=[hub] | env/1 {proc1} | stdout/* {proc2} | p2=[hub] | [hub eof=ignore] | stdout/* p2/* | env/2Note: stdio(3plumb) implements stdout and STDOUT [hub]s with and without ignoring eof.
hub 3plumb | 2012-08-13 |
---|