SharedBarrier

class streaming.base.shared.SharedBarrier(filelock_path, shm_name)

A barrier that works inter-process using a filelock and shared memory.

We set the number of processes (and thereby initialize num_exit) on the first time this object is called. This is because the object is created in a per-rank process, and called by worker processes.

Parameters:

• filelock_path (str) – Path to lock file on local filesystem.

• shm_name (str) – Shared memory object name in /dev/shm.

property flag

Get property flag.

Returns:

bool – The flag value.

property lock

Get the file lock of the current process.

A FileLock holds a threading lock, so it cannot be pickled, and filelock 3.31+ refuses to acquire a lock in any process other than the one that created it. So each process creates its own lock on first use, and a lock inherited across fork is replaced.

Returns:

FileLock – The file lock.

property num_enter

Get property num_enter.

Returns:

int – Number of processes that have entered the barrier.

property num_exit

Get property num_exit.

Returns:

int – Number of processes that have exited the barrier.