Module: vemomoto_core.concurrent.sharedmem_ext#

Created on 07.12.2016

@author: Samuel


MapAsyncResult(fetcher, result, exceptions)

MapReduce([backend, np])


class MapAsyncResult(fetcher, result, exceptions)[source]#

Bases: object



class MapReduce(backend=<class 'sharedmem.sharedmem.ProcessBackend'>, np=None)[source]#

Bases: sharedmem.sharedmem.MapReduce



map_async(func, sequence[, result, reduce, ...])

Map-reduce with multile processes.

map_async(func, sequence, result=[], reduce=None, star=False, minlength=0)[source]#

Map-reduce with multile processes.

Apply func to each item on the sequence, in parallel. As the results are collected, reduce is called on the result. The reduced result is returned as a list.

  • func (callable) –

    The function to call. It must accept the same number of arguments as the length of an item in the sequence.


    func is not supposed to use exceptions for flow control. In non-debug mode all exceptions will be wrapped into a SlaveException.

  • sequence (list or array_like) – The sequence of arguments to be applied to func.

  • reduce (callable, optional) – Apply an reduction operation on the return values of func. If func returns a tuple, they are treated as positional arguments of reduce.

  • star (boolean) – if True, the items in sequence are treated as positional arguments of reduce.

  • minlength (integer) – Minimal length of sequence to start parallel processing. if len(sequence) < minlength, fall back to sequential processing. This can be used to avoid the overhead of starting the worker processes when there is little work.


results – The list of reduced results from the map operation, in the order of the arguments of sequence.

Return type



SlaveException – If any of the slave process encounters an exception. Inspect SlaveException.reason for the underlying exception.