API¶
delayed |
Wraps a function or object to produce a Delayed . |
compute (\*args, \*\*kwargs) |
Evaluate more than one Delayed at once. |
-
dask.delayed.
delayed
()¶ Wraps a function or object to produce a
Delayed
.Delayed
objects act as proxies for the object they wrap, but all operations on them are done lazily by building up a dask graph internally.Parameters: obj : object
The function or object to wrap
name : string or hashable, optional
The key to use in the underlying graph for the wrapped object. Defaults to hashing content.
pure : bool, optional
Indicates whether calling the resulting
Delayed
object is a pure operation. If True, arguments to the call are hashed to produce deterministic keys. Default is False.nout : int, optional
The number of outputs returned from calling the resulting
Delayed
object. If provided, theDelayed
output of the call can be iterated intonout
objects, allowing for unpacking of results. By default iteration overDelayed
objects will error.Examples
Apply to functions to delay execution:
>>> def inc(x): ... return x + 1
>>> inc(10) 11
>>> x = delayed(inc, pure=True)(10) >>> type(x) == Delayed True >>> x.compute() 11
Can be used as a decorator:
>>> @delayed(pure=True) ... def add(a, b): ... return a + b >>> add(1, 2).compute() 3
delayed
also accepts an optional keywordpure
. If False (default), then subsequent calls will always produce a differentDelayed
. This is useful for non-pure functions (such astime
orrandom
).>>> from random import random >>> out1 = delayed(random, pure=False)() >>> out2 = delayed(random, pure=False)() >>> out1.key == out2.key False
If you know a function is pure (output only depends on the input, with no global state), then you can set
pure=True
. This will attempt to apply a consistent name to the output, but will fallback on the same behavior ofpure=False
if this fails.>>> @delayed(pure=True) ... def add(a, b): ... return a + b >>> out1 = add(1, 2) >>> out2 = add(1, 2) >>> out1.key == out2.key True
The key name of the result of calling a delayed object is determined by hashing the arguments by default. To explicitly set the name, you can use the
dask_key_name
keyword when calling the function:>>> add(1, 2) Delayed('add-3dce7c56edd1ac2614add714086e950f') >>> add(1, 2, dask_key_name='three') Delayed('three')
Note that objects with the same key name are assumed to have the same result. If you set the names explicitly you should make sure your key names are different for different results.
>>> add(1, 2, dask_key_name='three') >>> add(2, 1, dask_key_name='three') >>> add(2, 2, dask_key_name='four')
delayed
can also be applied to objects to make operations on them lazy:>>> a = delayed([1, 2, 3]) >>> isinstance(a, Delayed) True >>> a.compute() [1, 2, 3]
The key name of a delayed object is hashed by default if
pure=True
or is generated randomly ifpure=False
(default). To explicitly set the name, you can use thename
keyword:>>> a = delayed([1, 2, 3], name='mylist') >>> a Delayed('mylist')
Delayed results act as a proxy to the underlying object. Many operators are supported:
>>> (a + [1, 2]).compute() [1, 2, 3, 1, 2] >>> a[1].compute() 2
Method and attribute access also works:
>>> a.count(2).compute() 1
Note that if a method doesn’t exist, no error will be thrown until runtime:
>>> res = a.not_a_real_method() >>> res.compute() AttributeError("'list' object has no attribute 'not_a_real_method'")
Methods are assumed to be impure by default, meaning that subsequent calls may return different results. To assume purity, set pure=True. This allows sharing of any intermediate values.
>>> a.count(2, pure=True).key == a.count(2, pure=True).key True
As with function calls, method calls also support the
dask_key_name
keyword:>>> a.count(2, dask_key_name="count_2") Delayed("count_2")
-
dask.delayed.
compute
(*args, **kwargs)¶ Evaluate more than one
Delayed
at once.Note that the only difference between this function and
dask.base.compute
is that this implicitly wraps python objects inDelayed
, allowing for collections of dask objects to be computed.Examples
>>> a = delayed(1) >>> b = a + 2 >>> c = a + 3 >>> compute(b, c) # Compute both simultaneously (3, 4) >>> compute(a, [b, c]) # Works for lists of Delayed (1, [3, 4])