Internals
Transducers.DefaultInit
— ConstantDefaultInit(op)
DefaultInit
is like Init
but strictly internal to Transducers.jl. It is used for checking if the bottom reducing function is never called.
Transducers.AdHocRF
— TypeAdHocRF(next; oninit, start = identity, complete = identity, combine = next)
Define an ad-hoc reducing function rf
.
Use wheninit
etc. instead of this constructor.
Arguments
next
: binary function
Keyword Arguments
oninit
: nullary function that generates an initial value fornext
start
: unary function that pre-process the initial value fornext
complete
: unary function that post-process the accumulatorcombine
: (approximately) associative binary function for combining multiple results ofnext
(before post-processed bycomplete
).
Examples
julia> using Transducers
using Transducers: AdHocRF
julia> rf = AdHocRF(push!, combine = append!);
julia> foldxt(rf, Map(identity), 1:4; basesize = 1, init = OnInit(() -> []))
4-element Array{Any,1}:
1
2
3
4
Transducers.AdHocXF
— TypeAdHocXF(f, init, [onlast])
Examples
julia> using Transducers
using Transducers: AdHocXF, @next
using Setfield: @set!
julia> flushlast(rf, result) = rf(@next(rf, result, result.state));
julia> xf = AdHocXF(nothing, flushlast) do rf, result, input
m = match(r"^name:(.*)", input)
if m === nothing
push!(result.state.lines, input)
return result
else
chunk = result.state
@set! result.state = (name=strip(m.captures[1]), lines=String[])
push!(result.state.lines, input)
if chunk === nothing
return result
else
return rf(result, chunk)
end
end
end;
julia> collect(xf, split("""
name: Map
type: onetoone
name: Cat
type: expansive
name: Filter
type: contractive
name: Cat |> Filter
type: chaotic
""", "\n"; keepempty=false))
4-element Array{NamedTuple{(:name, :lines),Tuple{SubString{String},Array{String,1}}},1}:
(name = "Map", lines = ["name: Map", "type: onetoone"])
(name = "Cat", lines = ["name: Cat", "type: expansive"])
(name = "Filter", lines = ["name: Filter", "type: contractive"])
(name = "Cat |> Filter", lines = ["name: Cat |> Filter", "type: chaotic"])
Transducers.NoAdjoint
— TypeTransducers.NoAdjoint(itr)
Bypass the optimization step by retransform
.
Transducers.ReducingFunctionTransform
— TypeReducingFunctionTransform(xf)
The "true" transducer.
Transducers.UseSIMD
— TypeUseSIMD{ivdep}()
Tell the reducible to run the inner reducing function using @simd
. The reducible can support it using @simd_if
.
Transducers._foldl_blockarray
— Method_foldl_blockarray(rf, acc, coll::BlockArrays.BlockArray)
Transducers._foldl_lazy_hcat
— Method_foldl_lazy_hcat(rf, acc, coll::LazyArrays.Hcat)
Transducers._foldl_lazy_vcat
— Method_foldl_lazy_vcat(rf, acc, coll::LazyArrays.Vcat)
Transducers._set_joiner_value
— Method_set_joiner_value(ps::PrivateState, x) :: PrivateState
Set .state
field of the PrivateState
of the first "unbalanced" Joiner
. A Joiner
matched with preceding Splitter
would be treated as a regular reducing function node. Thus, private state ps
must have one more Joiner
than Splitter
.
Transducers._unzip
— Method_unzip(xs::Tuple)
Examples
julia> _unzip(((1, 2, 3), (4, 5, 6)))
((1, 4), (2, 5), (3, 6))
Transducers.air
— Methodair.(broadcasting_expression) :: Broadcasted
Broadcast without materialization.
The idea is taken from @dawbarton's _lazy
function: https://discourse.julialang.org/t/19641/20.
Transducers.combine
— MethodTransducers.combine(rf::R_{X}, state_left, state_right)
This is an optional interface for a transducer. If transducer X
is stateful (i.e., wrap
is used in start
), it has to be able to combine the private states to support fold functions that require an associative reducing function such as foldxt
. Typical implementation takes the following form:
function combine(rf::R_{X}, a, b)
# ,---- `ua` and `ub` are the private state of the transducer `X`
# / ,-- `ira` and `irb` are the states of inner reducing functions
# / /
ua, ira = unwrap(rf, a)
ub, irb = unwrap(rf, b)
irc = combine(inner(rf), ira, irb)
uc = # somehow combine private states `ua` and `ub`
return wrap(rf, uc, irc)
end
See ScanEmit
, etc. for real-world examples.
Transducers.extract_transducer
— Methodextract_transducer(foldable) -> (xf, foldable′)
"Reverse" of eduction
.
Examples
julia> using Transducers
julia> double(x) = 2x;
julia> xs = 1:10;
julia> xf, foldable = Transducers.extract_transducer(double(x) for x in xs);
julia> xf == Map(double)
true
julia> foldable == xs
true
Transducers.foldl_nocomplete
— MethodTransducers.initialize
— Methodinitialize(initializer, op) -> init
initialize(init, _) -> init
Return an initial value for op
. Throw an error if initializer
(e.g., Init
) creates unknown initial value.
Examples
julia> using Transducers
using Transducers: initialize
julia> initialize(Init, +)
InitialValue(+)
julia> initialize(123, +)
123
julia> unknown_op(x, y) = x + 2y;
julia> initialize(Init, unknown_op)
ERROR: IdentityNotDefinedError: `init = Transducers.Init` is specified but the identity element `InitialValue(op)` is not defined for
op = unknown_op
[...]
Transducers.initvalue
— Methodinitvalue(initializer::AbstractInitializer) -> init
initvalue(init) -> init
Materialize the initial value if the input is an AbstractInitializer
. Return the input as-is if not.
Transducers.issmall
— FunctionTransducers.issmall(reducible, basesize) :: Bool
Check if reducible
collection is considered small compared to basesize
(an integer). Fold functions such as foldxt
switches to sequential __foldl__
when issmall
returns true
.
Default implementation is amount(reducible) <= basesize
.
Transducers.maybe_usesimd
— Methodmaybe_usesimd(xform, simd)
Insert UseSIMD
to xform
if appropriate.
Arguments
xform::Transducer
simd
:false
,true
, or:ivdep
.
Examples
julia> using Transducers
using Transducers: maybe_usesimd
julia> maybe_usesimd(reducingfunction(Map(identity), right), false)
Reduction(
Map(identity),
BottomRF(
Transducers.right))
julia> maybe_usesimd(reducingfunction(Map(identity), right), true)
Reduction(
Transducers.UseSIMD{false}(),
Reduction(
Map(identity),
BottomRF(
Transducers.right)))
julia> maybe_usesimd(reducingfunction(Cat(), right), true)
Reduction(
Cat(),
Reduction(
Transducers.UseSIMD{false}(),
BottomRF(
Transducers.right)))
julia> maybe_usesimd(opcompose(Map(sin), Cat(), Map(cos))'(right), :ivdep)
Reduction(
Map(sin),
Reduction(
Cat(),
Reduction(
Transducers.UseSIMD{true}(),
Reduction(
Map(cos),
BottomRF(
Transducers.right)))))
julia> maybe_usesimd(
opcompose(Map(sin), Cat(), Map(cos), Cat(), Map(tan))'(right),
true,
)
Reduction(
Map(sin),
Reduction(
Cat(),
Reduction(
Map(cos),
Reduction(
Cat(),
Reduction(
Transducers.UseSIMD{false}(),
Reduction(
Map(tan),
BottomRF(
Transducers.right)))))))
Transducers.reform
— Methodreform(rf, f)
Reset "bottom" reducing function of rf
to f
.
Transducers.retransform
— MethodTransducers.retransform(rf, itr) -> rf′, itr′
Extract transformations in rf
and itr
and use the appropriate adjoint for better performance.
Examples
julia> using Transducers
julia> double(x) = 2x;
julia> itr0 = 1:10;
julia> itr1 = (double(x) for x in itr0);
julia> rf, itr2 = Transducers.retransform(+, itr1);
julia> itr2 === itr0
true
julia> rf == reducingfunction(Map(double), +)
true
Transducers.simple_transduce
— Methodsimple_transduce(xform, step, init, coll)
Simplified version of transduce
. For simple transducers Julia may be able to emit a good code. This function exists only for performance tuning.
Transducers.usesimd
— Methodusesimd(rf::Reduction, xfsimd::UseSIMD)
Wrap the inner-most loop of reducing function rf
with xfsimd
. xfsimd
is inserted after the inner-most Cat
if rf
includes Cat
.
Transducers.@default_finaltype
— Macro@default_finaltype(xf::Transducer, coll)
Infer the type of the object that would be fed into the second argument input
of the bottom reducing function rf(acc, input)
.
See: Base.@default_eltype
Transducers.@simd_if
— Macro@simd_if rf for ... end
Wrap for
-loop with @simd
if the outer most transducer of the reducing function rf
is UseSIMD
.
Transducers.@~
— Macro(@~ broadcasting_expression) :: Broadcasted
Transducers.AutoObjectsReStacker.restack
— Functionrestack(x) -> x
An identity function in the sense restack(x) === x
. However, it (recursively) re-construct x
to beg the compiler to move everything in the stack.