Skip to content

broadcasting unchunked diskarrays with string eltype fails #284

@tiemvanderdeure

Description

@tiemvanderdeure

MWE

using DiskArrays.TestTypes
a = UnchunkedDiskArray(fill("a", (10, 20, 5)))
a .* copy(a)

I'll post the entire stacktrace but what happens is that since Base.BroadcastStyle(typeof(a)) is ChunkStyle, it ends up dispatching on a copyto! that assume at least some of these are chunked - but none of them are. It only errors on types without a definitive size, though. But what I don't really understand is why non-chunked diskarrays need to broadcast as if they are chunked? Isn't the chunking information available at compile time?

ERROR: Type String does not have a definite size.
Stacktrace:
  [1] sizeof(x::Type)
    @ Base .\essentials.jl:783
  [2] call_composed
    @ .\operators.jl:1099 [inlined]
  [3] (::ComposedFunction{typeof(sizeof), typeof(eltype)})(x::Array{String, 3}; kw::@Kwargs{})       
    @ Base .\operators.jl:1096
  [4] MappingRF
    @ .\reduce.jl:92 [inlined]
  [5] afoldl(::Base.MappingRF{…}, ::Base._InitialValue, ::Array{…}, ::UnchunkedDiskArray{…})
    @ Base .\operators.jl:599
  [6] _foldl_impl(op::Base.MappingRF{…}, init::Base._InitialValue, itr::Tuple{…})
    @ Base .\reduce.jl:60
  [7] foldl_impl(op::Base.MappingRF{…}, nt::Base._InitialValue, itr::Tuple{…})
    @ Base .\reduce.jl:40
  [8] mapfoldl_impl(f::ComposedFunction{…}, op::typeof(Base.add_sum), nt::Base._InitialValue, itr::Tuple{…})
    @ Base .\reduce.jl:36
  [9] mapfoldl(f::Function, op::Function, itr::Tuple{Array{…}, UnchunkedDiskArray{…}}; init::Base._InitialValue)
    @ Base .\reduce.jl:167
 [10] mapfoldl
    @ .\reduce.jl:167 [inlined]
 [11] mapreduce
    @ .\reduce.jl:299 [inlined]
 [12] sum(f::Function, a::Tuple{Array{String, 3}, UnchunkedDiskArray{String, 3, Array{String, 3}}})  
    @ Base .\reduce.jl:524
 [13] common_chunks(::Tuple{Int64, Int64, Int64}, ::Array{String, 3}, ::Vararg{Any})
    @ DiskArrays C:\Users\tsh371\.julia\dev\DiskArrays\src\broadcast.jl:65
 [14] copyto!(dest::Array{…}, broadcasted::Base.Broadcast.Broadcasted{…})
    @ DiskArrays C:\Users\tsh371\.julia\dev\DiskArrays\src\broadcast.jl:34
 [15] materialize!
    @ .\broadcast.jl:905 [inlined]
 [16] materialize!
    @ .\broadcast.jl:902 [inlined]
 [17] _disk_copyto!(dest::Array{String, 3}, source::UnchunkedDiskArray{String, 3, Array{String, 3}}) 
    @ DiskArrays C:\Users\tsh371\.julia\dev\DiskArrays\src\array.jl:7
 [18] copyto!(dest::Array{String, 3}, source::UnchunkedDiskArray{String, 3, Array{String, 3}})       
    @ DiskArrays C:\Users\tsh371\.julia\dev\DiskArrays\src\array.jl:70
 [19] copymutable
    @ .\abstractarray.jl:1201 [inlined]
 [20] copy(a::UnchunkedDiskArray{String, 3, Array{String, 3}})
    @ Base .\abstractarray.jl:1144
 [21] top-level scope
    @ c:\Users\tsh371\.julia\dev\DiskArrays\test\runtests.jl:12
Some type information was truncated. Use `show(err)` to see complete types.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions