[BUG: Breaking] RuntimeError: Storage size calculation overflowed

[woct0rdho]

🧨 Describe the Bug

When I'm trying out Marker, it crashes with an error like RuntimeError: Storage size calculation overflowed with sizes=[4308746404676320256]. The number is large and changes every time, so it looks like a memory corruption.

📤 Output Trace / Stack Trace

Click to expand

PS C:\tmp\test_marker> marker_single in.pdf
Recognizing Layout:   0%|                                                                                                                                                                            | 0/23 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Python312\Scripts\marker_single.exe\__main__.py", line 10, in <module>
  File "C:\Python312\Lib\site-packages\click\core.py", line 1485, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\click\core.py", line 1406, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\click\core.py", line 1269, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\click\core.py", line 824, in invoke
    return callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\marker\scripts\convert_single.py", line 38, in convert_single_cli
    rendered = converter(fpath)
               ^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\marker\converters\pdf.py", line 195, in __call__
    document = self.build_document(temp_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\marker\converters\pdf.py", line 182, in build_document
    document = DocumentBuilder(self.config)(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\marker\builders\document.py", line 33, in __call__
    layout_builder(document, provider)
  File "C:\Python312\Lib\site-packages\marker\builders\layout.py", line 56, in __call__
    layout_results = self.surya_layout(document.pages)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\marker\builders\layout.py", line 88, in surya_layout
    layout_results = self.layout_model(
                     ^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\surya\layout\__init__.py", line 51, in __call__
    self.foundation_predictor.prediction_loop(
  File "C:\Python312\Lib\site-packages\surya\foundation\__init__.py", line 848, in prediction_loop
    updated_inputs, outputs = self.decode(
                              ^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\surya\foundation\__init__.py", line 356, in decode
    outputs = self.model(
              ^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\surya\common\surya\__init__.py", line 468, in forward
    outputs = self.decoder(
              ^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\surya\common\surya\decoder\__init__.py", line 504, in forward
    layer_outputs = decoder_layer(
                    ^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\surya\common\surya\decoder\__init__.py", line 302, in forward
    hidden_states, self_attn_weights = self.self_attn(
                                       ^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\torch\nn\modules\module.py", line 1776, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\torch\nn\modules\module.py", line 1787, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\surya\common\surya\decoder\__init__.py", line 188, in forward
    key_states, value_states = past_key_value.update(
                               ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\surya\foundation\cache\dynamic_ops.py", line 83, in update
    return update_fn(
           ^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\surya\foundation\cache\dynamic_ops.py", line 360, in _decode_update
    offsets = torch.arange(max_tokens, device=device).unsqueeze(0)  # [1, max_T]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Storage size calculation overflowed with sizes=[4308746404676320256]
Recognizing Layout:   0%|                                                                                                                                                                            | 0/23 [00:04<?, ?it/s]

⚙️ Environment

Please fill in all relevant details:

• Marker version: 1.10.2
• Surya version: 0.17.1
• Python version: 3.12.10
• PyTorch version: 2.10.0+cu130
• Transformers version: 4.57.6
• Operating System (incl. container info if relevant): Windows 11

📎 Additional Context

In this line

surya/surya/foundation/__init__.py

Line 268 in dd7bf4f

return input_ids, input_boxes, num_new_tokens.squeeze(1)

the cause seems to be that squeeze returns a view. It can be fixed by making a clone:

            return input_ids, input_boxes, num_new_tokens.squeeze(1).clone()