[BUG] %%pretty fails to render strings containing '++'

[RomanKosovets]

Describe the bug
When using the %%pretty magic command in Jupyter Notebook, strings containing two consecutive plus signs (++) are not displayed. Spark reads and filters the data correctly, but %%pretty fails to render these strings

To Reproduce
Start a Jupyter Notebook with SparkMagic. Run the following code:

%%pretty
spark.createDataset(Seq("abc++def", "normalString")).toDF("c").show(false)

Screenshots
result with ++

result with +

result without %%pretty

Versions:

• SparkMagic — 0.23.0
• Livy (if you know it)
• Spark — 3.5.1 / 3.3.0
• PySpark — 3.5.1 / 3.3.0

Additional context
The problem seems to be in %%pretty rendering logic. Single + or other characters work fine, the problem appears only with ++

[dosubot]

Hi @RomanKosovets! I'm Dosu and I’m helping the sparkmagic team.

This issue happens because the %%pretty rendering logic parses the ASCII table output from Spark DataFrames by looking for '+' characters to determine column boundaries. If your data contains '++', the parser mistakes it for a column separator, not part of the data, so the value gets dropped or misparsed. There's no special handling or escaping for '++' in the current implementation—it's a limitation of how the parser splits columns based on the ASCII table format, not the actual data content. The relevant logic is in the DataframeHtmlParser, which slices rows using the positions of '+' in the header line and expects all rows to match the header's width, so any deviation (like extra '+' in data) can break the output [source].

There aren't any documented workarounds in the code or issues. For now, avoiding '++' in string data is the only way to prevent this rendering bug with %%pretty.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}