Develop a function to split text exceeding 200 characters

[parikp06]

Feature Idea

Purpose

Split text exceeding 200 characters into multiple variables.
SDTMIG v3.3 | CDISC

Functionality

Based on the current requirement for the SAS v5 Transport file format, it is not possible to store the long text strings exceeding 200 characters using only one variable. Therefore, the SDTMIG has defined conventions for storing long text string using multiple variables. Follow the steps mentioned below to create a function to split long text into multiple variables.

1. Split the first 200 characters of text into split_var1 and each additional 200 characters of text into split_varN

2. When splitting a text string into several variables, the text should be split between words to improve readability, so it does not break a word.

3. Create a variable split_count to hold the number of new variables created by the split.

4. Default length to split on would be 200. However, please allow this value can be passed via split_length parameter to any other length.

Relevant Input

A character variable to be split into multiple variables.

An integer parameter split_length which would be set to 200 by default.

Relevant Output

One or more variables after splitting: split_var1 – split_varN.

Variable to hold the number of splits: split_count

Reproducible Example/Pseudo Code

split_vars(var_to_split,split_length=200)

[kamilsi]

Acknowledging the historical context of SAS V5 transport file restrictions, I'd like to point out that these limitations don't apply in R, offering us a chance to opt-out. However, my experience with submissions is limited, and I'm seeking clarity: are we considering this feature to meet current regulatory expectations, or is it more about maintaining legacy compatibility?

[parikp06]

Agree that SAS V5 limitations do not apply to R dataframes. However, we still want to have this feature to be compliant with SDTMIG rules. I am also lacking submission experience with R packages, so looking for more feedback.

[pendingintent]

The submission SAS files still have the V5 limitations, which include no string can exceed 200 characters in length. Even if the xpt files are generated using R, the SASV5 limitations still apply unfortunately.

[pendingintent]

I had a look at this in SQL and if we consider a text field mapping to DSTERM for example where the field can be a maximum of 600 characters and we cannot split words, the formula for calculating the max number of required output variables is:
(# required target variables) = CEILING(length source string / 200) + 1

The first (up to) 200 characters maps to DSTERM, with the remainder of the source string mapping to the QNAM variables DSTERM1, DSTERM2, DSTERM3.

I have used the SQL code:
dsterm := upper(substr(input_string, 1, instr(substr(input_string, 1, 200), ' ', -1, 1 )));

This will select up to 200 characters without splitting words and insert into the DSTERM DS variable. Similar statements can then be used to select the other strings for each of the subsequent supplemental QVAL values.

[kamilsi]

Hi @parikp06 and @pendingintent,

I appreciate the clarification on the compliance with SDTMIG rules and the current submission requirements. Given this context, I'm ready to take on the development of this feature. I'll await any additional feedback or a decision on moving forward with this approach.

[MaguluriVenkata]

• it's not clear where SPLITVARN will be stored in the specs? what's the plan?

[parikp06]

This function will be called within a dataframe with input variable to be split. Upon successful execution, it should generate SplitVarN variables along with SplitCount to the output dataframe.

[MaguluriVenkata]

So it does not push SPLITVARn into SUPP domains?

[parikp06]

no, it'll just create required list of variables. SplitCount will help in determining how many SUPP variables to be populated.