Performance: Eliminate redundant data copies and transformations

[anthropic-code-agent]

Description

Remove unnecessary data copying and transformation operations that create performance bottlenecks, especially for large datasets.

Current Implementation Issues

R/MappedData.R - Multiple copy operations

Lines 93-96: Unnecessary data frame copy

• Uses dplyr::select() to filter columns with NA values - creates a copy
• data.frame(data) explicitly creates another copy of input data
• Two copies created in quick succession

Line 110: unique() call

• Creates a copy when checking for unique values
• Could be optimized with early checking

Lines 176-181: Filtering creates new data frames

• dplyr::filter() calls create separate copies for secondary and primary axis data
• Could use data.table reference semantics instead

R/MappedDataTimeProfile.R - Split and recombine pattern

Lines 176-180 and 267

• Data is split into two separate copies via filtering
• Final rbind() at line 267 recombines data that was just split
• Inefficient pattern: split → transform → recombine
• Could transform in place or use more efficient approach

R/plotYVsX.R - Unnecessary cloning

Line 319: Full object clone

• mappedData$clone() creates full deep copy
• Used only for simple filtering operation (lines 321-323)
• Could modify a shallow copy or use different approach

R/MappedDataRangePlot.R - Multiple copies

Line 116-121: Redundant conversions

• copy() from data.table package
• Then converts to data.table again at lines 117, 121
• Multiple format conversions create unnecessary overhead

Suggested Implementation

1. Use data.table reference semantics: Modify data in place with := operator
2. Avoid premature copying: Only copy when absolutely necessary
3. Use subset/filter more efficiently: Access data by reference when possible
4. Refactor split-transform-merge: Transform without splitting when possible
5. Use shallow copies: When cloning is necessary, clone only what is needed

Example Refactoring

Before:

data_copy <- data.frame(data)  # Copy 1
filtered <- dplyr::select(data_copy, ...)  # Copy 2

After:

# Use data.table reference semantics
dt <- as.data.table(data)
dt[, filtered_cols, with = FALSE]  # No copy

Expected Benefits

• Reduced memory usage (can be 50%+ reduction)
• Faster execution for large datasets
• Better scalability
• Fewer garbage collection pauses

Implementation Notes

• Carefully test that reference semantics do not cause unintended side effects
• Document where data is modified in place vs copied
• Ensure R6 object cloning behavior is correct
• Follow OSP R coding standards: https://dev.open-systems-pharmacology.org/r-development-resources/coding_standards_r

Files to Modify

• R/MappedData.R
• R/MappedDataTimeProfile.R
• R/plotYVsX.R
• R/MappedDataRangePlot.R

Testing

• Run existing unit tests
• Add tests to verify data is not modified when it should not be
• Test with large datasets to measure performance improvement
• Verify all plots render correctly