Add already_sharded option to skip double-sharding of DataLoaders

[devangpratap]

What does this PR do?

When a user already shards their DataLoader across processes (for example with a rank-aware DistributedSampler), Accelerator.prepare shards it a second time. The data ends up split twice, so each process only iterates a fraction of the samples it should. This is a recurring report: #3520, #4062, #4075.

This adds an already_sharded option to DataLoaderConfiguration (threaded through to prepare_data_loader). When set, Accelerate assumes each process's DataLoader is already sharded and skips its own sharding, while still keeping the rest of the wrapper behaviour: device placement, set_epoch forwarding, and dataloader state tracking.

from accelerate import Accelerator
from accelerate.utils import DataLoaderConfiguration

accelerator = Accelerator(
    dataloader_config=DataLoaderConfiguration(already_sharded=True)
)

sampler = DistributedSampler(dataset, num_replicas=accelerator.num_processes, rank=accelerator.process_index)
dataloader = DataLoader(dataset, batch_size=batch_size, sampler=sampler)
dataloader = accelerator.prepare(dataloader)  # not split a second time

already_sharded=True is rejected together with dispatch_batches=True or split_batches=True, since both conflict with a dataloader that is already split per process. Accelerate's even_batches padding is also skipped in this mode, so the user is responsible for each process iterating the same number of batches.

Fixes #4075 (also addresses #3520 and #4062).

Before submitting

• Did you read the contributor guideline, Pull Request section?
• Was this discussed/approved via a Github issue? Linked above ([Feature request] Support already-sharded DataLoaders in Accelerator.prepare #4075).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

Who can review?

@SunMarc