Integrate with TorchRL

[Kaszanas]

Is your proposal related to a problem?

No.

Describe the solution you'd like

Full integration with torchrl as an offline dataset.

Additional context

This is a main issue for integrating with torchrl.

[Kaszanas]

This is not closed yet because it is not clear to me how to input the data as a TensorClass or TensorDict to any training pipeline as of now, but transforming these dataclasses clearly works.