Implementation of Voxel-based vision for SERL#77

Open

nisutte wants to merge 343 commits into

rail-berkeley:mainfrom

This pull request is big! We're only showing the most recent 250 commits

Commits on Apr 4, 2024

changed pressure to be within [0, 1]
nisutte
committed
addd gripper action span
nisutte
committed
minor imporvements
nisutte
committed
added RelativeFrame
nisutte
committed
added Reward Transformer
nisutte
committed
more streamlined reward
nisutte
committed

Commits on Apr 9, 2024

soft actor critic worked with this, so I commit :)
nisutte
committed
added evaluation to SAC actor
nisutte
committed
bugfix
nisutte
committed
namechange
nisutte
committed
namechange
nisutte
committed
new actor and learner config
nisutte
committed
more cleanup when terminated
nisutte
committed
params for boxes with rough surfaces
nisutte
committed
boxes tried out
nisutte
committed

Commits on Apr 11, 2024

higher reset height
nisutte
committed
added step cost and ignore xy cost
nisutte
committed
catch if actor is terminated early
nisutte
committed
set original pose (without shift) as the relative pose
nisutte
committed
pose shift and multiple reset poses implementation
nisutte
committed

Commits on Apr 12, 2024

some controller cleanup
nisutte
committed
bugfix
nisutte
committed
RelativeFrame Wrapper has been customized for Robotiq Env
nisutte
committed
clean a few TODO's
nisutte
committed
Merge branch 'rail-berkeley:main' into develop
nisutte
authored

Commits on Apr 19, 2024

added suction cost
nisutte
committed
new actor and learner params
nisutte
committed
added camera support (not tested yet)
nisutte
committed
Merge remote-tracking branch 'origin/develop' into develop
nisutte
committed

Commits on Apr 23, 2024

name change
nisutte
committed
added camera to environment
nisutte
committed
added camera to environment
nisutte
committed
cleanup
nisutte
committed
force as moving average in the controller (too much fluctuations otherwise)
nisutte
committed
added realtime plotter to env
nisutte
committed
drq tools for the box lifting task
nisutte
committed
new reset pose
nisutte
committed
readme update
nisutte
committed

Commits on Apr 24, 2024

bugfix of RuntimeWarning
nisutte
committed
output encoder definition
nisutte
committed
misc
nisutte
committed

Commits on Apr 25, 2024

Added depth only for realsense cameras
nisutte
committed

Commits on Apr 29, 2024

Commits on May 3, 2024

added depth images as option
nisutte
committed
new configs
nisutte
committed
added running reward info
nisutte
committed
added cost plotter
nisutte
committed
changed grip status representation
nisutte
committed
added wrapper to scale the obs space
nisutte
committed

Commits on May 6, 2024

small optimizations and modified env for SAC backwards compatibility
nisutte
committed

Commits on May 7, 2024

controller optimization (force check)
nisutte
committed
added encoded observation output (through print and parse)
nisutte
committed
try to use mean and std from demo images
nisutte
committed
DRQ TODO's
nisutte
committed
streamlined reward
nisutte
committed

Commits on May 16, 2024

controller corrections
nisutte
committed
backup entropy as in original SAC paper and slight loss change
nisutte
committed
translation scale change
nisutte
committed
small changes to drq policy
nisutte
committed
reward change
nisutte
committed
reward change
nisutte
committed

Commits on May 17, 2024

better lightning conditions for the camera sensor
nisutte
committed
new cameras
nisutte
committed
run that got drq working
nisutte
committed
new observation space definition (rgb, depth, both)
nisutte
committed
new observation space definition (rgb, depth, both)
nisutte
committed
run that got drq working (new loggers)
nisutte
committed
run that make drq work
nisutte
committed
cleanup
nisutte
committed
bugfix
nisutte
committed

Commits on May 21, 2024

depth images implemented in gym environment
nisutte
committed
bugfixes
nisutte
committed
bugfixes
nisutte
committed
obs space bugfix
nisutte
committed

Commits on May 22, 2024

added function to print params & small changes
nisutte
committed
added return for encoder == None
nisutte
committed
shape check in ResNet
nisutte
committed

Commits on May 23, 2024

added some model examination tools
nisutte
committed
cleanup
nisutte
committed
added model examination tools and cleanup
nisutte
committed
new reward computation (orientation & pose cost)
nisutte
committed
added evaluation script
nisutte
committed
bugfix 2
nisutte
committed
examination if trajs==0
nisutte
committed
do not log anymore
nisutte
committed

Commits on May 31, 2024

added env with 5 different boxes
nisutte
committed
depth to pointcloud calculation
nisutte
committed
get camera intrinsics for the calculation of the pointcloud
nisutte
committed

Commits on Jun 5, 2024

added pretrained ResNet18 from pytorch (converted)
nisutte
committed
improved hyperparameter handling
nisutte
committed

Commits on Jun 6, 2024

pretrained is from pytorch
nisutte
committed
do not use statistics stored in batch_stats (too much effort to implement...)
nisutte
committed
controller can somewhat ignore singularities (with human input & time)
nisutte
committed
downgraded RealSense firmware to 5.13
nisutte
committed
new training and actor params for ResNet18
nisutte
committed

Commits on Jun 10, 2024

name change
nisutte
committed
bugfixes
nisutte
committed

Commits on Jun 18, 2024

prototype distance sensor
nisutte
committed
added num keypoints in spatial softmax
nisutte
committed

Commits on Jun 19, 2024

cam change
nisutte
committed
cleanup
nisutte
committed

Commits on Jun 21, 2024

implemented pointcloud fusion & calibration (multi camera setup)
nisutte
committed

Commits on Jun 26, 2024

implemented pointcloud visualization and finalized pointcloud fusion with voxelization
nisutte
committed

Commits on Jun 27, 2024

continuous box change instead of random (better setup, easier to reproduce)
nisutte
committed
some bugfixes regarding pc calibration
nisutte
committed
added new X & Y max range for the robot arm in the final configuration
nisutte
committed
more bugfixes
nisutte
committed

Commits on Jun 28, 2024

Important Relative Env fix (do not use adjoint with matrix with actions)
nisutte
committed
Voxel Grid Env setup (for now too slow, optimize or use gpu)
nisutte
committed
New config file for Box setup
nisutte
committed
new cost config
nisutte
committed
implemented image space dynamic adaption (if multi or single cam)
nisutte
committed
updated config
nisutte
committed
pointcloud and voxel bugfixes
nisutte
committed
new actor learner configs
nisutte
committed
made now Env config to test policy on
nisutte
committed
test boxes at an angle
nisutte
committed
new name for wandb
nisutte
committed
added Observation statistics wrapper to investigate episodes
nisutte
committed

Commits on Jul 2, 2024

no invert
nisutte
committed
bugfix
nisutte
committed
added cost to info stats
nisutte
committed
better structure of cost info
nisutte
committed
cleanup
nisutte
committed
bugfix for starting resetq
nisutte
committed
implemented grey camera mode
nisutte
committed
grey bugfix
nisutte
committed
grey bugfix 2
nisutte
committed
range sensor bugfix
nisutte
committed
move away from open3d and do pointcloud fusion and voxelization in pure numpy (except for visualization)
nisutte
committed

Commits on Jul 8, 2024

new implementation of VoxNet encoder (conv3d on downsampled voxelgrid)
nisutte
committed
some slight improvements
nisutte
committed

Commits on Jul 10, 2024

added all the costs and total cost
nisutte
committed
slight change in voxel dim and visualization
nisutte
committed
improved memory efficiency of pointcloud data
nisutte
committed
small improvements
nisutte
committed

Commits on Jul 11, 2024

added voxel augmentation (3d shift)
nisutte
committed
do not use bool compacting anymore (interferes with augmentation & no significant speedup)
nisutte
committed
bugfix and new gripper release fix
nisutte
committed
changing pressure and grip status (more distinct)
nisutte
committed
16 features are enough (faster training)
nisutte
committed

Commits on Jul 15, 2024

bugfix with releasing the gripper
nisutte
committed

Commits on Jul 16, 2024

Commits on Jul 17, 2024

cleanup
nisutte
committed
dropout successful run
nisutte
committed
added voxnet final activation choice
nisutte
committed

Commits on Jul 18, 2024

tests done
nisutte
committed
new PointCloudFusion starting parameters
nisutte
committed

Commits on Jul 19, 2024

implemented 4-fold voxel augmentation (randomly rotate the state, voxel grid and action n*90 degrees)
nisutte
committed
augmentation bugfixes
nisutte
committed
reset position bugfix for controller
nisutte
committed

Commits on Jul 22, 2024

transform force and torque to relative frame as well
nisutte
committed
rlds path fix
nisutte
committed
action needs to be roatated as well in augmentation
nisutte
committed
voxel rotation needs to be clockwise (tcp frame is upside down)
nisutte
committed
drq bugfix (actions were not updated, stupid me...)
nisutte
committed

Commits on Jul 23, 2024

deactivate batch rotation for now
nisutte
committed

Commits on Jul 24, 2024

adjoint matrix should not be used at all, bug found after some velocity orientation tests
nisutte
committed
added a scale factor to scale voxel input in between [0, 1]
nisutte
committed
transform the velocity relative to the start rotation and not the current one
nisutte
committed
implemented real rot90 (orientation rotation is different because of euler "xyz" angles)
nisutte
committed
bugfixes and TODO removal
nisutte
committed
bugfixe 2
nisutte
committed

Commits on Jul 25, 2024

consider observation and action scaling in augmentation
nisutte
committed
consider observation and action scaling in augmentation
nisutte
committed
bugfixes for augmentation
nisutte
committed
added basic PointNet Encoder (not tested yet)
nisutte
committed

Commits on Jul 26, 2024

NaN bugfix
nisutte
committed
bugfix to prevent starting velocities
nisutte
committed

Commits on Jul 31, 2024

Commits on Aug 6, 2024

Commits on Aug 8, 2024

Commits on Aug 9, 2024

added observation rotation wrapper
nisutte
committed
some cleanup
nisutte
committed
bugfix
nisutte
committed
added a state mask to easily ignore observation states
nisutte
committed
re_added force and torque but ignore them with state mask
nisutte
committed

Commits on Aug 12, 2024

Added layernorm params and bugfix
nisutte
committed

Commits on Aug 13, 2024

cleanup & small improvements
nisutte
committed
added new flags for clearer policy definition
nisutte
committed
name change
nisutte
committed
new VoxNet architecture
nisutte
committed

Commits on Aug 14, 2024

final VoxNet hyperparams (not pretrained)
nisutte
committed

Commits on Aug 15, 2024

name change and bugfixes
nisutte
committed
some bugfixes in the augmentation handling
nisutte
committed
added the current action to the observation
nisutte
committed

Commits on Aug 20, 2024

random rotation in x y and z, triangular (so probability is higher near 0)
nisutte
committed
bugfixes and cleanup
nisutte
committed
added trajectory save and some minor changes
nisutte
committed
IMPORTANT CHANGE: proprio-latent-dim is not used anymore, substantial performance decreases if any Dense Layer is used in EncodingWrapper
nisutte
committed
bash files for the different experiments
nisutte
committed
deactivate evaluation if period==0
nisutte
committed
created demos for the experiment (rgb, depth and pointcloud)
nisutte
committed
cleaned some inconsistencies and created policy with only VoxNet
nisutte
committed
name change
nisutte
committed
change costs to be positive (better plots)
nisutte
committed

Commits on Aug 22, 2024

implemented behavior tree and behavioral cloning
nisutte
committed
new runfiles for the experiment
nisutte
committed
encoder kwargs bugfix
nisutte
committed
wrong name in config file
nisutte
committed

Commits on Aug 23, 2024

cleanup
nisutte
committed
implemented Temporal Action Ensemble for smoother action sequences in the evaluation
nisutte
committed
added evaluation bash files
nisutte
committed
added wandb evaluation logger for convenience
nisutte
committed
bugfixes
nisutte
committed
fix the seed for the evaluation
nisutte
committed

Commits on Aug 24, 2024

preolad bias as well
nisutte
committed
slight changes and bugfixes
nisutte
committed

Commits on Aug 28, 2024

bugfix
nisutte
committed
new config for evaluation on unseen boxes
nisutte
committed
new scale for depth images
nisutte
committed
use small encoder for depth images
nisutte
committed

Commits on Sep 9, 2024

random without numpy to not mess with seeded run
nisutte
committed
final experiment files
nisutte
committed
send eval metrics to wandb
nisutte
committed
seed change
nisutte
committed
final config
nisutte
committed
bt and bc changes to evaluate
nisutte
committed
results calculation to latex table from experiments
nisutte
committed

Commits on Oct 4, 2024

Readme update
nisutte
committed

Commits on Oct 9, 2024

name change from robotiq to ur and general cleanup
nisutte
committed
readme updates
nisutte
committed
fix
nisutte
committed

Commits on Oct 16, 2024

bugfix
nisutte
committed

Commits on Oct 22, 2024

cleanup for potential pr
nisutte
committed
upstream merge
nisutte
committed

Commits on Oct 25, 2024

cleanup
nisutte
committed
more cleanup
nisutte
committed