Development - new features and changes over last months, mainly new TF training script#66
Development - new features and changes over last months, mainly new TF training script#66marcinpaluch1994 wants to merge 129 commits intomasterfrom
Conversation
(cherry picked from commit 5f2cb2d)
…l on physical one.
…l on physical one.
…ller, along old trajectories.
…iment_3_04_2025 # Conflicts: # others/prepare_standard_experiment_folder.py
…neral state utilities.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
| high = np.array( | ||
| [ | ||
| self.theta_threshold_radians * 2, | ||
| self.angle_limit * 2, |
There was a problem hiding this comment.
Undefined angle_limit attribute crashes Cartpole_OpenAI initialization
High Severity
Cartpole_OpenAI.__init__ references self.angle_limit to build the observation space, but this attribute is never defined on the class. The old self.theta_threshold_radians was removed without replacing it with a self.angle_limit assignment. This causes an AttributeError crash whenever Cartpole_OpenAI is instantiated (e.g., when cartpole_type="openai" is selected in CartPoleEnv).
| s[POSITION_IDX+1]= vel | ||
|
|
||
| self.steps_beyond_terminated = None | ||
| return super().init_state(rng) |
There was a problem hiding this comment.
StabilizationOpenAI.init_state discards its constructed state
Medium Severity
StabilizationOpenAI.init_state carefully builds a near-upright state s with small angles and velocities (matching classic OpenAI CartPole specs), but then returns super().init_state(rng) which constructs and returns a different state with full-circle random angles. The locally built s is completely discarded, so episodes start with random angles instead of the intended near-upright configuration.
3 participants
Note
High Risk
High risk because it changes core CartPole dynamics/actuation plumbing (new neural-model stepping and acceleration modes, updated
Q2u, renamed noise parameters) and adds a new ZeroMQ-based remote control path, all of which can materially alter simulation fidelity and runtime behavior.Overview
Adds configurable dynamics execution paths to the simulator: CartPole can now advance state via
next_step_mode(Eulervs neural net) and compute accelerations viasecond_derivatives_mode(ODE, neural-only, or ODE+NN residual), with updatedQ2u(Q, u_max)usage throughout.Reworks control disturbance injection by replacing
add_control_noise/controlDisturbance*with a statefulControlNoiseGeneratorbacked by a new time-step-invariantCorrelatedNoiseGenerator, and renames exported parameters tocontrolNoiseScale/Bias/Correlation/controlNoise_mode.Introduces a new
cartpole_ekf.py(EKF + adaptive Q/R tuner with YAML persistence + live reload), updates CSV logging/header metadata, tweaks MPC cost/optimizer configs, and adds an additional neural controller (controller_difflg) plus bundled model artifacts.Modernizes the Gym env by decoupling physics from tasks and rendering: adds
Taskregistry (tasks.py), aPygameViewer, aCartPoleSimulatorBaseinterface, replaces the legacy sensors sim withCartpole_CustomSim, and addsCartpole_Remoteto run the environment against physical hardware over ZeroMQ.Written by Cursor Bugbot for commit 95c12c2. This will update automatically on new commits. Configure here.