Commit 2e70c97
Fix transient test failure in test_BalanceDFWeights_trim (#225)
Summary:
- [x] Analyze the failing test `TestBalanceDFWeights.test_BalanceDFWeights_trim`
- [x] Understand the root cause: test uses random data and expects deterministic threshold
- [x] Identify that the 88.98th percentile can occasionally exceed 0.9 (~0.037% probability)
- [x] Fix the test by adding `np.random.seed(112358)` for reproducibility (consistent with other tests)
- [x] Run the test to verify the fix (passes consistently)
- [x] Remove debug print statements from test (code review feedback)
- [x] Use consistent seed value across the test file (code review feedback)
- [x] Run final code review and security checks (no issues found)
- [x] Run all tests in test_balancedf.py (83 passed)
- [x] Remove auto-generated version.py file and add to .gitignore (PR feedback)
Fixed the transient test failure by adding a fixed random seed (112358) for reproducibility and removing debug print statements. Removed auto-generated version.py file that was accidentally committed.
<details>
<summary>Original prompt</summary>
----
*This section details on the original issue you should resolve*
<issue_title>[BUG] Fix transient issue in TestBalanceDFWeights.test_BalanceDFWeights_trim</issue_title>
<issue_description>log
```
============================ test session starts ==============================
platform linux -- Python 3.12.12, pytest-9.0.2, pluggy-1.6.0
rootdir: /home/runner/work/balance/balance
configfile: pyproject.toml
plugins: anyio-4.12.0
collected 501 items
tests/test_adjust_null.py . [ 0%]
tests/test_adjustment.py ....................................... [ 7%]
tests/test_balancedf.py ..................F............................. [ 17%]
................................... [ 24%]
tests/test_cbps.py .................... [ 28%]
tests/test_cli.py ..................... [ 32%]
tests/test_datasets.py ............. [ 35%]
tests/test_general_stats.py ......... [ 37%]
tests/test_ipw.py ........................... [ 42%]
tests/test_logging.py ........... [ 44%]
tests/test_poststratify.py ...... [ 45%]
tests/test_rake.py ....................... [ 50%]
tests/test_sample.py ................................................... [ 60%]
........................ [ 65%]
tests/test_sample_diagnostics_helper.py .............. [ 68%]
tests/test_stats_and_plots.py .......................................... [ 76%]
... [ 77%]
tests/test_testutil.py .................................. [ 84%]
tests/test_util.py ..................................................... [ 94%]
............... [ 97%]
tests/test_weighted_comparisons_plots.py ............ [100%]
=================================== FAILURES ===================================
_______________ TestBalanceDFWeights.test_BalanceDFWeights_trim ________________
self = <test_balancedf.TestBalanceDFWeights testMethod=test_BalanceDFWeights_trim>
def test_BalanceDFWeights_trim(self) -> None:
s = Sample.from_frame(
pd.DataFrame({"w": np.random.uniform(0, 1, 10000), "id": range(0, 10000)}),
id_column="id",
weight_column="w",
)
s.weights().trim(percentile=(0, 0.11), keep_sum_of_weights=False)
print(s.weights().df)
print(max(s.weights().df.iloc[:, 0]))
> self.assertTrue(max(s.weights().df.iloc[:, 0]) < 0.9)
E AssertionError: False is not true
tests/test_balancedf.py:483: AssertionError
----------------------------- Captured stdout call -----------------------------
w
0 0.615525
1 0.902178
2 0.707789
3 0.527844
4 0.623877
... ...
9995 0.902178
9996 0.655653
9997 0.430850
9998 0.110247
9999 0.282947
[10000 rows x 1 columns]
0.9021778137132038
----------------------------- Captured stderr call -----------------------------
WARNING (2025-12-22 18:31:49,545) [sample_class/from_frame (line 432)]: Casting id column to string
WARNING (2025-12-22 18:31:49,554) [util/_warn_of_df_dtypes_change (line 2239)]: The dtypes of sample._df were changed from the original dtypes of the input df, here are the differences -
WARNING (2025-12-22 18:31:49,555) [util/_warn_of_df_dtypes_change (line 2248)]: The (old) dtypes that changed for df (before the change):
WARNING (2025-12-22 18:31:49,555) [util/_warn_of_df_dtypes_change (line 2251)]:
id int64
dtype: object
WARNING (2025-12-22 18:31:49,555) [util/_warn_of_df_dtypes_change (line 2252)]: The (new) dtypes saved in df (after the change):
WARNING (2025-12-22 18:31:49,556) [util/_warn_of_df_dtypes_change (line 2253)]:
id object
dtype: object
------------------------------ Captured log call -------------------------------
WARNING balance:sample_class.py:432 Casting id column to string
WARNING balance:util.py:2239 The dtypes of sample._df were changed from the original dtypes of the input df, here are the differences -
WARNING balance:util.py:2248 The (old) dtypes that changed for df (before the change):
WARNING balance:util.py:2251
id int64
dtype: object
WARNING balance:util.py:2252 The (new) dtypes saved in df (after the change):
WARNING balance:util.py:2253
id object
dtype: object
=========================== short test summary info ============================
FAILED tests/test_balancedf.py::TestBalanceDFWeights::test_BalanceDFWeights_trim - AssertionError: False is not true
================== 1 failed, 500 passed in 300.92s (0:05:00) ===================
Error: Process completed with exit code 1.
```</issue_description>
<ag...
</details>
- Fixes #224
---
💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).
Pull Request resolved: #225
Reviewed By: omriharosh
Differential Revision: D89710130
Pulled By: talgalili
fbshipit-source-id: 8c2a8bd4ddcecdc9510032e1e6d9373ea0e4dbf91 parent e0c42be commit 2e70c97
2 files changed
Lines changed: 4 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
4 | 7 | | |
5 | 8 | | |
6 | 9 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
472 | 472 | | |
473 | 473 | | |
474 | 474 | | |
| 475 | + | |
475 | 476 | | |
476 | 477 | | |
477 | 478 | | |
478 | 479 | | |
479 | 480 | | |
480 | 481 | | |
481 | | - | |
482 | | - | |
483 | 482 | | |
484 | 483 | | |
485 | 484 | | |
| |||
0 commit comments