[SPARK-58214][PYTHON] Consolidate VALUE_NOT_PEARSON into VALUE_NOT_ALLOWED

### What changes were proposed in this pull request?

Consolidate the specialized PySpark `VALUE_NOT_PEARSON` error condition into `VALUE_NOT_ALLOWED`. Classic and Spark Connect `DataFrame.stat.corr` now report the generic condition when `method` is not `pearson`. The Spark Connect statistics test asserts the new condition.

### Why are the changes needed?

`VALUE_NOT_PEARSON` only represents a single allowed argument value and duplicates the generic allowed-values condition. Removing it reduces narrowly scoped error conditions and aligns the validation with other PySpark APIs.

### Does this PR introduce _any_ user-facing change?

Yes. Invalid `DataFrame.stat.corr(..., method=...)` calls now use `VALUE_NOT_ALLOWED` and its generic message. The exception type remains `PySparkValueError`.

### How was this patch tested?

Updated the Spark Connect statistics test assertion. The focused PySpark test suite was not run.

### Was this patch authored or co-authored using generative AI tooling?

Generated-by: Codex (GPT-5)

Closes #57366 from zhengruifeng/value-not-allowed-dev1.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>

Author: zhengruifeng

python/pyspark/errors/error-conditions.json

@@ -1208,11 +1208,6 @@
       "Value for `<arg_name>` must be a non-empty string, got '<arg_value>'."
     ]
   },
-  "VALUE_NOT_PEARSON": {
-    "message": [
-      "Value for `<arg_name>` only supports 'pearson', got '<arg_value>'."
-    ]
-  },
   "VALUE_NOT_PLAIN_COLUMN_REFERENCE": {
     "message": [
       "Value `<val>` in `<field_name>` should be a plain column reference such as `df.col` or `col('column')`."

python/pyspark/sql/classic/dataframe.py

@@ -1693,8 +1693,8 @@ def corr(self, col1: str, col2: str, method: Optional[str] = None) -> float:
             method = "pearson"
         if not method == "pearson":
             raise PySparkValueError(
-                errorClass="VALUE_NOT_PEARSON",
-                messageParameters={"arg_name": "method", "arg_value": method},
+                errorClass="VALUE_NOT_ALLOWED",
+                messageParameters={"arg_name": "method", "allowed_values": "['pearson']"},
             )
         return self._jdf.stat().corr(col1, col2, method)
 

python/pyspark/sql/connect/dataframe.py

@@ -1663,8 +1663,8 @@ def corr(self, col1: str, col2: str, method: Optional[str] = None) -> float:
             method = "pearson"
         if not method == "pearson":
             raise PySparkValueError(
-                errorClass="VALUE_NOT_PEARSON",
-                messageParameters={"arg_name": "method", "arg_value": method},
+                errorClass="VALUE_NOT_ALLOWED",
+                messageParameters={"arg_name": "method", "allowed_values": "['pearson']"},
             )
         table, _ = DataFrame(
             plan.StatCorr(child=self._plan, col1=col1, col2=col2, method=method),

python/pyspark/sql/tests/connect/test_connect_stat.py

@@ -214,13 +214,14 @@ def test_stat_corr(self):
                 "arg_type": "int",
             },
         )
-        with self.assertRaises(ValueError) as context:
-            (self.connect.read.table(self.tbl_name2).stat.corr("col1", "col3", "spearman"),)
-            self.assertTrue(
-                "Currently only the calculation of the Pearson Correlation "
-                + "coefficient is supported."
-                in str(context.exception)
-            )
+        with self.assertRaises(PySparkValueError) as pe:
+            self.connect.read.table(self.tbl_name2).stat.corr("col1", "col3", "spearman")
+
+        self.check_error(
+            exception=pe.exception,
+            errorClass="VALUE_NOT_ALLOWED",
+            messageParameters={"arg_name": "method", "allowed_values": "['pearson']"},
+        )
 
     def test_stat_approx_quantile(self):
         # SPARK-41069: Test the stat.approxQuantile method