feat(linter): add automated migration for gsutil to gcloud storage#4503
feat(linter): add automated migration for gsutil to gcloud storage#4503amiragamalyassin wants to merge 2 commits intoGoogleCloudPlatform:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a new error code for deprecated gsutil usage and adds substitutions to migrate gsutil commands to gcloud storage. However, several issues were identified: a syntax error was introduced in the substitutions dictionary due to a malformed entry and a duplicate key, the new error code is currently unused, and the gsutil substitutions in the replace_cl function do not yet affect the actual notebook content as intended.
| '{{vertex_nas_name}}', | ||
| 'Neural Architectural Search': '{{vertex_nas_name_short}}', | ||
| 'Vertex Workbench': '{{vertex_workbench_name}}', |
There was a problem hiding this comment.
This change introduces a SyntaxError because line 1305 is a single string literal instead of a key-value pair within the dictionary. Additionally, line 1306 is a duplicate of line 1304. These lines should be removed.
| '{{vertex_nas_name}}', | |
| 'Neural Architectural Search': '{{vertex_nas_name_short}}', | |
| 'Vertex Workbench': '{{vertex_workbench_name}}', | |
| 'Vertex Workbench': '{{vertex_workbench_name}}', |
| 'gsutil ': 'gcloud storage ', | ||
| '!gsutil ': '!gcloud storage ', | ||
| '! gsutil ': '! gcloud storage ', |
There was a problem hiding this comment.
The replace_cl function is currently only used for generating the discoverability index (add_index) and does not affect the actual notebook content. If the goal is to automatically migrate gsutil commands in the notebooks, these substitutions need to be applied to the notebook cells during the validation/fix process. Also, consider if the trailing spaces in the keys (e.g., 'gsutil ') might miss cases where the command is followed by a newline or tab.
| # git, colab and workbench link required | ||
| # links must be valid links | ||
| # Storage migration rules | ||
| ERROR_GSUTIL_DEPRECATED = 103, # Warning for deprecated gsutil usage |
Added StorageMigrationRule to validate and migrate deprecated gsutil commands to gcloud storage.
|
Thank you for the detailed feedback. I have pushed a follow-up commit that addresses all identified issues: |
This PR updates the notebook_template_review.py script to support the ongoing migration from gsutil to gcloud storage within Vertex AI notebooks.
Changes:
Added ERROR_GSUTIL_DEPRECATED (103) to the ErrorCode enum.
Updated the substitutions dictionary in replace_cl to automatically convert deprecated storage commands to the modern gcloud storage CLI.
These updates ensure that all sample notebooks align with current Google Cloud performance standards and best practices.