Skip to content

feat(linter): add automated migration for gsutil to gcloud storage #4503

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
amiragamalyassin wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat(linter): add automated migration for gsutil to gcloud storage #4503

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3d5860c
feat(linter): add automated migration for gsutil to gcloud storage
amiragamalyassin Apr 17, 2026
622846f
Implement StorageMigrationRule for gsutil migration
amiragamalyassin Apr 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 26 additions & 4 deletions notebooks/notebook_template_review.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,8 @@ class ErrorCode(Enum):
# H1 heading required
# git, colab and workbench link required
# links must be valid links
# Storage migration rules
ERROR_GSUTIL_DEPRECATED = 103, # Warning for deprecated gsutil usage
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The error code ERROR_GSUTIL_DEPRECATED is defined but not used anywhere in the script. To properly implement this feature, you should add logic to one of the NotebookRule classes to detect gsutil usage and report this error.

ERROR_TITLE_HEADING = 1,
ERROR_HEADING_CASE = 2,
ERROR_HEADING_CAP = 3,
Expand Down Expand Up @@ -1309,12 +1311,14 @@ def replace_cl(text : str ) -> str:
'Vertex AI': '{{vertex_ai_name}}',
'Ray on Vertex AI': '{{ray_vertex_ai_name}}',
'Google Cloud console': '{{console_name}}',

'Cloud Storage': '{{storage_name}}',
'GCS': '{{storage_name}}',
'GCP': '{{gcp_name}}',
'TensorFlow Enterprise': '{{tf4gcp_name}}',
'TensorFlow': '{{tensorflow_name}}',
'gsutil ': 'gcloud storage ',
'!gsutil ': '!gcloud storage ',
'! gsutil ': '! gcloud storage ',
Comment on lines +1319 to +1321
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The replace_cl function is currently only used for generating the discoverability index (add_index) and does not affect the actual notebook content. If the goal is to automatically migrate gsutil commands in the notebooks, these substitutions need to be applied to the notebook cells during the validation/fix process. Also, consider if the trailing spaces in the keys (e.g., 'gsutil ') might miss cases where the command is followed by a newline or tab.

}

for key, value in substitutions.items():
Expand All @@ -1339,7 +1343,25 @@ def replace_backtick(text: str) -> str:
return updated_text



class StorageMigrationRule(NotebookRule):
def validate(self, notebook: Notebook) -> bool:
"""
Check for deprecated gsutil commands and migrate to gcloud storage.
"""
ret = True
for cell in notebook._cells:
if cell['cell_type'] == 'code':
for i, line in enumerate(cell['source']):
if 'gsutil' in line:
# 1. التبليغ عن الخطأ باستخدام الكود (103)
notebook.report_error(ErrorCode.ERROR_GSUTIL_DEPRECATED, f"Deprecated gsutil usage: {line.strip()}")

# 2. الإصلاح التلقائي في محتوى الخلية
if args.fix:
if notebook.report_fix(FixCode.FIX_GSUTIL_TO_GCLOUD, "Migrating gsutil to gcloud storage"):
cell['source'][i] = line.replace('gsutil', 'gcloud storage')
ret = False
return ret
# Instantiate the rules
copyright = CopyrightRule()
notices = NoticesRule()
Expand All @@ -1361,11 +1383,11 @@ def replace_backtick(text: str) -> str:
enableapis = EnableAPIsRule()
setupproject = SetupProjectRule()

# Cell Validation
# Cell Validation
rules = [ copyright, notices, title, links, testenv, table, overview, objective,
recommendations, dataset, costs, setuplocal, helpers,
installation, restart, versions, beforebegin, enableapis,
setupproject
setupproject, storage_migration # <--- السطر ده هو اللي هيفعل القاعدة الجديدة
]

if args.web:
Expand Down