feat(goldencheetah): ✨ adding GoldenCheetah OSF athlete ZIP import #52#93
Draft
dvorka wants to merge 1 commit into
Draft
feat(goldencheetah): ✨ adding GoldenCheetah OSF athlete ZIP import #52#93dvorka wants to merge 1 commit into
dvorka wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new GoldenCheetah OSF athlete ZIP import path to MyTraL (desktop-only), covering both activity-summary import (from the ZIP’s JSON index) and optional time-series ingestion (CSV → Parquet) so imported activities can show analysis charts.
Changes:
- Introduces
GoldenCheetahOsfImportPluginto parse OSF ZIP JSON indices intoActivityEntityrecords with stablesrc/src_keytracking. - Adds
GoldenCheetahOsfImportTaskto run async import with conflict resolution plus Bulldozer-parallel CSV→blob/Parquet ingestion for Analysis charts. - Wires a new desktop-only import form + UI section, and adds comprehensive tests/fixtures for plugin + task behavior.
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
tests/test_golden_cheetah_osf.py |
New unit + integration coverage for metric parsing, ride mapping, real ZIP import, and task-level blob/parquet ingestion. |
mytral/templates/tools-import.html |
Adds GoldenCheetah OSF import section/link in the desktop import UI. |
mytral/templates/activity-get.html |
Formats avg/max cadence to 1 decimal place in the activity detail view. |
mytral/tasks/do/golden_cheetah_osf_import.py |
New async task implementing import, conflict handling, and Bulldozer CSV→Parquet ingestion into blobstore. |
mytral/recordings/parquet_converter.py |
Extends the recording conversion layer with gc_csv_to_parquet() for GoldenCheetah CSV timeseries. |
mytral/integrations/golden_cheetah_osf.py |
New integration plugin to parse GoldenCheetah OSF athlete ZIP structure and map rides to activities. |
mytral/forms.py |
Adds ImportGoldenCheetahOsfForm (ZIP path + conflict strategy). |
mytral/blueprints/import_uri_space.py |
Adds POST endpoint to submit the GoldenCheetah OSF import task and injects the form into the tools import page context. |
Comment on lines
+155
to
+179
| # convert CSV to Parquet and store as ACTIVITY_PARQUET | ||
| try: | ||
| pq_bytes = parquet_converter.gc_csv_to_parquet(csv_bytes, when_utc) | ||
| except Exception: | ||
| continue | ||
|
|
||
| pq_meta = _make_blob_metadata( | ||
| user_id=user_id, | ||
| activity_key=activity_key, | ||
| kind=BlobKind.ACTIVITY_PARQUET.value, | ||
| file_name="data.parquet", | ||
| original_file_name="data.parquet", | ||
| extension=".parquet", | ||
| size_bytes=len(pq_bytes), | ||
| sha256=hashlib.sha256(pq_bytes).hexdigest(), | ||
| created_at=now, | ||
| ) | ||
| sandbox_store.create_blob(pq_meta, io.BytesIO(pq_bytes)) | ||
|
|
||
| # link: recording blob UUID → parquet blob UUID (Analysis page lookup key) | ||
| d["recorded_blob_keys"] = list(d.get("recorded_blob_keys") or []) | ||
| d["recorded_blob_keys"].append(f"{csv_meta.blob_key}.csv") | ||
| d["recorded_parquet_keys"] = dict(d.get("recorded_parquet_keys") or {}) | ||
| d["recorded_parquet_keys"][csv_meta.blob_key] = pq_meta.blob_key | ||
| updated_activities.append(d) |
Comment on lines
+351
to
+363
| # delete old recording + parquet blobs for activities being re-imported | ||
| for a in activities: | ||
| for entry in list(a.recorded_blob_keys or []): | ||
| entry_uuid = entry.rsplit(".", 1)[0] | ||
| try: | ||
| self._blobstore.delete_blob(user_id, entry_uuid) | ||
| except Exception: | ||
| pass | ||
| for pq_key in list((a.recorded_parquet_keys or {}).values()): | ||
| try: | ||
| self._blobstore.delete_blob(user_id, pq_key) | ||
| except Exception: | ||
| pass |
Comment on lines
+567
to
+572
| Point to a single athlete ZIP file from the OSF dataset. | ||
| The archive must contain a <code>{uuid}.json</code> index with a | ||
| <code>RIDES</code> array. Activity summaries (duration, distance, | ||
| HR, power, cadence, elevation) are imported; time-series CSV | ||
| files are not imported because they lack GPS coordinates. | ||
| </p> |
Comment on lines
+26
to
+29
| Only the JSON index is used for import. The CSV time-series files are | ||
| intentionally skipped: they are summary-only imports of the pre-computed | ||
| METRICS (avg HR, avg power, distance, …). The CSV files contain no GPS | ||
| coordinates (lat/lon), so no map or route data is available anyway. |
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related: