Data Sources Management 📊

Data Sources Management Guide 📊

Complete guide to managing multiple data sources in Running Page 2.0.

🎯 Overview

The Data Sources page (/data-sources) is your central hub for managing all fitness platform integrations. Here you can:

• ✅ View all configured sources and their status
• ✅ Add new data sources from supported platforms
• ✅ Monitor sync status and performance
• ✅ Configure sync settings and preferences
• ✅ Troubleshoot connection issues with detailed diagnostics

🚀 Getting Started

Accessing Data Sources

1. Navigate to Data Sources

   • Click "Data Sources" in the sidebar
   • Or visit /data-sources directly

2. Interface Overview

   • Configured Sources tab - Your active integrations
   • Available Sources tab - Platforms you can add
   • Sync Results tab - Recent sync history (when available)

Adding Your First Data Source

1. Click "Add Data Source" button
2. Choose a platform from the available options
3. Follow setup instructions specific to that platform
4. Test connection to verify configuration
5. Start syncing your activities

📱 Platform Management

Configured Sources Tab

Source Cards Display

Each configured source shows:

• Platform name and type (API, File, Webhook)
• Connection status (Active, Inactive, Error)
• Last sync time and activity count
• Supported activity types (Run, Walk, Ride, etc.)
• Enable/disable toggle for quick control
• Sync and settings buttons for management

Status Indicators

Status	Icon	Meaning	Action
Active	✅	Working normally	Monitor regularly
Inactive	⏸️	Disabled or paused	Enable if needed
Error	❌	Connection issues	Check configuration

Quick Actions

• Sync Button - Trigger immediate sync for this source
• Settings Button - Modify configuration (coming soon)
• Enable/Disable Toggle - Control sync participation

Available Sources Tab

Platform Cards Show

• Platform name and description
• Authentication method required
• Supported activity types
• Estimated setup time
• Configuration status (Available/Already Added)

Current Platform Status

Platform	Status	Setup Time	Notes
Strava	✅ Configured	-	Already set up via environment
Nike Run Club	🚧 Beta	5 min	Access token method
Garmin Connect	🔄 Coming Soon	10 min	Global and China versions
Keep (咕咚)	🔄 Coming Soon	5 min	Chinese platform
GPX Files	🔄 Coming Soon	2 min	File upload interface

🔄 Sync Management

Automatic Sync

• Default Schedule - Every 6 hours
• Incremental Sync - Only new activities
• Smart Deduplication - Prevents duplicate activities
• Error Recovery - Automatic retry on failures

Manual Sync Options

Sync All Sources

1. Click "Sync All" button in top right
2. Monitor progress in real-time
3. View detailed results when complete
4. Check for any errors or issues

Sync Individual Source

1. Find the source card you want to sync
2. Click the "Sync" button on that card
3. Watch the spinning icon for progress
4. Check the last sync time when complete

Sync Status Monitoring

Real-time Indicators

• Spinning refresh icon - Sync in progress
• Last sync timestamp - When data was last updated
• Activity count changes - New activities detected
• Error messages - Detailed failure information

Sync Results Tab

When available, shows:

• Summary statistics - Total activities processed
• Per-source breakdown - Individual source performance
• Success/failure rates - Reliability metrics
• Error details - Specific failure reasons
• Processing time - Sync duration and performance

⚙️ Configuration Options

Environment Variables

# Global sync settings
ENABLE_MULTI_PLATFORM=true
SYNC_INTERVAL_HOURS=6
MAX_CONCURRENT_SYNCS=3
DEFAULT_SYNC_TIMEOUT=300000

# Platform-specific settings
NIKE_RATE_LIMIT=100
GARMIN_TIMEOUT=30000
KEEP_REGION=cn

Sync Preferences

// Future configuration options
const syncConfig = {
  schedule: '6h',           // Sync frequency
  deduplication: true,      // Remove duplicates
  sourcePriority: [         // Preference order
    'strava',
    'nike',
    'garmin'
  ],
  activityTypes: [          // Types to sync
    'Run', 'Walk', 'Ride'
  ],
  dateRange: {              // Sync date limits
    start: '2020-01-01',
    end: null               // null = no limit
  }
}

📊 Data Quality Management

Deduplication System

• Smart Detection - Identifies potential duplicates
• Time-based Matching - Activities within 5 minutes
• Distance Comparison - Similar distance and duration
• Source Priority - Configurable preference order
• Manual Review - Option to review before deletion

Data Validation

• Format Checking - Ensures data integrity
• Range Validation - Realistic values only
• GPS Validation - Valid coordinate ranges
• Metric Consistency - Cross-validation of related fields

Conflict Resolution

When the same activity exists in multiple sources:

1. Source Priority - Use configured preference order
2. Data Completeness - Prefer source with more data
3. Data Quality - Choose higher quality GPS/metrics
4. Manual Override - User can choose preferred source

🐛 Troubleshooting

Common Issues

"No Data Sources Available"

Symptoms: Empty available sources list Solutions:

1. Check internet connection
2. Verify API endpoints are accessible
3. Clear browser cache and reload
4. Check browser console for errors

"Sync Failed" Errors

Symptoms: Red error status on source cards Solutions:

1. Check Authentication

   • Verify tokens are still valid
   • Re-authenticate if needed
   • Check token permissions

2. Network Issues

   • Test internet connectivity
   • Check firewall settings
   • Verify API endpoints are accessible

3. Rate Limiting

   • Wait before retrying
   • Reduce sync frequency
   • Check platform API limits

"Duplicate Activities"

Symptoms: Same activity appears multiple times Solutions:

1. Enable Deduplication

   // In future configuration
   deduplication: true

2. Set Source Priority

   • Choose preferred data source
   • Configure priority order
   • Review duplicate detection settings

3. Manual Cleanup

   • Review flagged duplicates
   • Delete unwanted copies
   • Adjust detection sensitivity

Debug Tools

API Testing

# Test data sources API
curl -X GET "https://your-domain.com/api/data-sources"

# Test sync operation
curl -X POST "https://your-domain.com/api/sync" \
  -H "Content-Type: application/json" \
  -d '{"sources": ["nike"]}'

# Check sync status
curl -X GET "https://your-domain.com/api/sync/status"

Browser Console

// Check data source status
fetch('/api/data-sources')
  .then(r => r.json())
  .then(data => {
    console.log('Available types:', data.availableTypes)
    console.log('Configured sources:', data.configuredSources)
  })

// Monitor sync progress
fetch('/api/sync', {method: 'POST'})
  .then(r => r.json())
  .then(result => {
    console.log('Sync results:', result)
  })

// Check for errors
console.log('Check for any error messages in the console')

📈 Best Practices

Setup Recommendations

1. Start Simple - Add one source at a time
2. Test Thoroughly - Verify each integration works
3. Monitor Closely - Watch first few syncs carefully
4. Document Settings - Keep track of configurations

Performance Optimization

1. Stagger Sync Times - Avoid simultaneous syncs
2. Monitor API Limits - Stay within rate limits
3. Use Incremental Sync - Only sync new data
4. Clean Up Regularly - Remove unused sources

Data Quality

1. Set Source Priority - Choose preferred data sources
2. Enable Deduplication - Prevent duplicate activities
3. Validate Regularly - Check data accuracy
4. Backup Before Changes - Export data before major changes

Security

1. Rotate Tokens Regularly - Update authentication tokens
2. Monitor Access Logs - Check for suspicious activity
3. Use Strong Passwords - Secure platform accounts
4. Enable 2FA - Where available on platforms

🔮 Coming Soon

Planned Features

• Bulk Operations - Enable/disable multiple sources
• Advanced Filtering - Sync specific activity types only
• Custom Schedules - Per-source sync frequencies
• Data Export - Export data from specific sources
• Conflict Resolution UI - Manual duplicate management
• Source Analytics - Per-source performance metrics

Platform Roadmap

• Q1 2024: Nike Run Club (Beta → Production)
• Q2 2024: Garmin Connect (Global + China)
• Q3 2024: Keep, Codoon, JoyRun
• Q4 2024: Apple Health, Google Fit
• 2025: Suunto, Coros, Polar

Advanced Features

• Cross-platform Analytics - Compare data across sources
• Data Correlation - Find patterns across platforms
• Health Metrics - Comprehensive health tracking
• Social Features - Multi-platform activity sharing

💡 Tips & Tricks

Maximizing Data Quality

1. Use GPS-enabled activities for best map visualization
2. Enable heart rate monitoring for detailed analytics
3. Consistent activity naming across platforms
4. Regular data validation to catch issues early

Sync Optimization

1. Sync during off-peak hours for better performance
2. Monitor platform status pages for service issues
3. Use incremental sync to reduce processing time
4. Set up monitoring alerts for sync failures

Platform-Specific Tips

• Nike: Use access token method for reliability
• Strava: Already optimized, no changes needed
• Garmin: Use secret string method for automation
• Keep: Enable SMS verification for security

---

Need help with data source management?

• 🐛 Report Issues
• 💬 Ask Questions
• 📧 Email Support

Effective data source management ensures reliable, comprehensive activity tracking across all your fitness platforms! 📊✨