A music recommendation system combining Retrieval-Augmented Generation (RAG) with LLMs to provide song suggestions based on user mood and input from the conversation with AI. Utilizes an automated ETL pipeline to gather, process, and load data from YouTube and Spotify, ensuring efficient retrieval and scalable recommendations.
| Category | Tool | |
|---|---|---|
| Data Collection | Spotify API | Spotipy |
| YouTube Music API | Ytmusicapi | |
| Data Lake | Raw database | MongoDB |
| Data Transform | Document data processing | Pandas |
| Regular expression | ||
| Embedding model | OpenAI model: text-embedding-3-small | |
| Data Warehouse | Document database | PostgreSQL (registered on Feast) |
| Redis (registered on Feast for latest data) | ||
| Vector database | Chroma | |
| Data Pipeline | ETL Automation, logging and monitoring | Dagster |
| Exploratory Data Analysis and Visualization (document data) | Grafana | |
| AI Models | AI Agent | CrewAI |
| LLMs | OpenAI model: gpt-3.5 (lite version) | |
| OpenAI model: gpt-4o (advanced version) | ||
| Conversation and memory | LangChain | |
| Environment Setup | Kubernetes | |
| Harvester | ||
| Product Presentation | Web Deployment | Nginx |
| Gunicorn | ||
| Flask | ||
| Conversation frame | Chainlit | |
| Version Control | Git |