SonicInsight: AI-Powered Music Recommendation Backend

SonicInsight is a production-ready FastAPI backend that analyzes songs based on their actual sound—beat, vocals, instrumentation, and mood—using state-of-the-art models like MERT and CLAP for deep audio analysis, natural language feedback interpretation, scalable embedding storage, and interactive recommendation refinement through a conversational API, all designed to provide users with truly sonically similar track suggestions at scale.

See it in action

Interactive preview — try it before you build

Loading live preview...

How it works

  1. Pick your starting point
    Use this app or start from scratch

  2. Describe your changes
    Tell AI what to customize — in plain English

  3. Ship it
    Your app goes live with free hosting

The AI prompt behind this app

Build me a production-ready backend service for a music recommendation app with the following requirements:
1. Core Functionality
   - Recommendations are based on the actual sound of a song (beat, vocals, flow, instrumentation, mood, energy, dynamic changes), not genre or artist.
   - Analyze each track for: beat/tempo/rhythm, vocal timbre, instrumentation, energy/mood over time, and structure (verse, chorus, bridge).
   - Use MERT (Music Embedding Representation Transformer) as the primary embedding model.
   - Integrate CLAP (Contrastive Language-Audio Pretraining) to interpret user text feedback in embedding space.

2. Inputs
   - Accept audio file uploads (MP3/WAV).
   - Accept song references by title/ID and fetch audio via public APIs if possible.

3. Embeddings & Storage
   - Generate embeddings for every uploaded/referenced song.
   - Store embeddings and metadata in Pinecone (or equivalent vector DB).
   - Store raw audio in cloud storage (e.g., AWS S3).
   - Scalable to millions of songs.

4. Similarity Search
   - Query vector DB with song embeddings to return top N most sonically similar tracks.
   - Return: song title, artist, similarity score, and preview/playback link if available.

5. Feedback & Refinement
   - Endpoint accepts natural language feedback like “I love the intro guitar” or “make it more vocal-heavy.”
   - Use CLAP embeddings to adjust weighting in similarity search.
   - Return updated recommendations dynamically.

6. Conversational Assistant
   - Chat endpoint explains why each recommendation was chosen.
   - Accepts follow-ups to refine recommendations interactively.

7. API Design
   - Backend built with FastAPI (Python).
   - Endpoints: /analyze_song, /recommend, /feedback, /chat.

8. Scalability & Quality
   - GPU acceleration for embedding inference.
   - Modular backend so models can be swapped/upgraded easily.
   - Monitoring, logging, and error handling included.

Deliverable:
A fully working, production-ready backend that can:
- Take a song, analyze it like a human listener, and return sonically similar tracks.
- Refine results dynamically based on user feedback.
- Provide a conversational interface for explanation and refinement.
- Scale to handle large catalogs efficiently.