Matnn (Music Audio Tagger Neural Net) is a rest compliant music tag service utilizing a batch queuing system (Kueue) running on a Kubernetes cluster.

Enhance music metadata to improve music discoverability for any use case including music libraries, streaming, song selection.

Matnn utilizes the Discogs-EffNet model from Essentia to predict music classification, BPM (Beats per minute), tonal/key scale and approachability/engagement from over 400 genres of music. Musicnn is also offered as a prediction model from 50 genres of music.

To upload your music, use either the simple web form or the CLI driven API service.

Currently music files must be MP3 format and under 10 MB.

Web upload form

Vue App - https://matnn.intamixx.uk/predict

Direct link without reverse proxy - https://mat.intamixx.uk:8090/upload

Application Architecture

Submission / Upload API (POST)

Select musical attributes for analysis by specifying tags required. This reduces time taken for analysis to complete. Select either 'genre_discogs_effnet' OR 'genre_musicnn'. Set just the filename with no options to perform a full set of predictions.

curl -k -X POST 'https://mat.intamixx.uk:8090/api/upload' -H 'Content-Type: multipart/form-data' -F "bpm=true" -F "key=true" -F "genre_discogs_effnet=true" -F "classifiers=true" -F "file=@/path/to/audio.mp3;type=audio/mpeg"

import requests
url = 'https://mat.intamixx.uk:8090/api/upload'
data = {'genre_discogs_effnet':'True', 'bpm':'True', 'key':'True', 'classifiers':'True'}
file = {'file': ('audio.mp3', open('audio.mp3', 'rb'), 'audio/mpeg')}
resp = requests.post(url=url, files=file, data=data, verify=False)
print(resp.json())

A json response will provide the upload status and prediction ID.

{
  "id": "01f436c22d490885a90853d7d048c5ff-ntrh9",
  "status": "Successfully uploaded audio.mp3"
}

There is an average wait of around 30 seconds for a typical prediction to complete. The result will not be available immediately.

Status API (GET)

This will show the status of the prediction using the ID returned from the upload step above.

curl -k 'https://mat.intamixx.uk:8090/api/status/01f436c22d490885a90853d7d048c5ff-ntrh9'

{
  "id": "01f436c22d490885a90853d7d048c5ff-ntrh9"
  "detail": "Successful Job discogseffnet-01f436c22d490885a90853d7d048c5ff-ntrh9"
  "started_at": "08-12-2023 03:36:25",
  "completed_at": "08-12-2023 03:41:19",
}

Result API (GET)

Provides the result of prediction

curl -k 'https://mat.intamixx.uk:8090/api/result/01f436c22d490885a90853d7d048c5ff-ntrh9'

import requests
url = 'https://mat.intamixx.uk:8090/api/result/01f436c22d490885a90853d7d048c5ff-ntrh9'
resp = requests.get(url=url, verify=False)
print(resp.json())

{
  "id": "01f436c22d490885a90853d7d048c5ff-ntrh9",
  "audiofile": "audio.mp3",
  "started_at": "08-12-2023 03:36:25",
  "completed_at": "08-12-2023 03:41:19",
  "completed": true,
  "result": {
    "genre": [
      "Drum n Bass",
      "Halftime",
      "Dark Ambient"
    ],
    "bpm": "174.3",
    "key": "Ab Minor",
    "classifiers": {
      "approachability": "low",
      "engagement": "high"
    }
  }
}

Webhook

HTTP-based callback function to desired URL with header signature created from a secret and the payload in attribute X-Matnn-Signature. Body contains original id and event status. Please contact for the webhooksecret key

digest = hmac.new(bytes(key, 'UTF-8'), bytes(payload, 'UTF-8'), hashlib.sha256)
signature = digest.hexdigest()

Use webhook_url in the request

curl -k -X POST 'https://mat.intamixx.uk:8090/api/upload' -H 'Content-Type: multipart/form-data' -F "webhook_url=http://yoursite.domain:1234" -F "file=@/path/to/audio.mp3;type=audio/mpeg"

Webhook received at destination

POST / HTTP/1.1
Host: yoursite.domain:1234
User-Agent: python-requests/2.31.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
X-Matnn-Signature: 0cad7db6d6a210bacb734744770be2305ba69664c3497c97d05248c8e98df9eb
Content-Type: application/json
Content-Length: 81

{"id": "01f436c22d490885a90853d7d048c5ff-ntrh9", "event": {"status": "finished"}}

Example Usage

BPM Tag a file

id3v2 --TBPM `curl -k 'https://matnn.intamixx.uk/api/result/01f436c22d490885a90853d7d048c5ff-ntrh9' \
 | jq -r '.result["bpm"]'` audio.mp3

Genre Tag a file

id3v2 --TCON `curl -k 'https://matnn.intamixx.uk/api/result/01f436c22d490885a90853d7d048c5ff-ntrh9' \
 | jq -r '.result["genre"]' | tr -d "\n" | sed 's/ //g; s/[]["]//g'` audio.mp3

Or use a method / language of your choice.

Contact

Questions? - intamixx@hotmail.com

To Do

• Improve App Frontend

Donate

To help with server costs and administration Buy Me A Coffee ☕