platypush/blog
2
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Music automation article

Browse source
This commit is contained in:
Fabio Manganiello 2022-09-19 21:57:45 +02:00
parent 4c72748b9f
commit 4da14c5405
1 changed files with 248 additions and 14 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

262
markdown/Automate-your-music-collection.md
View file

@ -3,6 +3,7 @@
[//]: # (image: /img/music-automation.png)
[//]: # (author: Fabio Manganiello <fabio@platypush.tech>)
[//]: # (published: 2022-09-19)
[//]: # (latex: 1)
I have been an enthusiastic user of mpd and mopidy for nearly two decades. I
have already [written an
@ -27,16 +28,16 @@ The problem is that these services come with heavy trade-offs:
1. Their algorithms are closed. You don't know how Spotify figures out which
songs should be picked in your smart playlists. In the past months, Spotify
would often suggest me tracks from the same artists that I had already
listened to or skipped in the past, and there's no easy way to tell the
algorithm "hey, actually I'd like you to suggest me more this kind of music -
or maybe calculate suggestions only based on the music I've listened to in
this time range".
listened to or skipped in the past, and there's no transparent way to tell
the algorithm "hey, actually I'd like you to suggest me more this kind of
music - or maybe calculate suggestions only based on the music I've listened
to in this time range, or maybe weigh this genre more".
2. Those features are tightly coupled with the service you use. If you cancel
your Spotify subscription, you lose those smart features as well.
Companies like Spotify use such features as a lock-in mechanism -
"you can check out any time you like, but if you do then nobody else will
provide you with our clever suggestions".
you can check out any time you like, but if you do then nobody else will
provide you with their clever suggestions.
After migrating from Spotify to Tidal in the past couple of months (TL;DR:
Spotify f*cked up their developer experience multiple times over the past
@ -199,7 +200,7 @@ create table music_track(
create unique index track_artist_title_idx on music_track(lower(artist), lower(title));
create index track_artist_idx on music_track(lower(artist));
-- music_activity holds the listen history
-- music_activity holds the listened tracks
drop table if exists music_activity cascade;
create table music_activity(
id serial not null,
@ -304,9 +305,10 @@ Now that all the dependencies are in place, it's time to configure the logic to
store your music activity to your database.
If most of your music activity happens through mpd/mopidy, then storing your
activity to the database is as simple as creating a hook on a
[`NewPlayingTrackEvent`](https://docs.platypush.tech/platypush/events/music.html)
that inserts any newly playing track on `tmp_music`. Paste the following
activity to the database is as simple as creating a hook on
[`NewPlayingTrackEvent`
events](https://docs.platypush.tech/platypush/events/music.html)
that inserts any new played track on `tmp_music`. Paste the following
content to a new Platypush user script (e.g.
`~/.config/platypush/scripts/music/sync.py`):
@ -354,15 +356,15 @@ def on_new_track_playing(event, **_):
)
```
Alternatively, if you also want to sync music activities that happens through
Alternatively, if you also want to sync music activity that happens on
other clients (such as the Spotify/Tidal app or web view, or over mobile
devices), you may consider leveraging Last.fm. Last.fm (or its open alternative
Libre.fm) are _scrobbling_ websites that are compatible with most of the music
Libre.fm) is a _scrobbling_ service compatible with most of the music
players out there. Both Spotify and Tidal support scrobbling, the [Android
app](https://apkpure.com/last-fm/fm.last.android) can grab any music activity
on your phone and scrobble, and there are even [browser
on your phone and scrobble it, and there are even [browser
extensions](https://chrome.google.com/webstore/detail/web-scrobbler/hhinaapppaileiechjoiifaancjggfjm?hl=en)
that allow you to record any music activity from any browser tab.
that allow you to keep track of any music activity from any browser tab.
So an alternative approach may be to send both your mpd/mopidy music activity,
as well as your in-browser or mobile music activity, to last.fm / libre.fm. The
@ -993,4 +995,236 @@ Let's put these pieces together in a new user script stored under e.g.
```python
# ~/.config/platypush/scripts/music/releases.py
import html
import logging
import re
import threading
from datetime import date, timedelta
from typing import Iterable, List
from platypush.context import get_plugin
from platypush.cron import cron
from platypush.event.hook import hook
from platypush.message.event.rss import NewFeedEntryEvent
from scripts.music.db import (
music_db_engine, get_db_session, NewRelease
)
create_lock = threading.RLock()
logger = logging.getLogger(__name__)
def _split_html_lines(content: str) -> List[str]:
"""
Utility method used to convert and split the HTML lines reported
by the RSS feed.
"""
return [
l.strip()
for l in re.sub(
r'(</?p[^>]*>)|(<br\s*/?>)',
'\n',
content
).split('\n') if l
]
def _get_summary_field(title: str, lines: Iterable[str]) -> str | None:
"""
Parse the fields of a new album from the feed HTML summary.
"""
for line in lines:
m = re.match(rf'^{title}:\s+(.*)$', line.strip(), re.IGNORECASE)
if m:
return html.unescape(m.group(1))
@hook(NewFeedEntryEvent, feed_url='https://newalbumreleases.net/category/cat/feed/')
def save_new_release(event: NewFeedEntryEvent, **_):
"""
This hook is triggered whenever the newalbumreleases.net has new entries.
"""
# Parse artist and album
summary = _split_html_lines(event.summary)
artist = _get_summary_field('artist', summary)
album = _get_summary_field('album', summary)
genre = _get_summary_field('style', summary)
if not (artist and album):
return
# Check if we have listened to this artist at least once
db = get_plugin('db')
num_plays = int(
db.select(
engine=music_db_engine,
query=
'''
select count(*)
from music_activity a
join music_track t
on a.track_id = t.id
where artist = :artist
''',
data={'artist': artist},
).output[0].get('count', 0)
)
# If not, skip it
if not num_plays:
return
# Insert the new release on the database
with create_lock:
db.insert(
engine=music_db_engine,
table='new_release',
records=[{
'artist': artist,
'album': album,
'genre': genre,
}],
key_columns=('artist', 'album'),
on_duplicate_update=True,
)
def get_new_releases(days=7):
"""
Retrieve the new album releases from the database.
:param days: Look at albums releases in the past <n> days
(default: 7)
"""
with get_db_session() as session:
return [
{
'artist': t[0],
'album': t[1],
}
for t in session.query(
NewRelease.artist,
NewRelease.album,
)
.select_from(
NewRelease.__table__
)
.filter(
# Filter by recent activity
NewRelease.created_at >= date.today() - timedelta(days=days)
)
.all()
]
def search_tidal_new_releases(albums):
"""
Search for Tidal albums given a list of objects with artist and title.
"""
tidal = get_plugin('music.tidal')
expanded_tracks = []
for album in albums:
query = album['artist'] + ' ' + album['album']
logger.info('Searching "%s"', query)
results = (
tidal.search(query, type='album', limit=1)
.output.get('albums', [])
)
if results:
album = results[0]
# Skip search results older than a year - some new releases may
# actually be remasters/re-releases of existing albums
if date.today().year - album.get('year', 0) > 1:
continue
expanded_tracks += (
tidal.get_album(results[0]['id']).
output.get('tracks', [])
)
else:
logger.warning('Could not find "%s" on TIDAL', query)
return expanded_tracks
def refresh_release_radar():
tidal = get_plugin('music.tidal')
# Get the latest releases
tracks = search_tidal_new_releases(get_new_releases())
if not tracks:
logger.info('No new releases found')
return
# Retrieve the existing new releases playlists
playlists = tidal.get_playlists().output
new_releases_playlists = sorted(
[
pl for pl in playlists
if pl['name'].lower().startswith('new releases')
],
key=lambda pl: pl.get('created_at', 0)
)
# Delete all the existing new releases playlists
# (except the latest one)
for playlist in new_releases_playlists[:-1]:
logger.info('Deleting playlist "%s"', playlist['name'])
tidal.delete_playlist(playlist['id'])
# Create a new releases playlist
playlist_name = f'New Releases [{date.today().isoformat()}]'
pl = tidal.create_playlist(playlist_name).output
playlist_id = pl['id']
tidal.add_to_playlist(
playlist_id,
[t['id'] for t in tracks],
)
@cron('0 7 * * 1')
def refresh_release_radar_cron(**_):
"""
This cron will execute every Monday at 7 AM.
"""
try:
refresh_release_radar()
except Exception as e:
logger.exception(e)
get_plugin('ntfy').send_message(
topic='mirrored-notifications-topic',
title='Release Radar playlist generation failed',
message=str(e),
priority=4,
)
```
Just like in the previous case, it's quite easy to test that it works by simply
running `refresh_release_radar_cron` in the Python interpreter. Just like in
the case of the discovery playlist, things will work also if you use Spotify
instead of Tidal - just replace the `music.tidal` plugin references with
`music.spotify`.
If it all goes as expected, you will get a new playlist named _New Releases
[date]_ every Monday with the new releases from artist that you have listened.
## Conclusions
Music junkies have the opportunity to discover a lot of new music today without
ever leaving their music app. However, smart playlists provided by the major
music cloud providers are usually implicit lock-ins, and the way they select
the tracks that should end up in your playlists may not even be transparent, or
even modifiable.
After reading this article, you should be able to generate your discovery and
new releases playlists, without relying on the suggestions from a specific
music cloud. This could also make it easier to change your music provider: even
if you decide to drop Spotify or Tidal, your music suggestions logic will
follow you whenever you decide to go.