Retention · 2026-09-18

Why they leave after one listen

412 accounts submitted a track, played one or two of the recommendations and never came back. This is what they had in common — the device they were on — and what turned out not to matter at all: the recommendations.

Who is being compared

A pattern among people who left means nothing without people who did not, so everything below is a comparison of two groups, measured on the first search each of them ever ran — the moment that decided it.

1. The recommendations they got were fine

This is the finding that mattered most, because it is the hypothesis everyone reaches for first. Measured on the top 10 of each group's very first search:

Genre agreement counts a recommendation as a hit when it shares at least one canonical genre label with the submission; rows without genre labels on either side are excluded rather than counted as misses.

The playability row is the one worth sitting with. Dead rows — results with no embeddable preview on any platform — were almost twice as common for the people who stayed. Whatever drives people away, a result they cannot hear is not it.

2. What was actually different: how far they listened

The time-to-first-play row runs the opposite way to intuition: the people who left hit play almost twice as fast. Stayers spend a minute with the list before playing anything — reading it, presumably — and then work their way down to rank seven. Drop-offs play the top result within half a minute, maybe the second one, and go.

3. Listening is the thing that predicts coming back

Widening from the two groups to every account old enough to have had the chance to return, the relationship between tracks played in the first 24 hours and ever searching again on a later day is steep and monotone — but only after the fifth track.

The first three bars are the same bar. Playing nothing, playing one track and playing five all land between 16% and 18%. The curve only starts climbing at six, and by twenty-six it has tripled. So "played one or two tracks and left" is not a distinct failure with a cause of its own — it is the flat bottom of one continuous relationship, and the threshold that matters sits somewhere around the sixth track.

4. Where recommendation quality does bite: the outlier tail

One measure of the recommendations does track retention, and it is not any of the metadata ones. It is how close the nearest neighbour sat in the embedding — the top result's cosine similarity, which is a direct read on whether the submitted track had anything like it in the catalogue at all.

Monotone across every step, a 1.6× spread end to end, and it survives controlling for how long each account has had the chance to come back. Notably, genre agreement over the same population does nothing: accounts whose top 10 shared almost no genre with their submission returned at 24.4%, those whose top 10 matched almost perfectly at 28.3%. The metadata is the wrong lens here; the vector is the right one.

What an outlier submission actually is

Comparing the 331 first searches whose top result came back under 0.85 against the 4 954 that came back at 0.93 or better:

Six times the rate of non-electronic music, eight times the rate of sub-100-BPM tracks. These are people pasting a music video into an engine whose catalogue is house and techno. The far end of the tail, all of them accounts that never came back:

Nothing is broken in these searches. The engine answered the question it was asked, honestly, and the honest answer was "there is nothing like this here". The product never says so — it returns a hundred rows ranked 0.4 to 0.6 exactly as it returns a hundred rows ranked 0.96, and the person has no way to tell those apart.

5. What we ruled out

6. One asymmetry worth a second look

How the first search was started separates the two groups more cleanly than anything about the results. Pasting a URL of your own track goes with a 32.7% return rate; picking something out of the catalogue with 22.8%. And on their first search the drop-offs pasted a URL far less often than the stayers — 22.6% against 39.7%.

Not controlled for account age, and the causation could run either way: arriving with a specific track in mind is a sign of intent as much as a cause of it. But it is the largest first-session split in the data, and it is something the product can act on.

7. Phone versus computer — the largest split in the data

Device is the one attribute where the drop-offs stop looking like everyone else. It lives in the analytics rather than the database, so this section is measured against a slightly looser reconstruction of the cohort (229 accounts rather than 412 — the event log's idea of a "search day" is more generous than the database's). Within it the comparison is apples to apples.

The drop-offs are two-thirds phone users; the people who stayed are a majority desktop. The mobile-to-desktop ratio flips from 0.74 among the returners to 2.2 among the people who left — a bigger separation than anything about the recommendations, the catalogue or the submission.

Holding the observation window fixed and looking only at accounts whose device we know, the whole engagement profile shifts with it:

A phone session is a third of the length of a desktop one, produces a third fewer plays, and is two and a half times as likely to end at the second track — which is exactly the shape of the drop-off cohort. Saving a track, the strongest sign that something landed, happens half as often.

It is not the player

The obvious mechanism would be playback failing on a phone — autoplaying someone else's iframe in a mobile browser is a classic way to lose people. It is not happening. Counting 3.5-second plays against preview starts, mobile converts at 1.07 and desktop at 0.96; no account started a preview and never reached the threshold, on either device, and there are no rage-clicks. Every preview a phone user starts, plays.

Coverage: 2 621 of 7 524 signed-in accounts (35%) send only server-side events — an ad-blocker or a declined consent banner means no device is ever recorded for them, and nothing can recover it. They average 99 events against 214 for the rest, so the accounts with no device are systematically less engaged than the ones counted here. Everything above is computed on the 4 903 where the device is known, and the true mobile share of the drop-offs could be higher or lower depending on which way that missing third leans. One row was left out of the table as an instrumentation artefact: phones appear to use filters twelve times as often, because on desktop the filters are always on screen and never fire an "opened" event.

Caveats

• Most of the user base is very new. 4 160 of 8 215 accounts ran their first search in the last 14 days, and they return at 17–19% simply because they have not had long. Accounts older than 60 days return at 51%. Every retention number above is restricted to accounts with at least a 14-day window; the raw funnel figure — 74% of everyone who ever searched has not come back for a second day — is inflated by this censoring and should not be quoted on its own.
• The device split is measured on two thirds of the accounts. A third send only server-side events, so their device is unknowable, and they are less engaged than average — which means the mobile share of the drop-offs is an estimate with a real error bar, not a measurement.
• Correlation, throughout. "Played six tracks" predicting a return does not mean pushing someone to a sixth track would cause one. The honest reading is that both are symptoms of the same thing.
• A "play" is 3.5 seconds of playback, first time only. There is no repeat-play count and no duration, so "played two tracks" means two distinct tracks, not two listens, and someone who loved one track and replayed it eight times is indistinguishable from someone who sampled it once.
• No scroll data. Whether drop-offs ever saw rank 11 is the obvious next question and the analytics event for it is not firing. The session replays linked from each account on the drop-off tool are the way in.
• The cut is arbitrary and the conclusion should survive moving it. "One or two plays, one day, silent 14 days" is a definition, not a fact — the tool lets you move all three, and a pattern that only appears at one setting is a pattern about the setting.

Where this points

The lever is not better neighbours for the average search. Those are already as good as the ones that retain. The two things the data actually supports:

1. Say when the answer is weak. 3.8% of first searches come back with a top result under 0.85 and the interface presents them exactly like a 0.96. Telling someone their track has no near relatives here — and suggesting one that does — is a cheap fix for the one recommendation-quality effect that measurably exists.
2. Get past the second row. Every account that reaches a sixth track returns at a materially higher rate, and drop-offs stop at rank two after 37 seconds. Whatever makes row three worth reaching is the thing to find, and the place to look is the replays, not the database.
3. Look at the phone. Two thirds of the drop-offs are on one, their sessions run five minutes against fifteen, and they hit the second-track wall two and a half times as often — while the player itself works perfectly. The same hundred-row list is being served to a five-minute session and a fifteen-minute one.

Built from search_history's stored result snapshots joined against play records, on prod, 18 Sep 2026. The per-account view these numbers were derived from is the Drop-off tool, where each of the 412 can be opened and the recommendations they were given can be played.