You can download the raw source code for these lecture notes [here](compmus2026-w09.qmd).

## Course Meeting Plan

### Wednesday · 26 February · Lecture

  - Demo: [The Eternal Jukebox](http://v90084.1blu.de/) (15 min)
  - Portfolio critiques (15 min)
  - Lecture: Self-similarity matrices (15 min)
  - Lecture: MFCCs and Spotify's timbre feature (20 min)
  - Breakout: Avril 14 (15 min)
  - Discussion: Breakout findings (10 min)

### Wednesday · 26 February · Lab

  - Demo: Introduction to new compmus functions (15 mins)
  - Breakout: Interpreting timbre features (or not) (20 mins)
  - Discussion: Breakout findings (10 mins)
  - Demo: Self-similarity matrices with compmus (10 mins)
  - Breakout: Self-similarity matrices (20 mins)
  - Jam session: Breakout findings (15 mins)

## Software setup

Visit the Canvas modules for this week to get RStudio connected to Github. 

In order to get the timbre functions working you will also want to reinstall `compmus` from GitHub, using the following code:

```{r}
#| eval: false
remotes::install_github('jaburgoyne/compmus')
```

Quit and restart RStudio after you reinstall `compmus`, to be sure to start fresh.

## Breakout 1: Avril 14

Below, you can see two self-similarity matrices for [Aphex Twin's 'Avril 14th'](https://open.spotify.com/track/2MZSXhq4XDJWu6coGoXX1V).
Listen to the piece together and try to explain the patterns you see in the matrices. You can download the relevant CSV files here:
  - [pitches](../dat/avril-14th-pitches.csv)
  - [timbre](../dat/avril-14th-timbre.csv)

You can download a CSV file for this analysis [here](../dat/wood.csv); once we start making portfolios next week, we'll use code like the following example.

```{r, echo = FALSE, message = FALSE}
library(tidyverse)
library(compmus)
avril_14th_pitches <- read_csv("../dat/avril-14th-pitches.csv")
avril_14th_timbre <- read_csv("../dat/avril-14th-timbre.csv")
bind_rows(
  avril_14th_pitches |>
    compmus_wrangle_chroma() |>
    filter(row_number() %% 50L == 0L) |>
    mutate(pitches = map(pitches, compmus_normalise, "manhattan")) |>
    compmus_self_similarity(pitches, "manhattan") |>
    filter(!is.na(d)) |> 
    mutate(d = d / max(d), type = "Chroma"),
  avril_14th_timbre |>
    compmus_wrangle_timbre() |>
    filter(row_number() %% 50L == 0L) |>
    mutate(timbre = map(timbre, compmus_normalise, "euclidean")) |>
    compmus_self_similarity(timbre, "cosine") |>
    filter(!is.na(d)) |> 
    mutate(d = d / max(d), type = "Timbre")
) |>
  mutate() |>
  ggplot(
    aes(
      x = xstart + xduration / 2,
      width = 100 * xduration,
      y = ystart + yduration / 2,
      height = 100 * yduration,
      fill = d
    )
  ) +
  geom_tile() +
  coord_fixed() +
  facet_wrap(~type) +
  scale_fill_viridis_c(guide = "none") +
  theme_classic() +
  labs(x = "", y = "")
```

## Breakout 2: Cepstrograms

```{r setup, message = FALSE}
library(tidyverse)
library(compmus)
```

Last week, we used several custom functions from to work with the Spotify API:

  - `get_tidy_audio_analysis` to load audio analyses from Spotify, one track at a time
  - `compmus_normalise` to normalise audio features using common techniques, including:
      - `manhattan`
      - `euclidean`
      - `chebyshev`
  - `compmus_long_distance` to compare to series of audio features against each other using common distance metrics, including:
      - `manhattan`
      - `aitchison`
      - `euclidean`
      - `cosine`
      - `angular`
  
### Common Norm and Distance Combinations

| Domain                      | Normalisation | Distance  |
| ----------------------------|---------------|-----------|
| Non-negative (e.g., chroma) | Manhattan     | Manhattan |
|                             |               | Aitchison |
|                             | Euclidean     | cosine    |
|                             |               | angular   |
|                             | Chebyshev     | [none]    |
| Full-range (e.g., timbre)   | [none]        | Euclidean |
|                             | Euclidean     | cosine    |
|                             |               | angular   |

### 'Bloed, Zweet en Tranen'

To make a cepstrogram, we will use a similar procedure to last week. In Sonic Visualiser, use Then go to the Transform menu and select Analysis by Category / Low Level Features / Mel Frequency Cepstral Coefficients [libextract] and export it as before. (There are other MFCC options, but they will **not** work with the template code for this course.) Then we use a new `compmus` function `compmus_wrangle_timbre()` instead of `compmus_wrangle_pitches()`.

Here is an example from André Hazes's 'Bloed, Zweet en Tranen'. TYou can download the relevant CSV file for this analysis here:
  - [pitches](../dat/bzt-pitches.csv)
  - [timbre](../dat/bzt-timbre.csv)

```{r}
bzt <- read_csv("../dat/bzt-timbre.csv")
```

```{r}
bzt |>
  compmus_wrangle_timbre() |> 
  mutate(timbre = map(timbre, compmus_normalise, "manhattan")) |>
  compmus_gather_timbre() |>
  ggplot(
    aes(
      x = start + duration / 2,
      width = duration,
      y = mfcc,
      fill = value
    )
  ) +
  geom_tile() +
  labs(x = "Time (s)", y = NULL, fill = "Magnitude") +
  scale_fill_viridis_c() +                              
  theme_classic()
```

### Instructions

1. Try different norms. Which seem to give the clearest visualisation?
  
2. Once you are happy with your choices, make cepstrograms for several other tracks from your corpus, and look for timbre components where there are clear changes (e.g., `mfcc_02` in 'Bloed, Zweet en Tranen'). Listen to these tracks and follow along with the cepstrograms. Can you think of words to describe what is changing in the music when you see sharp changes in the cepstrogram? Could you give any of the MFCC components a name?


## Breakout 3: Self-Similarity Matrices

The function `compmus_self_similarity` is a wrapper around `compmus_long_distance` from last week, for the case where the distances are computed form the same track. 

```{r}
bzt |>
  compmus_wrangle_timbre() |> 
  filter(row_number() %% 50L == 0L) |> 
  mutate(timbre = map(timbre, compmus_normalise, "euclidean")) |>
  compmus_self_similarity(timbre, "cosine") |> 
  ggplot(
    aes(
      x = xstart + xduration / 2,
      width = 50 * xduration,
      y = ystart + yduration / 2,
      height = 50 * yduration,
      fill = d
    )
  ) +
  geom_tile() +
  coord_fixed() +
  scale_fill_viridis_c(guide = "none") +
  theme_classic() +
  labs(x = "", y = "")
```

### Instructions

1. Try different distance metrics to see which one is most useful for this track.
2. Make another self-similarity matrix based on chroma (`pitches`). Adjust your summary and norm (from the `pitches` line of the previous breakout), as well as the distance metric, to get the best visualisation.
3. Once you are happy with your settings in steps 1 and 2, make self-similarity matrices for the tracks you used in the previous breakout. How can you explain the patterns you see?
