Skip to content

1. Why encode music? From notation to machine-readable scores

Session 1, 15 October 2026 (double session, 2 × 90 minutes).

Every notation is already an encoding. It was made for a particular reader and a particular use, and it writes down some things while leaving others to the reader's knowledge. Computers, and now language models, are readers without that knowledge, so music encoding formats must state explicitly what human readers infer. Formats differ for the same reason historical notations differ: they serve different users and tasks.

How to use this page

The demos on this page run in your browser; nothing needs to be installed. They load Verovio (about 12 MB) the first time you use them. The same demos come as Jupyter notebooks, where you can change the inputs and keep your work: see Notebooks at the end.

What is Verovio?

Verovio is the program that turns an encoding into a score you can see on this page. It engraves music: it reads an encoded score and draws it as notation (an SVG image). It is built around MEI and converts the other formats on the fly: kern, MusicXML, ABC, Plaine & Easie, EsAC, and the chant formats GABC and Volpiano. It is free and open source (LGPL licence), and the RISM Digital Center leads its development.

Verovio is written in C++ and comes in versions for other languages. This page uses the JavaScript version, which runs inside your browser; the notebooks use the Python version (pip install verovio). Both are version 6.3.0, so page and notebooks show the same score. You can also try it in the Verovio editor, and CAMAT, the library we use from session 2, renders scores with it.

Verovio only shows what the encoding says. When a score looks wrong, look at the encoding first, and then at how Verovio reads it: parts 8 and 10 do both.

By the end of the session you can

  1. explain why a score made for human readers is not enough for a computer;
  2. ask three questions of any notation or format: Who reads it? What does it record? What does it leave out?;
  3. tell a scan, a PDF, an encoded score and a facsimile-linked edition apart, and say who can use each;
  4. place key systems on a timeline from mechanical instruments to kern and the Essen folksong collection;
  5. edit a short ABC, kern or MEI text and render it with Verovio;
  6. turn a melody into a 0/1 grid and say what the grid loses;
  7. check an LLM-written melody against what was asked for.

1. One melody, seven encodings

Here are the first four bars of Alle meine Entchen, seven times. Before anything is rendered: Which of these is the music? Which can you read? Which can a computer read? The last tab shows the same bars as a score, for reference.

X:1
T:Alle meine Entchen
M:4/4
L:1/4
K:C
C D E F | G2 G2 | A A A A | G4 |]

!!!OTL: Alle meine Entchen
**kern
*clefG2
*k[]
*M4/4
=1
4c
4d
4e
4f
=2
2g
2g
=3
4a
4a
4a
4a
=4
1g
==
*-

@clef:G-2
@keysig:
@timesig:4/4
@data:'4CDEF/2GG/4AAAA/1G//

DEMO01
CUT[Alle meine Entchen]
REG[Europa, Mitteleuropa, Deutschland]
KEY[X0001 04 C 4/4]
MEL[1234 5_5_ 6666 5__ //]
>>

<?xml version="1.0" encoding="UTF-8"?>
<score-partwise version="4.0">
  <work><work-title>Alle meine Entchen</work-title></work>
  <part-list><score-part id="P1"><part-name>Voice</part-name></score-part></part-list>
  <part id="P1">
    <measure number="1">
      <attributes><divisions>1</divisions><key><fifths>0</fifths></key>
        <time><beats>4</beats><beat-type>4</beat-type></time>
        <clef><sign>G</sign><line>2</line></clef></attributes>
      <note><pitch><step>C</step><octave>4</octave></pitch><duration>1</duration><type>quarter</type></note>
      <note><pitch><step>D</step><octave>4</octave></pitch><duration>1</duration><type>quarter</type></note>
      <note><pitch><step>E</step><octave>4</octave></pitch><duration>1</duration><type>quarter</type></note>
      <note><pitch><step>F</step><octave>4</octave></pitch><duration>1</duration><type>quarter</type></note>
    </measure>
    <measure number="2">
      <note><pitch><step>G</step><octave>4</octave></pitch><duration>2</duration><type>half</type></note>
      <note><pitch><step>G</step><octave>4</octave></pitch><duration>2</duration><type>half</type></note>
    </measure>
    <measure number="3">
      <note><pitch><step>A</step><octave>4</octave></pitch><duration>1</duration><type>quarter</type></note>
      <note><pitch><step>A</step><octave>4</octave></pitch><duration>1</duration><type>quarter</type></note>
      <note><pitch><step>A</step><octave>4</octave></pitch><duration>1</duration><type>quarter</type></note>
      <note><pitch><step>A</step><octave>4</octave></pitch><duration>1</duration><type>quarter</type></note>
    </measure>
    <measure number="4">
      <note><pitch><step>G</step><octave>4</octave></pitch><duration>4</duration><type>whole</type></note>
      <barline location="right"><bar-style>light-heavy</bar-style></barline>
    </measure>
  </part>
</score-partwise>

<?xml version="1.0" encoding="UTF-8"?>
<?xml-model href="https://music-encoding.org/schema/dev/mei-all.rng" type="application/xml" schematypens="http://relaxng.org/ns/structure/1.0"?>
<?xml-model href="https://music-encoding.org/schema/dev/mei-all.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
<mei xmlns="http://www.music-encoding.org/ns/mei" meiversion="6.0-dev">
   <meiHead>
      <fileDesc>
         <titleStmt>
            <title>Alle meine Entchen</title>
         </titleStmt>
         <pubStmt>
            <unpub>This MEI file was created by Verovio's Humdrum converter. When published, this unpub element should be removed, and the enclosing pubStmt element should be properly filled out.</unpub>
         </pubStmt>
      </fileDesc>
      <encodingDesc>
         <appInfo>
            <application isodate="2026-10-14T14:24:16" version="6.3.0-425dd7b">
               <name>Verovio</name>
               <p>Transcoded from Humdrum</p>
            </application>
         </appInfo>
      </encodingDesc>
      <workList>
         <work xml:id="work0_encoded" type="encoded">
            <title type="uniform">
               <titlePart type="main" analog="humdrum:OTL">Alle meine Entchen</titlePart>
            </title>
         </work>
      </workList>
      <extMeta>
         <frames xmlns="http://www.humdrum.org/ns/humxml">
            <metaFrame n="0" token="!!!OTL: Alle meine Entchen" xml:id="L1">
               <frameInfo>
                  <startTime float="0" />
                  <frameType>reference</frameType>
                  <referenceKey>OTL</referenceKey>
                  <referenceValue>Alle meine Entchen</referenceValue>
               </frameInfo>
            </metaFrame>
         </frames>
      </extMeta>
   </meiHead>
   <music decls="#work0_encoded">
      <body>
         <mdiv xml:id="e6nh3r1">
            <score xml:id="fhc7yxj">
               <scoreDef xml:id="g185h7q1" tempo.dist="3vu">
                  <pgHead xml:id="qhtix66" func="first">
                     <rend xml:id="rjw3qv2" halign="center" valign="middle">
                        <rend xml:id="scqmx8j" fontsize="large">Alle meine Entchen</rend>
                        <lb xml:id="u5b79pu" /> 
<lb xml:id="w1kluom4" /> </rend>
                  </pgHead>
                  <staffGrp xml:id="rlpmy72">
                     <staffDef xml:id="staffdef-L2F1" n="1" lines="5">
                        <clef xml:id="clef-L3F1" shape="G" line="2" />
                        <keySig xml:id="keysig-L4F1" sig="0" />
                        <meterSig xml:id="metersig-L5F1" count="4" unit="4" />
                     </staffDef>
                  </staffGrp>
               </scoreDef>
               <section xml:id="section-L2F1">
                  <measure xml:id="measure-L1" n="1">
                     <staff xml:id="staff-L2F1" n="1">
                        <layer xml:id="layer-L1F1N1" n="1">
                           <note xml:id="note-L7F1" dur="4" oct="4" pname="c" />
                           <note xml:id="note-L8F1" dur="4" oct="4" pname="d" />
                           <note xml:id="note-L9F1" dur="4" oct="4" pname="e" />
                           <note xml:id="note-L10F1" dur="4" oct="4" pname="f" />
                        </layer>
                     </staff>
                  </measure>
                  <measure xml:id="measure-L11" n="2">
                     <staff xml:id="staff-L11F1N1" n="1">
                        <layer xml:id="layer-L11F1N1" n="1">
                           <note xml:id="note-L12F1" dur="2" oct="4" pname="g" />
                           <note xml:id="note-L13F1" dur="2" oct="4" pname="g" />
                        </layer>
                     </staff>
                  </measure>
                  <measure xml:id="measure-L14" n="3">
                     <staff xml:id="staff-L14F1N1" n="1">
                        <layer xml:id="layer-L14F1N1" n="1">
                           <note xml:id="note-L15F1" dur="4" oct="4" pname="a" />
                           <note xml:id="note-L16F1" dur="4" oct="4" pname="a" />
                           <note xml:id="note-L17F1" dur="4" oct="4" pname="a" />
                           <note xml:id="note-L18F1" dur="4" oct="4" pname="a" />
                        </layer>
                     </staff>
                  </measure>
                  <measure xml:id="measure-L19" right="end" n="4">
                     <staff xml:id="staff-L19F1N1" n="1">
                        <layer xml:id="layer-L19F1N1" n="1">
                           <note xml:id="note-L20F1" dur="1" oct="4" pname="g" />
                        </layer>
                     </staff>
                  </measure>
               </section>
            </score>
         </mdiv>
      </body>
   </music>
</mei>

One row per pitch, one column per quarter note, 1 = the pitch sounds.

      bar 1   bar 2   bar 3   bar 4
A4    0000    0000    1111    0000
G4    0000    1111    0000    1111
F4    0001    0000    0000    0000
E4    0010    0000    0000    0000
D4    0100    0000    0000    0000
C4    1000    0000    0000    0000

For reference: the four bars in common staff notation, drawn by Verovio from the MEI tab.

Loading the score…

The same 11 notes take 60 bytes in Plaine & Easie, 75 in ABC, 104 in kern, 120 in EsAC, 1,886 in MusicXML and about 4,850 in MEI (as Verovio writes it; its random IDs change the length by a few bytes each time). The difference is not the notes but what else is recorded: title, metadata, an identifier for every element, layout. Six of the seven can be rendered back into the same score (part 8). The grid cannot, at least not without guessing (part 9).

Language models read music as text too: ChatMusician (2024) was trained on ABC (part 10).

2. Why encode music?

  • Notation is a selection made for a reader who knows the conventions: performance practice, tuning, how to realise an ornament, how repeats work.
  • A computer knows none of this. A scanned score is an image. To search, analyse, render or play music, a machine needs symbols with explicit meaning.
  • Written and performed differ. One written sign (a trill, a grace note, a fermata) stands for performed events that look different from what is written. MEI can keep both, for example @dur (written duration) next to @dur.ges (performed duration).
  • Three questions to ask of every system this semester:
    1. Who reads it? A singer, a lutenist, a cataloguer, a machine, a language model.
    2. What does it record? Pitch, duration, physical action, sound, structure, layout, metadata.
    3. What does it leave out, or leave to the reader?

A digital edition (MEI) encodes what a source says. Analysis (kern, note tables, piano rolls) treats notes as data. The same encoding has to serve both, and that is what this course is about.

3. Scores today: scans, PDFs and facsimile-based editions

Where do you meet scores today? In library digitisations (for example the digital collections of the Bayerische Staatsbibliothek), in PDF archives such as IMSLP, in publishers' editions, and in a few encoded editions. Most "digital" music is a picture of a page.

Form What the computer has You can… You cannot…
Scan (TIFF, JPEG; often served through a IIIF viewer) pixels zoom, compare sources, cite a page search for a melody, play it, transpose it
PDF page images in a container, at most a text layer download, print, share the same as a scan: the notes are still pictures
Encoded score (MusicXML, MEI, kern) symbols with meaning render, play, search, analyse see what the source looked like, unless it is linked
Facsimile-linked edition (MEI with <facsimile>) symbols and the page images, linked bar by bar all of the above, and check every bar against the source —

Access: who may use a score? That depends on rights: in the EU, a composer's works are free 70 years after the end of the year of their death, but an edition, an arrangement or a lyricist can still be protected, and countries differ.

Showcase: the eu-pd-composers viewer

The eu-pd-composers viewer (data and code on GitHub) applies the EU rule to the 3,369 composers on Wikipedia's List of 20th-century classical composers, with life dates from Wikidata:

  • 777 are public domain in the EU now (they died in 1955 or earlier), 1,870 are not yet, and 722 have no death date on record. 21 more follow on 1 January 2027.
  • 1,254 are matched to their IMSLP category, with links to 29,033 IMSLP work pages; 20,391 of those pages belong to composers who are public domain now.
  • You can filter by instrumentation, style, citizenship, popularity and the year the works become free.

Figures from the published viewer (dump r019, checked on 2026-10-14). The status is a heuristic based on the composer's death year only, not legal advice: it cannot see lyricists, editions, arrangements or posthumous publications. Check the work itself before you copy, perform or publish it.

Try it: whose music becomes free in the EU on 1 January 2027? Does IMSLP have scores of it?

Accessibility: who can use a score? A screen reader cannot read a scan aloud, a scan blurs when enlarged, and it cannot be turned into music Braille or played back. An encoded score can be all of these (music Braille comes up again in part 4).

Facsimile-based editions keep the source and the encoding together. MEI stores the page images in a <facsimile> element: one <surface> per page, one <zone> (a rectangle) per bar. Each bar of the music points to its rectangle:

<facsimile>
  <surface xml:id="surface_aea1770d9c33" n="1">
    <graphic target="https://api.digitale-sammlungen.de/iiif/image/v2/bsb00023199_00013/full/4134,/0/default.jpg"
             width="4134" height="5495"/>
    <zone xml:id="zone_6ff938b3aec349c48e" type="measure" ulx="852" uly="945" lrx="1593" lry="2290"/>
    <!-- one zone for each of the other 14 bars on the page -->
  </surface>
</facsimile>
...
<pb n="1" facs="#surface_aea1770d9c33"/>
<measure n="1" facs="#zone_6ff938b3aec349c48e"> ... </measure>

(Shortened from the file in Demo E below. The coordinates are pixels in the 4,134-pixel-wide version of the scan.) The encoding stays tied to its witness: every bar can be checked against the page, the source's page and system breaks are kept, and the editor's reading sits next to what the source shows. A traditional critical edition replaces the source with a new engraving and moves the differences into a critical report.

This is where the course is going. The tools of CAMAT, which we use from the next sessions on, grew out of encoding volumes of Denkmäler deutscher Tonkunst from the Bayerische Staatsbibliothek's scans: optical music recognition (OMR) page by page, correction in mei-friend, automatic checks, and measure zones that link every bar back to the scan.

Demo E: a facsimile-linked page

The first page of Dietrich Buxtehude's Sonata I in F major, Op. I, for violin, viola da gamba and harpsichord, as printed in Denkmäler deutscher Tonkunst, vol. 11 (1903). Left: the Bayerische Staatsbibliothek's scan, with the zones from the MEI file drawn on it. Right: the same MEI file rendered by Verovio, with the page's three systems. Point at a bar on either side to find it on the other; click to compare it close up. Tick Show the bar's MEI to read how the selected bar is encoded.

Look at bar 11. The 1903 print has the violin's C♯ as a sixteenth note; the encoding follows the first edition, printed in Buxtehude's lifetime, and makes it an eighth note, and an <annot> in the MEI file says so. With Show the bar's MEI ticked you can read it in the code: <staff n="1"> is the violin, n="2" the viola da gamba and n="3" the harpsichord's bass line; each <harm> holds a figure of the figured bass; and the <annot> points at the C♯ with plist. The scan stays the witness, and the encoding records where and why it departs from it. (The note calls the first edition "the 1696 print"; sources date Op. I to about 1694 and Op. II to 1696, so treat the year with care.) The full workflow (OMR, correction, checks) comes later in the course.

Source and licence

MEI: the first page (bars 1–15) of buxtehude_op1_01_sonata_f_major_corr.mei from the CAMAT corpus edition DdT 1, vol. 11: Dietrich Buxtehude, Instrumental Works (work in progress, commit 7af4ae8), MIT License; two ties into bar 16 on the next page were removed. Download the MEI file. Scan: Bayerische Staatsbibliothek, bsb00023199, image 13 (copy of the Hochschule für Musik und Theater München, Bibliothek), licensed CC BY-NC-SA 4.0. The page loads the scan from the library's IIIF server when you open it; it is not copied here.

Discussion: Is a facsimile-linked encoding more authentic than a clean modern edition? Authentic to what: the source, the work, the performance?

4. Notation for human readers: systems made for users

Click a source to see the notation itself. The manuscripts from St. Gallen and Munich open in the library's own viewer, where you can zoom in and turn the pages; the other pictures are on Wikimedia Commons.

System Records Computational analogy Look at a source
Greek alphabetic notation (Seikilos epitaph) symbols mapped to pitches lookup table the stele, 1st or 2nd century, National Museum of Denmark: the pitch signs are written above the text
Neumes (9th century onward) melodic contour, often without fixed pitch relative constraints Cantatorium of St. Gallen, about 922–925 (Stiftsbibliothek St. Gallen, Cod. Sang. 359, p. 23, on e-codices): neumes above the text, no staff lines
Guidonian hand, staff lines pitch as a position in space spatial lookup a hand with a small staff on each finger, 16th century (UC Berkeley, Music Library MS 1087)
Mensural notation durations that depend on context rule engine Hartmann Schedel's songbook, 1460s (Bayerische Staatsbibliothek, Cgm 810, f. 18r): one voice after another, no score
Lute and organ tablature where to put the fingers, not which pitch instruction set for one instrument organ: Buxheim Organ Book, about 1460–1470 (Bayerische Staatsbibliothek, Mus.ms. 3725, f. 9r): the top voice on a staff, the lower voices as letter names; vihuela (a Spanish relative of the lute): Miguel de Fuenllana, Orphénica lyra, Seville 1554: fret numbers on six lines, one line per course of strings
Jeongganbo (Korea, 15th century onward) time as a grid of cells temporal grid, close to a piano roll Daeak hubo (1759), a collection of court music, photographed in the Museum of Traditional Korean Music, Seoul: one cell per beat; Six Dragons Fly Again (ISMIR 2024) encodes jeongganbo for machine learning

Notations were also made for particular groups of users:

Notation Made for Look at an example
Shape notes singing schools: the shape of the note head gives the syllable "Star in the East" in The Southern Harmony (1854 edition)
Jianpu teaching and amateur music-making: scale degrees as numbers "Amazing Grace" in jianpu
Music Braille blind musicians, who read it with their fingers a summary of the signs
Klavarskribo keyboard players: the music runs down the page, low notes left and high notes right, and the staff lines are the black keys a keyboard with the staff growing out of it: a piano roll on paper

There was never one "music notation"; notations are tools built for a use, and encoding formats continue that tradition.

Chant is still encoded today, in GABC and Volpiano; Verovio reads both (the GABC example is in part 8).

5. Notation for machines: barrels, rolls, punched cards

When the reader is a machine, notation becomes a program: positions on a cylinder or holes in paper that trigger sound.

Click a source to see the machine or its notation: book scans on the Internet Archive, photographs on Wikimedia Commons, and a museum's reconstruction.

When What Why it matters here Look at a source
9th century, Baghdad Banū Mūsā, The Instrument That Plays by Itself: an automatic flute player driven by a rotating pinned cylinder often called an early programmable machine (a popular claim, not a consensus) a reconstruction built from the treatise at ZKM Karlsruhe (2015)
17th century onward pinned barrels in barrel organs and music boxes music stored as physical positions Salomon de Caus's water-driven barrel organ (1615), as redrawn in the 1911 Encyclopædia Britannica; Joseph Engramelle, La tonotechnie, ou l'art de noter les cylindres ("the art of notating cylinders"), Paris 1775, plate III: a march written out for a barrel, with the barrel's divisions numbered under the notes
1801–1804 Jacquard loom: punched cards control weaving patterns not music, but punched cards become the medium of music machines and computers a Jacquard loom with its chain of punched cards, National Museum of Scotland: each card controls one row of the weave
1843 Ada Lovelace's notes on the Analytical Engine if the relations of pitched sounds could be expressed in symbols, the engine might compose music: the encoding problem, stated before computers existed her Note A, p. 694, in Scientific Memoirs, vol. 3 (London, 1843): the sentence on music is in the upper half of the page
1890s Edwin Votey's piano player, sold by Aeolian as the Pianola (1897) executable notation for the mass market a roll in a Pianola (Pianola Museum, Amsterdam): the player steers the pointer along the wavy Metrostyle line, which sets the tempo; the first Pianola, photographed in 1922 when it was given to the Smithsonian
1904 Welte-Mignon reproducing piano (Edwin Welte and Karl Bockisch, Freiburg; public launch 1905) records tempo, dynamics and pedalling, not only notes Debussy's roll from Demo A in Stanford's viewer: zoom in to see the holes
1908 US manufacturers agree on an 88-note roll standard in Buffalo (most sources say December 1908; some give 1910 or 1911) a format standard: any roll plays on any instrument the box of an 88-note roll (Aeolian, Follies of 1910): "88 NOTE" on the label tells the buyer which players it fits
from 1948 Conlon Nancarrow punches his Studies for Player Piano by hand the roll as a medium for music no human can play Nancarrow at his punching machine and a page of his score of Study No. 37 (Paul Sacher Stiftung, Basel, which holds his rolls and manuscripts)

Demo A: Debussy plays Children's Corner

Debussy recorded Children's Corner himself for the Welte-Mignon (roll 2733). Stanford's SUPRA archive scanned the roll; you can see it in Stanford's viewer. SUPRA turned the scan into two MIDI files:

  • raw: every hole becomes an event: 33,508 holes in 4 tracks (bass and treble notes, and two tracks of expression holes that control loudness);
  • expressive: holes merged into notes and the expression holes turned into loudness and pedalling: 6,031 notes.

Compare the two versions over the same seconds. A long note on the roll is punched as a chain of holes: at 5.1 s the C3 is a 0.1-second hole in the raw file but a 3-second note in the expressive file. The expressive file joins such chains: the 28,365 holes in the raw file's two note tracks become 6,031 notes.

Is a roll a score, a recording or a program? Where are the bar lines and the quarter notes? What is a "note" when 33,508 holes become 6,031 notes?

Source and licence

MIDI files: SUPRA, Stanford University Piano Roll Archive, roll dc920rk8870, licensed CC BY-NC-SA 4.0. Cite: Zhengshan Shi, Craig Stuart Sapp, Kumaran Arul, Jerry McBride and Julius O. Smith III, "SUPRA: Digitizing the Stanford University Piano Roll Archive", ISMIR 2019. The page fetches the files from GitHub when you click the button. Notes are played by a plain synthesiser in your browser, not a piano.

6. Setup: Jupyter4NFDI

The second half of block 1 is for getting everyone into Jupyter4NFDI: see Setup on Jupyter4NFDI. If it does not work for you today, follow block 2 on this page, which needs no login.

7. Notation for computers, 1940s to the 1990s

Click a source to see or hear it. The encoded scores open in the Verovio Humdrum Viewer, which is built on Verovio and shows the code next to the notation.

When System People Made for Look at a source
1940s punched cards for Barbara Allen tune variants Bertrand Bronson sorting folk tunes into families the cards are not online; Barbara Allen as sung in a California migrant camp in 1940 (Library of Congress blog, two recordings) is one of the variants
1950–1951 CSIRAC (Australia) plays tunes; the BBC records the Ferranti Mark 1 in 1951 Christopher Strachey (Ferranti) computers play music (no CSIRAC recording survives; it was reconstructed) the music of CSIRAC (University of Melbourne): the reconstructed Colonel Bogey and the 1951 Ferranti recording; how the Ferranti recording was restored (British Library)
1957 Illiac Suite for string quartet Lejaren Hiller, Leonard Isaacson computers compose (the output was copied into a score by hand) the first page of the score, printed in their book Experimental Music (1959)
1960s DARMS (Ford-Columbia Music Representation) Stefan Bauer-Mengelberg computer-printed parts for the New York Philharmonic, typed by counting lines and spaces how DARMS numbers the lines and spaces (a diagram drawn for Wikipedia): the bottom line of the staff is 21
1964 Plaine & Easie Code Barry S. Brook, Murray Gould incipits typed on an ordinary typewriter; RISM adopted it for incipits and still uses it a RISM record of a Buxtehude sonata copied in the 1690s (Uppsala University Library): the incipit of each movement is stored as Plaine & Easie and drawn as notation
from 1982 EsAC and the Essen Folksong Collection Helmut Schaffrath folksongs on DOS PCs; pitches as scale degrees, inspired by jianpu Schlaf, Kindlein, schlaf, song K0001 of the collection in its kern version: the curly brackets mark the phrases of the EsAC original (EsAC itself is in part 8)
1983 MIDI instrument makers performance events between instruments, not notation: the piano roll's descendant the two MIDI files of the Debussy roll in Demo A; how MIDI works: addendum
1984 CCARH and MuseData Walter Hewlett large encoded repertories Corelli, Op. 1 No. 1, first movement in MuseData
1980s–1990s Humdrum toolkit and **kern David Huron analysis with Unix text tools, one column ("spine") per voice; kern is German for "core" Bach's chorale Aus meines Herzens Grunde (BWV 269), one spine per voice
1980s–1990s ABC Chris Walshaw folk tunes typed as letters; the text format behind music language models such as ChatMusician (2024) The Kesh, an Irish jig on The Session: dozens of settings, each in ABC and as notation

The 1960s needed music for printing (DARMS) and cataloguing (Plaine & Easie). The 1980s brought corpora for analysis (Essen, MuseData, kern). Folk song made the first large data sets because single melodic lines were simple enough to encode at scale. After 1999 came MEI (Perry Roland's XML schema, built on the principles of TEI) for editions, and from 2000 MusicXML (Recordare) for exchange between notation programs.

Dates that are given differently in different sources
  • DARMS: conceived 1961 (Wikipedia) or 1966 (CCARH wiki); funding is given as 1964–1976. Hence "1960s".
  • Humdrum: "1980s" in some sources, "1990s" in others; the toolkit was reviewed in Music Theory Online in 1996.
  • Illiac Suite: usually 1957; one source names a 1956 premiere.
  • Bronson: the punched-card study is dated "1940s" in the source found; the original article is not confirmed.
  • Lovelace: the wording of the quotation varies between transcriptions.

8. Today's formats in Verovio

Verovio reads all six text encodings from part 1 and renders them as notation. Edit any box: the score follows as you type. Show as MEI converts the text to MEI; Piano roll draws the notes as time × pitch, the view from Demo A.

Where each format comes from, and where to find real files in it (the history is in part 7):

Format Made by, when Made for Documentation Real files
ABC Chris Walshaw, 1980s–1990s folk and traditional tunes, typed and shared as plain text ABC standard 2.1 The Session: folk tunes, each setting in ABC and as notation
Humdrum **kern David Huron, 1980s–1990s analysis with text tools: one column ("spine") per voice the **kern reference Bach's 370 chorales on GitHub; open any kern file in the Verovio Humdrum Viewer
Plaine & Easie Barry S. Brook and Murray Gould, 1964 incipits in library catalogues; RISM still uses it the code, maintained by IAML RISM Online stores its incipits in it, for example in this Buxtehude sonata
EsAC Helmut Schaffrath, Essen, from 1982 folk-song collections: one record per song, with region (REG), key and metre (KEY) and the melody as scale degrees (MEL) how the Verovio Humdrum Viewer reads EsAC, with a record from the Essen collection the Essen Folksong Collection, over 6,000 songs, converted to kern
MusicXML Michael Good (Recordare), from 2000; now the W3C Music Notation Community Group exchange between notation programs MusicXML 4.0 (2021), with a tutorial a list of sites with MusicXML files; notation programs such as MuseScore export it
MEI Perry Roland, from 1999; now the MEI community scholarly editions the MEI Guidelines (version 5) MEI sample encodings on GitHub; the Buxtehude page in Demo E
GABC (chant) a monk of the Abbey of Sainte-Madeleine du Barroux, improved by Élie Roux and others typesetting Gregorian chant with the Gregorio software GABC, described by the Gregorio project GregoBase, a database of chants, each one downloadable as GABC

MIDI is not on this list: it records what a player does, not what a score shows. See the addendum.

Try this:

  1. Change a note in the ABC and in the kern version. Add a sharp (^F in ABC, f# in kern).
  2. Change the metre to 3/4 (M:3/4 in ABC, *M3/4 in kern). What does Verovio do with bars that no longer fit?
  3. Compare what each format records beyond the notes: the phrase brackets in EsAC, the spines in kern, <divisions> in MusicXML, the header and the xml:id of every element in MEI.
  4. Break the syntax on purpose and read the message under the score. (Some broken inputs make Verovio crash rather than complain; the page catches those and says what is wrong.)

9. Binary notation: holes, grids and what survives

The roll in Demo A is binary too: there is a hole at a position, or there is not. But a roll is an instruction for a machine: its time runs on continuously along the paper, and a long note is punched as a chain of holes. The grid below is something else: an array that a program computes from a list of notes, with one row per pitch and one column per time step (1 = the pitch sounds). CAMAT builds this kind of grid later in the course to search for patterns. Which bars of Alle meine Entchen can you no longer tell apart?

  • Bars 2 (two half notes), 3 (four quarter notes) and 4 (one whole note) all become a run of four cells. Pure occupancy loses where one note ends and the next begins.
  • Filling the grid needs a rule. The demo has two, under Rules. This demo's rule asks what sounds at the start of each cell: a note shorter than a cell can fall between two cell starts and disappear, and an eighth note on the beat grows to a quarter. CAMAT's rule asks what sounds at any time in the cell: no note is lost, but every note grows to fill whole cells, so on a quarter-note grid two eighth notes in one cell sound together, as a chord. Add eighth notes to the melody and switch the rules. Finer grids keep more but cost size; CAMAT's default cell is a sixteenth note.
  • Reading the grid back needs a rule too. Both rule sets read a run of 1s in one row as one held note, so 1111 comes back as a whole note in bars 2, 3 and 4 alike. The opposite rule, "every 1 starts a new note", would give only quarter notes: bar 3 right, bars 2 and 4 wrong. Without onsets, no rule gets all three right. This demo reads the grid column by column, as a sequence of chords, which works for one melodic line but cuts a note that is held while another pitch moves. CAMAT reads each row on its own. Click the C4 cells in the rest of bar 1 to hold the C under D, E and F, then switch the rules.
  • What would keep the note boundaries? A second grid that marks where notes start ("keep an onset grid", with this demo's rules), or a note table next to the grid, as CAMAT keeps.
  • Two grids are one grid with three values. With the onset grid ticked, each cell is in one of three states: 0 = silence, 1 = a note continues, 2 = a note starts. Two yes/no grids would allow four combinations, but "a note starts where nothing sounds" cannot happen, so three are left: the notation is no longer binary but ternary. Every cell that is not 0 still says "sounding", so a tool written for the 0/1 grid can read the new grid too. CAMAT has no onset layer yet, so the box is greyed out under CAMAT's rules; CAMAT keeps the onsets in its note table.
  • CAMAT does not read its grid back as the music. It slides a small grid, such as a motif, over the grid of a whole piece and scores each position by the cells they share. For every 1 it can keep a pointer to the notes in the score behind it, so a match is shown in the original score, not in a score rebuilt from 0s and 1s. Statistics such as how often each pitch occurs come from its note table, not from the grid. CAMAT's notebooks binary_roundtrip.ipynb and binary_pattern_search.ipynb show both.

10. An LLM writes ABC

ABC is compact text, and there are many tune collections online in it, so language models have seen a lot of it. We ask a model for a melody and check the answer. Because the answer is an encoding, we can test it, which a picture of a score would not allow.

Does every bar add up to the metre? Does Verovio warn? Are the key and the range what was asked for? Is the kern valid at all? Note that the checks rely on how Verovio reads the text: Verovio 6.3, for example, ignores a length written after a chord ([DFA]2), and the page warns when it sees one.

You do not need your own LLM account for this: the recorded answers and the instructor's demo are enough.

11. Wrap-up

  • The three questions once more: who reads it, what does it record, what does it leave out? Apply them to a scan, to the grid and to the LLM's ABC.
  • Where the course goes: digital edition (MEI, facsimiles) → CAMAT note tables and piano rolls → statistics and pattern search.

Homework

  1. If setup did not work in class, finish it with the setup page and run 01_formats_verovio.ipynb.
  2. Encode the first phrase of a melody of your choice in ABC or kern and render it (here in part 8 or in the notebook).
  3. Write down one thing about the melody that your encoding cannot express.

Addendum: MIDI on a cable and in a file

MIDI comes up in Demo A and in part 7. The name is used for two things: messages sent between instruments, and files that store them.

On a cable: messages, sent as they happen. MIDI 1.0 was published in August 1983 so that instruments from different makers could control each other. A message says what a player does, not what it sounds like: channel 1, key 60 down, velocity 90 is three bytes, 90 3C 5A, and a second message follows when the key is released. The receiving instrument makes the sound; MIDI itself carries no audio. The original cable has five-pin DIN plugs, carries messages in one direction only (from an OUT to an IN socket) at 31,250 bits per second, and has 16 channels, so one cable can play 16 instruments or sounds. A message carries no time: the note starts when the message arrives, and each three-byte message takes about a millisecond on that cable. Today the same messages also travel over USB, Bluetooth and networks, or between programs on one computer.

Here is Alle meine Entchen on the cable, as a sequencer sends it when it plays the file further down. A MIDI monitor, a program that lists the messages coming in, shows each one with the moment it arrived. That time is the monitor's; the cable carries only the bytes.

arrived (s)  bytes      meaning
 0.000       90 3C 5A   key 60 (C4) down, velocity 90
 0.496       90 3C 00   key 60 up
 0.500       90 3E 5A   key 62 (D4) down
 0.996       90 3E 00   key 62 up
 1.000       90 40 5A   key 64 (E4) down
 1.496       90 40 00   key 64 up
 1.500       90 41 5A   key 65 (F4) down
 1.996       90 41 00   key 65 up
 2.000       90 43 5A   key 67 (G4) down
 2.996       90 43 00   key 67 up
 3.000       90 43 5A   key 67 (G4) down
 3.996       90 43 00   key 67 up
 4.000       90 45 5A   key 69 (A4) down
 4.496       90 45 00   key 69 up
 4.500       90 45 5A   key 69 (A4) down
 4.996       90 45 00   key 69 up
 5.000       90 45 5A   key 69 (A4) down
 5.496       90 45 00   key 69 up
 5.500       90 45 5A   key 69 (A4) down
 5.996       90 45 00   key 69 up
 6.000       90 43 5A   key 67 (G4) down
 7.996       90 43 00   key 67 up

That is the whole tune: 22 messages, 66 bytes, eight seconds. Nothing in it says bar, quarter note or tempo. A G lasts one second because its key-up arrives one second after its key-down. A person playing the tune on a keyboard sends the same kinds of messages, at less even times and with a different velocity for each key.

The MIDI specification also has messages it calls real-time messages, in a narrower sense: one-byte messages that keep several machines in step. A sequencer that drives a drum machine sends FA (Start), then F8 (Timing Clock) 24 times per quarter note, at 120 quarter notes a minute one every 21 milliseconds, and FC (Stop) at the end. They carry the beat, but no notes.

In a file: the same messages, each with a wait before it. A Standard MIDI File (.mid) stores the messages and writes before each one how long to wait after the previous one, counted in ticks. The header says how many ticks make a quarter note, and a tempo message says how long a quarter note lasts. Verovio writes the four bars of Alle meine Entchen as a file of 144 bytes. Here is all of it:

4D 54 68 64  00 00 00 06       "MThd": the header, 6 bytes follow
00 01  00 02  00 78            format 1, 2 tracks, 120 ticks per quarter
4D 54 72 6B  00 00 00 0B       "MTrk": track 1, 11 bytes follow
  00     FF 51 03 07 A1 20     wait 0, tempo
  00     FF 2F 00              wait 0, end of track
4D 54 72 6B  00 00 00 67       "MTrk": track 2, 103 bytes follow
  00     FF 58 04 04 02 18 08  wait 0, time signature 4/4
  00     90 3C 5A              wait 0, key 60 (C4) down, velocity 90
  77     90 3C 00              wait 119, key 60 up
  01     90 3E 5A              wait 1, key 62 (D4) down
  77     90 3E 00              wait 119, key 62 up
  01     90 40 5A              wait 1, key 64 (E4) down
  77     90 40 00              wait 119, key 64 up
  01     90 41 5A              wait 1, key 65 (F4) down
  77     90 41 00              wait 119, key 65 up
  01     90 43 5A              wait 1, key 67 (G4) down
  81 6F  90 43 00              wait 239, key 67 up
  01     90 43 5A              wait 1, key 67 (G4) down
  81 6F  90 43 00              wait 239, key 67 up
  01     90 45 5A              wait 1, key 69 (A4) down
  77     90 45 00              wait 119, key 69 up
  01     90 45 5A              wait 1, key 69 (A4) down
  77     90 45 00              wait 119, key 69 up
  01     90 45 5A              wait 1, key 69 (A4) down
  77     90 45 00              wait 119, key 69 up
  01     90 45 5A              wait 1, key 69 (A4) down
  77     90 45 00              wait 119, key 69 up
  01     90 43 5A              wait 1, key 67 (G4) down
  83 5F  90 43 00              wait 479, key 67 up
  00     FF 2F 00              wait 0, end of track

The numbers are hexadecimal: 77 is 119, 3C is 60, and the tempo 07 A1 20 is 500,000 microseconds per quarter note, 120 quarter notes a minute. A wait longer than 127 ticks takes two bytes: 81 6F is 1 × 128 + 111 = 239 (a half note), 83 5F is 3 × 128 + 95 = 479 (a whole note). In 90, the 9 means "key down" and the 0 channel 1. Each key goes up one tick before the next key goes down, so the two Gs of bar 2 are struck twice, not held as one note. The 90 messages are exactly the ones on the cable above; only the waits are new. The FF messages (tempo, time signature, end of track) exist only in files; on a cable, FF alone means Reset. A key "down" at velocity 0 counts as released: the MIDI 1.0 specification allows both this and a separate Note Off message.

On a cable (or USB, Bluetooth) In a file (.mid)
What it is a live stream of messages the same messages, stored
Time the moment a message arrives a wait in ticks before each message, and a tempo
Parts up to 16 channels per cable channels, and tracks played together (format 1)
Example a keyboard playing a synthesiser the two files of the Debussy roll in Demo A

The three questions, asked of MIDI.

  • Who reads it? Instruments, sequencers and recording software; SUPRA turned the Debussy roll into MIDI files.
  • What does it record? Which key, how hard (velocity, 0 to 127), when, on which channel, and controllers such as the pedals: the pedal bands in Demo A are Control Change messages 64 (sustain pedal) and 67 (soft pedal).
  • What does it leave out? What a score adds for a reader. C♯ and D♭ are the same key, 61. A note lasts so many ticks; it has no note value, so a short quarter note and an eighth note followed by a rest can look the same. There are no stems, beams, voices, ties, slurs or staff layout.

Why MIDI does not simply turn back into MEI. Turning a score into MIDI is easy: Verovio does it, and there is one right answer. The way back is not. Verovio itself cannot read MIDI (6.3.0 answers "Input format 'midi' is not supported"). A program that imports MIDI into a score has to put back what the score had and the file dropped, and that means guessing. Even for the file above, which a computer wrote from a score, a converter has to decide:

  • Note values. The file says "119 ticks", not "quarter note". Here rounding is easy, because every note is one tick short of a quarter, half or whole note. A person playing the same tune holds the keys for uneven times and drifts in tempo, so a converter must first find the beat and then round each length to a note value (quantisation). Is a short note a staccato quarter, or an eighth followed by a rest? Are three quick notes a triplet or a dotted rhythm? In Demo A's expressive Debussy file, the tempo changes all the time.
  • Metre and bars. The cable carries no bars at all. A file may contain a time signature (Verovio writes FF 58, 4/4) or not. Does the tune start on the first beat, or with an upbeat?
  • Pitch spelling. Key 61 is C♯ or D♭, key 70 A♯ or B♭. The converter has to guess the key and spell each note from it. A file may contain a key signature (FF 59), but it does not have to; Verovio wrote none for this tune.
  • Voices and staves. A piano part is one stream of keys. Which notes belong to the melody and which to the accompaniment? Which go on the upper staff and which on the lower? Where does a voice need a rest?
  • What MIDI never had. Slurs, dynamics (only velocities: 90 for every note here), articulation, beams, layout, and ornaments: a played trill arrives as many short notes. What was not recorded cannot be converted back.

Each of these questions has several possible answers, and different programs answer them differently. Demo C's "Back from the grid to notes" in part 9 is a tiny converter of this kind. It is given the metre, writes every black key as a sharp, chooses the treble or bass clef from the middle pitch, puts all notes into one voice, and needs the onset grid to keep repeated notes apart. MIDI does not need that last help: every note has its own key-down and key-up.

Notebooks

The notebooks repeat the demos with inputs you can change. On Jupyter4NFDI, one cell copies them into your workspace (see setup); you can also download them here and upload them. Each one starts with a cell that installs what it needs (Verovio 6.3.0, mido, matplotlib, ipywidgets, and Bokeh for interactive piano rolls).

Sources