Linnell wrote the chorus (all seven titular words of it). Linnell on the song (from a 1994 interview):
On some songs, like for example, "Puppet Head"...I wrote the song, and I didn't like the verses. So I gave it to John [Flansburgh], and he filled in all the blanks.
The spoken break ('Memo to myself...') was added at the last minute during the recording of 1985 Demo Tape.
The lyrics 'Quit my job down at the car wash / Didn't have to write no-one a good-bye note' are most likely inspired by the Elvis Presley song 'Guitar Man' that begins 'I quit my job down at the car wash / Left my mamma a good-bye note'.
The synthesized instrument heard playing arpeggios for most of the song and playing chords through the chorus is the "FUNNY" preset on the Casio MT-100 keyboard.
Per the commentary from Direct From Brooklyn, the video was filmed on the tails (pieces of film cut if a whole scene cannot fit on them) of the movie Married to the Mob.
Filming location: empty lot next to Con Edison plant, at North 12th St & Kent Ave in Greenpoint, Brooklyn (also the shooting location for the duo couch interview scenes in Gigantic)
In the original script for Spike Jonze and Charlie Kaufman's film Being John Malkovich, this song was supposed to run over the end credits.
The song is referenced in an episode of The Simpsons, "Homer's Odyssey," where Homer writes a suicide note on a memo that has the heading "Dumb things I gotta do today."