John Linnell wrote the song's music and the chorus (all seven words of it). Linnell on the song (from a 1994 interview):
On some songs, like for example, "Puppet Head"...I wrote the song, and I didn't like the verses. So I gave it to John [Flansburgh], and he filled in all the blanks.
The spoken break ("Memo to myself...") was added at the last minute during the recording of 1985 Demo Tape.
The lyrics "Quit my job down at the car wash / Didn't have to write no-one a good-bye note" are most likely inspired by the Elvis Presley song "Guitar Man", which begins 'I quit my job down at the car wash / Left my mamma a good-bye note".
The synthesized instrument heard playing arpeggios for most of the song and playing chords through the chorus is the "funny" preset on the Casio MT-100 keyboard.
The music video, made before the album came out, uses a slightly different mix of the song. This mix is also featured on the Hello re-issue of the 1985 Demo Tape.
Per the commentary from Direct From Brooklyn, the music video for this song, directed by Adam Bernstein, was filmed on the tails (pieces of film cut if a whole scene cannot fit on them) of the movie Married to the Mob. It was shot in an empty lot next to Con Edison plant, at North 12th St & Kent Ave in Greenpoint, Brooklyn. This was also the shooting location for the duo couch interview scenes in Gigantic.
In the original script for Spike Jonze and Charlie Kaufman's film Being John Malkovich, this song was going to run over the end credits.
The song is referenced in an episode of The Simpsons, "Homer's Odyssey," where Homer writes a suicide note on a memo that has the heading "Dumb things I gotta do today."
The song can be heard in the background of a bar scene from the 1988 John Cusack movie Tapeheads.