Timeshifting NPR's news programs

Updated Dec. 24, 2005: Fix code for MPlayer command, include Ogg Vorbis encoding, include regexp matching to download partial programs, include special code for OTM. And probably a few bug fixes.

Updated Mar. 21, 2005: Now record multiple NPR streams to increase speed. Also, a few minor changes in default behavior from the previous version. Sorry for any inconveniences, but I'm just a part-time geek, not a reliable developer. If you use these scripts, please donate to National Public Radio.

Thanks.

I live in the Netherlands and commute over an hour to work each day. I really missed listening to NPR on my way to work and so I wrote some scripts to stream NPR's news programs overnight and put them either on a CD or an mp3 player.

My solution is perhaps idiosyncratic. I wrote it for my personal use and not for a broad userbase. There is a more complete HOWTO and set of tools for timeshifting radio programs at Linux Radio Timeshift HOWTO, but maybe someone will find my script useful, too. Anyway, when I started looking into this problem, that HOWTO wasn't around yet.

NPR Podcast news: NPR is currently podcasting some of their articles. It should be fairly simple to just grab the podcast from the source, but you won't get the entire broadcast of ATC, say. Check out NPR's Podcast Directory for details.

Overview

The main script is npr-grabber.sh. This script uses mplayer to stream the program and save the streams as WAVE files (exception: See OTM details). Then it either burns a CD or converts the WAVE file to MP3s. It records multiple streams at once --- see below.

The basic usage is

npr-grabber.sh [options] PROGCODE

PROGCODE is NPR's program code for the program. Here are the codes I use regularly. They must be capitalized.

Show	Code
Morning Edition	ME
All Things Considered	ATC
Weekend Edition Saturday	WESAT
Weekend Edition Sunday	WESUN
Fresh Air	FA
On The Media	OTM

Options include the following.

`-d` date	Fetch the program for the given date.
`-r` root	Choose alternate root directory.
`-x`	Write CD.
`-m`	Create MP3s (default).
`-w`	Do not create MP3s.
`-o`	Create Oggs.
`-a`	Do not delete old files.
`-n`	Maximum number of streams recorded at once (Default: 10).
`-1`	Just one stream at a time

Audio files are saved in /storage/multimedia/npr/PROGCODE, and MP3s in the subdirectory mp3. After all the MP3s are created, they're copied to /mnt/stick/PROGCODE. I mount my MP3 player at /mnt/stick.

mplayer does a good job saving streamed Real audio streams, but there is one annoying problem. It takes some time before mplayer notices it has reached the end of stream --- a minute or two. This isn't a problem for me, since I run the script as a cron job. Here are the relevant entries for my crontab (I create both MP3s and an audio CD for the car).

    00 4 * * * /home/jesse/bin/npr-grabber.sh -m -d yesterday ATC >/storage/multimedia/npr/ATC/log

    00 18 * * mon-fri  /home/jesse/bin/npr-grabber.sh -m ME >/storage/multimedia/npr/ME/log

    00 19 * * sat  /home/jesse/bin/npr-grabber.sh -m WESAT > /storage/multimedia/npr/WESAT/log

    00 19 * * sun  /home/jesse/bin/npr-grabber.sh  -m WESUN > /storage/multimedia/npr/WESUN/log

You'll need to check when the shows are available for you. They become available shortly after they finish broadcasting on the West Coast.

When you run the script, it deletes any old audio files from previous shows (unless you use the -a flag). So if you want to save a program, move it out of /storage/multimedia/npr!

Multiple streams at once

The latest version of npr-grabber.sh fetches ten streams at once by default. This means that an hour and a half program can be fetched in under thirty minutes, typically (including making the mp3s).

You may want to test this a time or two to see if the result works well for you and adjust the number fetched by -n. There seems to be a limit in the number of streams that can be recorded all at once --- when I tried twenty, most of the streams were unlistenable screeches. No more than one version of lame is run at a time, since multiple versions slow the process rather than speed it (lame is bloody well CPU intensive).

Each instance of mplayer is put into its own screen, so you can keep tab of how the streaming and encoding is going via screen -r.

OTM details

On The Media is podcast-friendly. They provide an mp3 of each show available for download at their homepage, so you don't really need my script to listen to this program.

However, I didn't want one large mp3 of all the segments. I prefer each segment in a different file. So I wrote a simple script to do this (otm-grabber.sh) and altered npr-grabber.sh to use otm-grabber. otm-grabber fetches the mp3s for each segment and changes the id3 tags slightly. It is very fast, since it does not use streaming at all.

Downloads

Here are all of the scripts I use. You should have a look at them and see if any directories, etc., need to be changed. If you have any questions, just ask.

`npr-grabber.sh`	The main script, described above
`npr-grab-seg.sh`	Helper that grabs and encodes individual segments.
`otm-grabber.sh`	Helper for grabbing individual segments of On The Media.
`npr-player.sh`	A simple script for playing the streamed program directly from the hard drive.
`cd-write-audio.sh`	A simple script for writing the audio CD.

My own personal pledge drive