Delay a Shoutcast stream by 12 hours (Linux/bash)

Hey,

I am living on the other side of the world from my home (GMT+1 now, GMT+13 is home), and I miss my old terrestrial radio station. It has a Shoutcast stream, and I would like to simply delay it by 12 hours so that it is always available when I want to listen to it, in a way that would make its timezone be synchronised to my timezone.

I envision this as a script being run on my server host.

A naive approach would simply be to allocate enough ram in a ringbuffer to store the entire 12 hour delay, and pipe in the output from streamripper. But the stream is a 128kbps mp3, which would mean (128/8) * 60 * 60 = ~56MB per hour, or 675MB for the whole 12 hour buffer, which isn't really so practical. Plus, i might have to deal with my server host just killing the process after a certain timeout.

So, what are some strategies that might actually be practical?

From stackoverflow

• Why don't you just download it with a stream ripper like Ripshout or something?

  Karl : I like to rip the stream so I can listen to shows whenever, not just exactly 12 hours later...

  damian : That's not my aim. I want to replicate the terrestrial nature of the original station, basically. The show that's on 9pm on Monday back home will also be on at 9pm on Monday where I am.

  Echostorm : It probably is your best option. Most shoutcast streams will boot you after 23 or so hours at best and caching to disk is infinitely better than ram. A good ripper will break up the different shows into separate files. When you're ready you could just play that show and delete the others.

  damian : The stream is old-school terrestrial radio piped into a web streamer, so there's no show metadata, so no show separation. The point is to take away my decision-making requirement. I just turn on 'the radio', and if it's good I leave it on. I can't believe it's not possible.

• A stream ripper would be the Easy way, and probably the Right way, but if you want to do it the Programmer way....

  • Most development machines have quite a bit of RAM. Are you SURE you can't spare 675 MB?
  • Rather than store the output in a buffer can't you store it in a file or files(s), say an hour at a time? (essentially, you would be writing your own stream ripper)
  • Convert the stream to a lower bitrate, if you can tolerate the loss in quality

• to answer my own question, here's a script that starts up as a cron job every 30 minutes. it dumps the incoming stream in 5-minute chunks (or set by FILE _ SECONDS ) to a particular directory. block borders are synchronised to the clock, and it doesn't start writing until the end of the current time chunk, so the running cronjobs can overlap without doubling up data or leaving gaps. files are named as (epoch time % number of seconds in 24 hours).str .

  i haven't made a player yet, but the plan was to set the output directory to somewhere web-accessible, and write a script to be run locally that uses the same timestamp-calculating code as here to sequentially access (timestamp 12 hours ago).str, tack them back together again, and then set up as a shoutcast server locally. then i could just point my music player at http://localhost:port and get it.

  edit: New version with timeouts and better error condition checking, plus nice log file. this is currently running hitch-free on my (cheap) shared webhost, with no problems.

  #!/usr/bin/python
  import time
  import urllib
  import datetime
  import os
  import socket
  
  # number of seconds for each file
  FILE_SECONDS = 300
  
  # run for 30 minutes
  RUN_TIME = 60*30
  
  # size in bytes of each read block
  # 16384 = 1 second
  BLOCK_SIZE = 16384
  
  MAX_TIMEOUTS = 10
  
  # where to save the files
  OUTPUT_DIRECTORY = "dir/"
  # URL for original stream
  URL = "http://url/path:port"
  
  debug = True
  log = None
  socket.setdefaulttimeout(10)
  
  class DatestampedWriter:
  
   # output_path MUST have trailing '/'
   def __init__(self, output_path, run_seconds ):
    self.path = output_path
    self.file = None
    # needs to be -1 to avoid issue when 0 is a real timestamp
    self.curr_timestamp = -1
    self.running = False
    # don't start until the _end_ of the current time block
    # so calculate an initial timestamp as (now+FILE_SECONDS)
    self.initial_timestamp = self.CalcTimestamp( FILE_SECONDS )
    self.final_timestamp = self.CalcTimestamp( run_seconds )
    if debug:
     log = open(OUTPUT_DIRECTORY+"log_"+str(self.initial_timestamp)+".txt","w")
     log.write("initial timestamp "+str(self.initial_timestamp)+", final "+str(self.final_timestamp)+" (diff "+str(self.final_timestamp-self.initial_timestamp)+")\n")
  
    self.log = log
  
   def Shutdown(self):
    if self.file != None:
     self.file.close()
  
   # write out buf
   # returns True when we should stop
   def Write(self, buf):
    # check that we have the correct file open
  
    # get timestamp
    timestamp = self.CalcTimestamp()
  
    if not self.running :
     # should we start?
     if timestamp == self.initial_timestamp:
      if debug:
       self.log.write( "starting running now\n" )
       self.log.flush()
      self.running = True
  
    # should we open a new file?
    if self.running and timestamp != self.curr_timestamp:
     if debug:
      self.log.write( "new timestamp "+str(timestamp)+"\n" )
      self.log.flush()
     # close old file
     if ( self.file != None ):
      self.file.close()
     # time to stop?
     if ( self.curr_timestamp == self.final_timestamp ):
      if debug:
       self.log.write( " -- time to stop\n" )
       self.log.flush()
      self.running = False
      return True
     # open new file
     filename = self.path+str(timestamp)+".str"
     #if not os.path.exists(filename):
     self.file = open(filename, "w")
     self.curr_timestamp = int(timestamp)
     #else:
      # uh-oh
     # if debug:
     #  self.log.write(" tried to open but failed, already there\n")
     # self.running = False
  
    # now write bytes
    if self.running:
     #print("writing "+str(len(buf)))
     self.file.write( buf )
  
    return False
  
   def CalcTimestamp(self, seconds_offset=0):
    t = datetime.datetime.now()
    seconds = time.mktime(t.timetuple())+seconds_offset
    # FILE_SECONDS intervals, 24 hour days
    timestamp = seconds - ( seconds % FILE_SECONDS )
    timestamp = timestamp % 86400
    return int(timestamp)
  
  
  writer = DatestampedWriter(OUTPUT_DIRECTORY, RUN_TIME)
  
  writer_finished = False
  
  # while been running for < (RUN_TIME + 5 minutes)
  now = time.mktime(datetime.datetime.now().timetuple())
  stop_time = now + RUN_TIME + 5*60
  while not writer_finished and time.mktime(datetime.datetime.now().timetuple())<stop_time:
  
   now = time.mktime(datetime.datetime.now().timetuple())
  
   # open the stream
   if debug:
    writer.log.write("opening stream... "+str(now)+"/"+str(stop_time)+"\n")
    writer.log.flush()
   try:
    u = urllib.urlopen(URL)
   except socket.timeout:
    if debug:
     writer.log.write("timed out, sleeping 60 seconds\n")
     writer.log.flush()
    time.sleep(60)
    continue
   except IOError:
    if debug:
     writer.log.write("IOError, sleeping 60 seconds\n")
     writer.log.flush()
    time.sleep(60)
    continue
    # read 1 block of input
   buf = u.read(BLOCK_SIZE)
  
   timeouts = 0
   while len(buf) > 0 and not writer_finished and now<stop_time and timeouts<MAX_TIMEOUTS:
    # write to disc
    writer_finished = writer.Write(buf)
  
    # read 1 block of input
    try:
     buf = u.read(BLOCK_SIZE)
    except socket.timeout:
     # catch exception but do nothing about it
     if debug:
      writer.log.write("read timed out ("+str(timeouts)+")\n")
      writer.log.flush()
     timeouts = timeouts+1
  
    now = time.mktime(datetime.datetime.now().timetuple())
   # stream has closed,
   if debug:
    writer.log.write("read loop bailed out: timeouts "+str(timeouts)+", time "+str(now)+"\n")
    writer.log.flush()
   u.close();
   # sleep 1 second before trying to open the stream again
   time.sleep(1)
  
   now = time.mktime(datetime.datetime.now().timetuple())
  
  writer.Shutdown()