Friday, April 29, 2011

Limiting a script from sending alerts

I've got a bash script which detects a failed system component running on a unix box. The script sends out an email on failure. The script runs via run every minute (via cron). What's the easiest way to throttle sending the alerts to something like every 15 minutes?

Could I create/update a file when I send an alert and only send the file if the date of said file is so many minutes old?

From stackoverflow
  • You could implement a go-between for the script sending alerts. Instead of sending an email 'directly' (via sendmail, another mail app) you could send it to another script which would actually make the call. This script could then track (using another file) the last time at which a mail was sent. This way, you could check how often you're sending them out. :)

  • Something like this might work:

    stamp=/tmp/mystamp
    # create stamp file, if we haven't yet
    [ ! -f $stmp ] && touch $stamp
    tmp=$(tempfile)
    
    # see if 15 minutes has passed...
    diff=$(echo $(date -d "15 minutes ago"  +%y%d%m%H%M) - $(date -d "$(stat $tmp |grep Change |cut -d: -f2-)" +%y%d%m%H%M)  |bc)
    rm $tmp
    
    # if we last touched the stamp file less than 15 minutes ago
    # then quit
    [ $diff -le 0 ] && exit 0
    
    # update the stamp file
    touch $stamp
    
    # do your thing...
    echo 'Warning! Warning!' |mail -s "FOOBAR" your@address.com
    
  • If you encounter the error condition, touch a temp file (say /tmp/alert.email). Next in the script, check to see if the file exists and if it does and the # of minutes is 0,15,30,45, then send an email. I'm not sure if this does exactly what you want. Basically it:

    • Sends an email no more often than once every 15 minutes.
    • Sends an email for all error conditions, but not necessarily immediately.
    • May send an error message immediately or up to 14 minutes later.

    So have something later like:

    #!/bin/bash 
    
    MIN=`date '+%M'`
    
    if [ *ERROR_COND* ] ; then
       touch /tmp/alert.email
    fi
    
    if [[ -f /tmp/alert.email &&  ( $MIN = "15" || $MIN = "30" || $MIN = "45" || $MIN = "00" ) ]] ; then
       # email here
       rm -f /tmp/alert.email
    fi
    

0 comments:

Post a Comment