How can I run the first process from a list of processes stored in a file and immediately delete the first line as if the file was a queue and I called "pop"?

I'd like to call the first command listed in a simple text file with \n as the separator in a pop-like fashion:

---

Figure 1:

cmdqueue.lst :

proc_C1
proc_C2
proc_C3
.
.

---

Figure 2:

Pop the first command via popcmd:

proc_A | proc_B | popcmd cmdqueue.lst | proc_D

---

Figure 3:

cmdqueue.lst :

proc_C2
proc_C3
proc_C4
.
.

From stackoverflow

• I think you would need to rewrite the file - e.g. run a command to list all lines but the first, write that to a temporary file and rename it to the original. That could be done using tail or awk or perl depending on the commands you have available.

• pop-cmd.py:

  #!/usr/bin/env python
  import os, shlex, sys
  from subprocess import call
  filename = sys.argv[1]
  lines = open(filename).readlines()
  if lines:
      command = lines[0].rstrip()
      open(filename, "w").writelines(lines[1:])
      if command:
          sys.exit(call(shlex.split(command) + sys.argv[2:]))
  

  Example:

  proc_A | proc_B | python pop-cmd.py cmdstack.lst | proc_D
  

  Svante : Python is a bit overkill for this.

  J.F. Sebastian : @Harleqin: The script is cross-platform. It has no dependencies other than the python itself. What alternative could you suggest?

  secr : This shlex-thing is really useful.

  bruceatk : How do you prevent overwriting an update that occurs between your read and write?

  J.F. Sebastian : @bruceatk: It doesn't prevent it. It is a good question how to do it in a simple portable way.

• If you want to treat a file like a stack, then a better approach would be to have the top of the stack at the end of the file.

  Thus you can easily cut off the file at the beginning of the last line (= pop), and simply append to the file as you push.

• Ooh, that's an amusing one-liner.

  Okay, here's the deal. What you want is a program that, when called, prints the first line of the file to stdout, then delete that line from the file. Sounds like a job for sed(1).

  Try

  proc_A | proc_B | `(head -1 cmdstack.lst; sed -i -e '1d' cmdstack.lst)` | proc_D
  

  I'm sure that someone who had already had their coffee could change the sed program to not need the head(1) call, but that works, and shows off using a subshell ("( foo )" runs in a sub-process.)

  Svante : Nice. Your parentheses do not match, by the way.

  Charlie Martin : Augh. I said I was pre-coffee. Thx.

  J.F. Sebastian : Does it work on Windows?

  Charlie Martin : Point. Did you know these comments must have 10 characters?

• You can't write to the beginning of a file, so cutting out line 1 would be a lot of work (rewrite the rest of the file (which isn't actually that much work for the programmer (it's what every other answer post has written for you :) ) ) ).

  I'd recommend keeping the whole thing in memory and using a classic stack rather than a file.

  Svante : counterpoint: sed -i

  Rob Kennedy : How do you keep the whole thing in memory and still get the pipeline described in the use case? For a small file, rewriting it really isn't all that much work anyway.

• You can use a little bash script; name it "popcmd":

  #!/bin/bash
  cmd=`head -n 1 $1`
  tail -n +2 $1 > ~tmp~
  mv -f ~tmp~ $1
  $cmd

  edit: Using sed for the middle two lines, like Charlie Martin showed, is much more elegant, of course:

  #!/bin/bash
  cmd=`head -n 1 $1`
  sed -i -e '1d' $1
  $cmd

  edit: You can use this exactly as in your example usage code:

  proc_A | proc_B | popcmd cmdstack.lst | proc_D

• I assume that you are constantly appending to the file also, so rewriting the file puts you in danger of overwriting data. For this type of task I think you would be better using individual files for each queue entry, using date/time to determine order, and then as you process each file you could append the data to a log file and then delete the trigger file.

  Really need more information in order to suggest a good solution. It's important to know how the file is getting updated. Is it a lot of separate processes, just one process, etc.