As I was running the script, I've noticed it ran very slow. I have left it for a few days to run (700K executions), I've noticed the speed increased linearly over time.

As Rundeck has both DB and files to store its data, it looked to me that it was the file system which was the slow part. Sure enough, when I tried to access the directory where the execution data was stored, it was slow - 1.4M files would take a while to get organised :)

To short out the process even more, I've decided that removing old files should be in order:

find :/var/lib/rundeck/logs/rundeck/tasks/job \
  -type f \
  -mtime +35 \
  -exec rm {} \;

This command would find all files with a creation date older than 35 days and remove them. We keep a buffer of 5 days compared to the python script (which actually cleans the DB properly) and we assume that no mischief was done to the file system :)

If you're patient, the command above would clean the ridiculaously large directory (still slow, but much faster than scanning the directory every some number of rm commands).

If you're curious, you can monitor the progress with something like:

do df -BM |grep sda1 ; sleep 5; done

which would display how the disk begins to free itself up bit by bit.

HTH,