Yesterday I tried to set up Authentik, but it was failing in gunicorn with an obscure message that a file could not be accessed. I got very frustrated and enabled debug logging in Authentik's server, and I let it run.

Big mistake! I forgot about it and the next day I noticed that none of the other processes were running properly! Ouch! :)

The debug logging was spewing messages every second and... it filled up the empty space (23G!). I remembered that sometimes removing the file would just create a new one, so I did. This time instead, the logger was just offsetting to the end and trying to append a new line. So I had to:

  1. Stop the container
  2. Remove the log file
  3. Recreate the container with normal (info) logging

Now I'm monitoring it to see how quickly fills up the space and I'm looking to move the logging to a rotating logger, or, better yet, to a prometheus instance.

HTH,