elastic Monitor Docker Logs with ELK - 2 This second post in the series provides a starting point for visualising the log data. It makes some assumptions that logs are collected in a way described by the first part, so I would suggest to read through it if you haven't already. Verify First, make sure you
docker Monitor Docker Logs with ELK - 1 Following my attempts to monitor ethOS data via ElasticSearch and Kibana, I've thought it would be a good idea to start monitoring the logs produced by my various docker instances. This first part deals with configuring ELK and logspout. Prerequisites I assume the following: * ELK is installed. You
configure Monitor ethOS Distro with Elastic Search - 2 Following the previous post, we are now in position to install the monitoring code on the monitoried machine, of course. The easiest way is to download the code on the machine in /home/ethos. Following steps will assume you have the unpacked code in /home/ethos/ethos-elasticsearch Once you have
elasticsearch Monitor ethOS Distro with Elastic Search - 1 TL;DR: It's possible to monitor your ethos distro install on an ElasticSearch instance. I like EthOS. I like its panel too, but at the time of writing it had some issues on updating values This was somewhat unfortunate because it's the same time when I
command Docker Exec Path-based Commands TL;DR: Use sh -c "command with arguments" I've tried to execute docker exec <container> du / in order to see the size used by a container's logs. Unfortunately, this resulted in an error along the lines: du: C:/Program Files/Git: No
convert Fix *standard_init_linux.go:XXX: exec user process caused "no such file or directory"* Error TL;DR: Use dos2unix While switching from Ubuntu to Alpine images, I've encountered the fillowing error: standard_init_linux.go:195: exec user process caused "no such file or directory" Quite unfortunate, because the script stayed in the same spot and only the image changed. After
400 HTTP 400 Bad Request with docker-maven-plugin TL;DR: Check your Dockerfile (try to build it manually) before anything else! Today I was struggling with a 400 error: Exception caught: Request error: POST https://192.168.99.101:2376/build?t=localhost:5000/eris-django:2.0.1-ivanlla: 400: HTTP 400 Bad Request -> [Help 1] when
docker Initialise Python with GoCD Following the deployment of a python-aware agent, we can start creating a pipeline. The first stage would be to: 1. Clean up any previous data 2. Create a virtual environment for python 3. Install all dependencies from a requirements.txt file in our project Note: we assume the global pipeline
cd Create a Python-friendly GoCD Agent in Docker I've got two requirements to make GoCD useful: 1. Run both the server and agent(s) in Docker 2. Make it python-friendly Fortunately, GoCD offers Docker images (and Dockerfiles) for both, so it's just a matter of extending it: FROM gocd/gocd-agent-ubuntu-16.04:v17.11.0
container Allow Docker Container Access Behind Proxy In the previous post, I've described how to build an image when you're behind a proxy. Now, if your container needs access to the internet for e.g. downloading python dependencies, you'll need to give it explicitly. The way to do it is via
elastic Don't Symlink Indices in ElasticSearch! The other day I've been trying to improve the downtime when updating an index in elasticsearch (ES) and I've decided the best approach would be to create a new index and do a symlink to the active one, keeping the valid index. so, my structure would
command Show What's on Swap in Linux If you find your machine running sluggish, it might be the case that you have too many things running and it's hitting the swap. You can check how bad (or good) things are with htop, but it won't tell you which process is sitting on swap.
devops Fix Jenkins POM Freezer "Wrong Relative POM" Error If you get a build error in Jenkins like: [ERROR] The build could not read 5 projects -> [Help 1] [ERROR] [ERROR] The project com.laurivan.project:project-common:1.0-34-9 (/var/lib/jenkins/jobs/project/workspace/project-common/frozen.pom.xml) has 1 error [ERROR] Non-resolvable parent POM: Could not
clean up Clean Up Your Zookeeper History TL;DR: Use zkCleanup.sh -n 5 command: By default, Zookeeper does not delete any of its history to allow recovery. This however can lead to disk space issues over long-ish periods of time. Therefore, it is advisable to clean up once in a while. Update: I found out that
columns Match String on Multiple Columns TL;DR: Use concatenation, Luke! I had to search a name for an user in a table containing the first and last names in separate columns. After a while I got tired on looking up the first_name and last_name columns one at a time and I decided to
arguments Make a Shell Script with Arguments (Native) I was building a bash script and I needed several arguments to make it more versatile (e.g. dry-run). The list of parameters was: * help as -h or --help * dry run ad -d or --dry-run to go through the motions without actually executing commands * keep last N images ad -k
global Handler for Global Logger in Python While writing a command line tool, I've stumbled upon a strange situation: Some logs were written to a file while some others were printed to console. Quite annoying, particularly since every module I use has its own logger via logging.getLogger(__name__). After some investigation I found out
alpha Some Resources for Sony Alpha Development Every time I get a gadget I think of building something for it. Since I've gotten a Sony A6000 things are not different, particularly because I have some issues with the default Sony Android app requiring GPS location (and insisting not to work without it). Therefore, I would
airflow Messy rig My current rig has: * 1 x Vega 56 blower * 1 x EVGA FTW2 1070 * 3 x RX580 Nitro + * 1 x RX580 Red Devil It's nice and I can mine either Ethereum with the hybrid Claymore miner or ZCash (albeit I still need to tune it because it crashes)
56 Vega 56 and ETH - Keep Your Cool(er) Following my writing, I've rearranged my rig to have the Vega card on one edge (where the intake fan would get "fresh" air from the room rather than heated air from one of the other cards). Results are quite epic. With the settings: * Frequency % = 0 * Voltage
django DRF: What is 'source' in Fields? For a couple of days I've been trying to de-serialise a JSON to a model. My problem is the model and the JSON have different fields, mainly to keep the model pythonic and independent of a specific JSON source. So... I've been trying to use the
datetime Timestamp to DateTime Serializer Field for DRF I'm currently trying to deserialize JSON data to a Django model. This is quite straight forward for most numeric fields, but I have a `DateTime` field which is stored as a timestamp in the original JSON.
db Redis on Windows - Save RDB but Not Able to Persist TL;DR: My windows service wasn't installed properly. Change the config file to point the DB to an accessible directory and reinstall the service. When trying to build a perioduc Celery task, I got the following error with Redis on Windows (running as a windows service): redis.exceptions.
celery Run Celery 4.1.0 on Windows 10 TL;DR: Use set FORKED_BY_MULTIPROCESSING=1 on the worker script/shell When running Celery on Windows 10, I got the following error: ValueError: not enough values to unpack (expected 3, got 0) This is quite unfortunate as Windows is supported on "best effort". A pull request
8 Use Docker Behind Proxy on Windows 8 The other day I had to install Docker at work, aka behind a corporate proxy (on Windows 8). While the docker installer is stand-alone and all is good, dealing with the VM is a different matter... The problem popped up when I tried to create a Django image and pip
