Continuous delivery workflow: Build script for PHP Composer (Drupal) on Docker I'd like to set up a continuous delivery workflow for our new Drupal site. When someone pushes to our Git repo (typically) master branch, a build job on our CI server is triggered. This job will check out the source code, import the database from production, and build the Drupal project running in a Docker container.

I'm now looking to create the actual script that the CI job will run. I come from a Jenkins background, and there one would create a Jenkinsfile for this. We're probably not going to be using Jenkins for this project, so I'm looking for a tool that can be run regardless of the build server.

The most straight forward way would probably be to script this build using bash or python or something. I'm quite new to both Drupal and PHP, so I'm wondering if there are any tools out there similar to Jenkinsfile I can make use of, which I can run on Gitlab or whatever other CI server we go for?

Each CI/CD system is different when it comes to performing the job(s) that the execution pipeline consists of. In most (if not all) cases the details are part of a configuration which is specific to the particular CI/CD system used. As you mentioned Jenkins uses a Jenkinsfile, GitLabCI uses .gitlab-ci.yml, others use "live"/GUI-based configurations, etc.

So I don't think you can find a common solution directly pluggable in an arbitrary CI/CD system. But you can get close to that: fundamentally each of these configurations are simply adaptation layers between the CI/CD system and the automated task(s) to be performed.

So if you can create a (set of) standalone script(s) to perform your automated step(s) in as portable way you need as possible you should be able to integrate with in almost any CI/CD system without too much trouble: any decent CI/CD system should support a custom script invocation mode to cover for whatever special case that doesn't properly fit into its pre-defined cases.

Using python/bash is generally quite portable, up to you to decide if that indeed covers all possible execution environments you'd be willing to consider.

I'd recommend structuring such script itself as a wrapper around smaller scripts implementing its logical/functional steps - some CI systems may require individual control of the steps or may offer advantages if such control is provided. In your case I'd have separate scripts for:

checking out the source code
importing the database from production
building the Drupal project running in a Docker container (maybe this can/should be further split if it makes sense?)

And I'd add a wrapper script tying them together for one-shot end-to-end execution, either by CI/CD systems not interested in the individual steps or even for manual invocation.


Rancher & gitlab-runner - is there a way to get the gitlab-runner to tell Rancher to start the CI test environment instead? I’ve got an AWS gitlab instance successfully doing CI with gitlab-runners running under vagrant on my local dev machine. I’d like to move the runners off my machine and onto a standalone server...

I’ve put Rancher on a box and can get the gitlab runner installed, but I can’t get my head around how to not do docker-in-docker, or whether this actually matters.

My understanding is that unless I do _some_configuration_I_don’t_know_ that Rancher runs the gitlab-runner as a Docker container, and the gitlab runner will in turn start a docker container inside itself to run my CI test environment as specified in the .gitlab-ci.yml

Is there a way to get the gitlab-runner to tell Rancher to start the CI test environment instead?

The short answer is "it depends". Rancher itself runs using system-docker and the docker instances it starts for you, such as gitlab-runner are run separately in regular docker. Gitlab-runner then will start runners depending on its configuration:

  There are three methods to enable the use of... docker run during
  jobs; each with their own tradeoffs.

https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#runner-configuration

My eventual solution

I'd initially installed Rancher 1.6 and just couldn't get the dependencies working (Rancher OS running Rancher 1.6 and Rancher agent running on a couple of Alpine nodes).

The upgrade to Rancher v2 was a steep learning curve as you need to learn the Kubernetes ecosystem tooling (helm, kubectl, etc). I still couldn't get a gitlab-runner working under Rancher v2, but upgrading to v2.0.8 fixed it. I could add the custom helm chart for gitlab-runner and after cribbing the settings from https://gitlab.com/charts/gitlab-runner/blob/master/values.yaml I could get gitlab-runner working under Rancher v2.


Does the Chef bash resource create history entries? I know that using a password in the command line is a bad idea because bash commands can be recorded in history.

If I run a command in the chef bash resource though, when I look up the history I don't see any entries.

Does this mean that the Chef bash resource does not record it's history? Is it safe to use a password in the Chef bash resource?

In this case, the best way would be to use chef data bags which is specifically known for this.
A great documentation is given at https://docs.chef.io/data_bags.html

Also, As Roland pointed above in the comment, make sure to use sensitive true in your bash resource to avoid any such information printed on screen or going to the logs.


Globally configuring Gitlab with Jenkins in docker container environment I have Jenkins users authenticated by OpenLDAP and also gitlab CE instance authenticated by the same OpenLDAP.

And all the three are running in three different docker containers in the same network.

I am trying to pre configure Jenkins and Gitlab so that they can be resolved globally in the container world.

I have Jenkins Gitlab plugin pre configured in Jenkins. To add Gitlab in the global configuration, a Gitlab API token is required to be added in Gitlab section along with the server URL for accessing all the resources of the Gitlab instance.

I am trying to do a curl operation from a Jenkins container to Gitlab using a user name and password to Gitlab requesting for Authorization code and a Private token for each a user as to be created in Gitlab. All the user authentication is happening using the OpenLDAP. For Gitlab to generate an Authorization code the user has to be created in Gitlab, which would not be happening if the users are authenticated using OpenLDAP.

My questions are:

Is it necessary to configure Jenkins with Gitlab globally to have Jenkins complete access at Gitlab resources, since we would require to authenticate once again from each job on a project level?
How to preconfigure a private token or personal access token in Gitlab from a docker container perspective to be persistent and to be shared with Jenkins when the container spins up for the first time?
When Gitlab is authenticated using OpenLDAP, without the user previously signing in into Gitlab, is a user not created in Gitlab? How to generate
an authorization code for a user in this case?

yes, it will need access to Gitlab API to report back Jenkins build status.
You can create a service account in LDAP such as Jenkins which can be used to generate Gitlab API tokens and use it to preconfigure Jenkins.
Same as point #2

For more info - https://docs.gitlab.com/ee/integration/jenkins.html


What is the difference between Jenkin's `Replay` and `Rebuild` options? If you want to re-run a Jenkins job, you can click the Rebuild button which sometimes appears as the Replay button.
What is the difference Rebuilding a job versus Replaying it?
Why is it that some jobs display Replay while others display Rebuild?

Rebuild, using the Rebuilder plugin allows a completed job to be re-executed while allowing you to change the parameters, assuming your job is parameterized.
Replay does something similar, except it doesn't show you parameters. Instead, it shows the pipeline code directly. This is helpful when wanting to test changes iteratively as you can click Replay, change one or two lines of code, then let it execute. The job will run using the same parameters as it did the first time it ran/completed.


Move Load balancer certificate to another AWS account I have two AWS accounts. Created a load balancer on "first" account, requested a certificate and using it for HTTPS.

Now I need to move this load balancer certificate to "second" account because it is requested for the same domain I need to use.

What is the procedure to do this.

Delete certificate from first aws and request again from
the second aws?
Is there any way to export the certificate?

There's no need to delete it from the first account. AWS will issue certificates as long as you can prove domain ownership. Nothing stopping you from having a cert for the same domain in multiple accounts.


Git seems to recognize http.sslcainfo property from .gitconfig but ignores it on execution? SEC_E_UNTRUSTED_ROOT all, I have a working certificates chain (testable with OpenSSL) but somehow I cannot manage to tell Git to load these certificates.

I get the same "untrusted root authority" error (SEC_E_UNTRUSTED_ROOT) independently of whether my git configuration points to an existing or a fake certificate chain file.

For details, please check the attached screenshot.

Setting I use in .gitconfig for fake file:

sslCAInfo = C:/tmp/foobar.crt

or, for real file which works with OpenSSL:

sslCAInfo = C:/tmp/ca-bundle.crt

Console transcript:

C:\tmp>openssl version
OpenSSL 0.9.8h 28 May 2008

C:\tmp>git --version
git version 2.12.2.windows.2

C:\tmp>git config --list
http.sslverify=true
http.sslcainfo=C:/tmp/ca-bundle.crt

C:\tmp>dir
24.04.2017 13:45 10.875 ca-bundle.crt

c:\tmp>openssl s_client -state -connect https://mygithost:443 -CAfile .\ca-bundle.crt

Verify return code: 0 (ok)

C:\tmp git clone https://mygithost/bitbucket/scm/my.git
Cloning into ...
fatal: unable to access ... : schannel: next InitializeSecurityContext failed: SEC_E_UNTRUSTED_ROOT (0x80090325) - The certificate chain was issued by an authority that is not trusted.

C:\tmp> git -c http.sslverify=false clone https://mygithost/bitbucket/scm/my.git
Cloning into ...
Resoliving deltas: 100%, done.

C:\tmp>git config --list
http.sslverify=true
http.sslcainfo=C:/tmp/foobar.crt

C:\tmp\xxx\git pull
fatal: unable to access ... : schannel: next InitializeSecurityContext failed: SEC_E_UNTRUSTED_ROOT (0x80090325) - The certificate chain was issued by an authority that is not trusted.

Been hit by this just today after a fresh install so here's how I got over it:

From your logs (emphasis is mine):

  fatal: unable to access ... : schannel: next InitializeSecurityContext failed:

git is configured to use schannel (windows native implementation), but schannel use windows certs bundle and not a the cainfo bundle.

To switch to openssl to use a custom bundle file use this:

git config --system http.sslbackend openssl

And now git will honor the CA bundle passed in http.sslcainfo.

Alternatively as my error was a problem of cypher suite you may be interested by the following links:

https://support.microsoft.com/en-us/help/3161639
in case you have still an old system (Win2k3) https://support.microsoft.com/en-us/help/948963/an-update-is-available-to-add-support-for-the-tls-rsa-with-aes-128-cbc


Docker Containers fails to DNS resolve on subnet 10.0.2.0/24 The ultimate problem I am facing is this:

[root@527eb70c04c9 my-service]# ping google.com
ping: google.com: Name or service not known

Inside my container, I can't resolve public domains, although the network is defined as not internal:

networks:
  my-service-network:
    driver: overlay
    internal: false
    attachable: true
    ipam:
      driver: default
      config:
        - subnet: 10.0.2.0/24

As soon as I change my subnet from 10.0.2.0 to 10.0.5.0 (really just "NOT" have it be 10.0.2.0) everything works:

networks:
  ess-team-service:
    driver: overlay
    internal: false
    attachable: true
    labels:
      - "group=ess-services"
    ipam:
      driver: default
      config:
      - subnet: 10.0.5.0/24

ping google.com
PING google.com (172.217.22.14) 56(84) bytes of data.
64 bytes from fra16s14-in-f14.1e100.net (172.217.22.14): icmp_seq=1 ttl=61 time=25.0 ms
64 bytes from fra16s14-in-f14.1e100.net (172.217.22.14): icmp_seq=2 ttl=61 time=22.7 ms
64 bytes from fra16s14-in-f14.1e100.net (172.217.22.14): icmp_seq=3 ttl=61 time=22.5 ms
^C
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 22.575/23.462/25.083/1.161 ms

A colleague referred to the problem as being caused because Docker's own nameserver is on 10.0.2.0 but I can't find information to back this theory up let alone confirm it.

The main problem is: Sometimes it works. Most times it doesn't but sometimes I can ping google inside the container while the subnet IS 10.0.2.0

We have not been able to figure out what the problem is or what a solution could be.

The workaround is actually not using the 10.0.2.0 subnet, so I will add that as an answer as requested in the comments.

If someone finds the real issue and solution here, I will gladly accept that answer though.


openshift fails to deploy jenkins When I try to deploy jenkins-ephemeral on openshift 1.5, it fails with the following error message

error: couldn't get deployment jenkins-3: Get https://172.30.0.1:443/api/v1/namespaces/projectname/replicationcontrollers/jenkins-3: dial tcp 172.30.0.1:443: getsockopt: network is unreachable

I have services enabled

oc get services
NAME           CLUSTER-IP       EXTERNAL-IP   PORT(S)     AGE
jenkins        172.30.86.254    <none>        80/TCP      59m
jenkins-jnlp   172.30.132.179   <none>        50000/TCP   59m

but from the node I can't telnet to port 80 of 172.30.86.254

telnet: connect to address 172.30.86.254: Connection refused

but the 443 port of the same machine is reachable.

This seems to be a proxy issue in your config. It is solved by setting NO_PROXY as follows:

  Edit the proxy environment variables in the OpenShift Origin control files. Ensure all of the files in the cluster are correct.

HTTP_PROXY=http://<user>:<password>@<ip_addr>:<port>/
HTTPS_PROXY=https://<user>:<password>@<ip_addr>:<port>/
NO_PROXY=master.hostname.example.com,10.1.0.0/16,172.30.0.0/16

  Supports host names and CIDRs. Must include the SDN network and service IP ranges 10.1.0.0/16,172.30.0.0/16 by default.


how to give a user privilege to run Ansible commands or just an specific tag? for example, I have a tag named 'installagent' and i want to run this command ansible -playbook -i /path  agent.yml --tags installagent from another server via ssh on my Ansible server with user 'testusr'. what should i do?

can i limit 'testusr' to only use specific tags?
( I read somewhere that i should give sudo permission to that user. if its true what should i do if i don't want to give sudo permission?)

There is no tool in Ansible to authorize a user to execute selected commands or tags only.

It seems a good idea to use sudo to authorize user to execute selective commands.


How to implement the manual step at the end of continuous delivery? The accepted answer to my question about "How does continuous integration relate to continuous delivery / deployment?" also explains the small difference between continuous delivery and continuous deployment. It appears to be related to the answer to a question like "How do you want to deploy to production, whereas these are the (exclusive) options to pick from:

Auto(matic).
Manual.

I cannot imagine that there will be a poor "operator" at the other side of the DevOps wall, who is going to have to do something that corresponds to whatever that "manual" means ... My questions:

Is my reference (in my question) to "distribute" versus "install" close to a possible implementation of such "manual"-thing? Here is a relevant quote of my related question:

  distribute to some target environment, using something like FTP (if standard copies cannot bridge the gap), but do not yet activate it in the target. That's what should be similar/close to continuous delivery, or not?
  install (or activate) in some target environment, combined with things like binds, stop/start operations, etc. That's what should be similar/close to continuous deployment, or not?

What are other possible implementations of it?

Personally I consider the distribution of the software to a target just an intermediary step of a deployment - installation/activation of that software being necessary to complete that deployment.

To me the delivery (as in continuos delivery) stops when the software to be deployed is created and made available for deployment (i.e. for distribution, installation and activation)

So, to answer your 1st question, no, I wouldn't consider distribution and installation as reflecting the manual step differentiating continous delivery from continous deployment.

Yes, in some (hopefully rare) cases that manual step is just the final human decision for deploying into production, reflecting the cultural mistrust in process automation and the mental comfort of having a human double-checking and signing off the deployment decision (thus assuming responsability for it) even if that decision is purely made based on an algorithm that can be automated (like counting pass/fail testing results).

But in general it simply reflects the fact that the decision for performing the deployment in production is not simply the result of an automated algorithm. Here would be some example such cases:

the automated decision is overwritten

deployment can be signed off even if not all quality criteria are met (we all know that this is not just a theoretical case)
the deployment is held for whatever reason, even if all criteria is met (due to market timing implications for example)

the automated algorithm is not (yet) implemented/deployed
the algorithm includes checks for some criteria depending on human decisions (say results from manual testing)
deployment is actually done in a 3rd party customer environment, following additional customer checks

So I wouldn't look at the manual step simply as an implementation issue.


Does terraform support blue green deployments and roll backs out of the box I have  been trying use terraform for a while . any suggestions/tips on how I can handle blue green deployments . I see how terraform enables it with auto scale groups on aws . But I’d like run some Cucumber acceptance tests on the new Asg before routing traffic to it . Is there a way in terraform to retain the old asg for a period , if In case I need to do a rollback . Can this be accomplished with terraform ... thanks in advance .

It all depends on how you are deploying your code.

(First, IMHO, I don't think that Terraform is the right hammer for this operation. Terraform responsibility is to build the necessary infrastructure for your application, not to be the tool used for deploying it. But, as always, "the right/best one" will depend on lots of factors and, depending on the case/scale, it may be the "good enough" solution)

My suggestion would be, in summary:

Deploy your application in the "unused" environment
Test it
Change the domain name to point to it

Assuming you have 2 AutoScaling groups already configured. One called asg-blue and the other called asg-green. And that the current version resides on asg-blue and in terraform you have an aws_route53_record pointing to it. You would need to follow these steps, each one with a terraform apply before the next:

Deploy the newer version of your application on asg-green

I.e. update everything that is needed to deploy your application version. It may be the AMI you point on your Launch Configuration or any other way that you do exclusively using Terraform.

Test that everything is running as expected

Here you could use the domain-name provided for each ASG/ELB or one already configured on terraform

Update the domain name to pont to asg-green ASG/ELB on the aws_route53_record
Reduce the number of machines needed on asg-blue to don't use too many resources

Depending on your case you can configure for 0 machines

This way you can have your B/G deployment, but for every deployment you'd need at least two manual interventions in your terraform (the deployment of the new version and updating the domain name).

But with this strategy you should take a lot of care to not deploy on the current ASG being used.

Of course if your terraform/aws fu is good, instead of reusing ASGs, you could delete and create them as needed.

Hope that this help in having a "big picture" of the entire process. But I'd suggest that you research and start using tools more focused on deployment, like AWS CodeDeploy.


Gitlab CI - Change branch dynamically I have a nodeJS application which I deploy with Gitlab CI.
When pushing to branch XYZ, the corresponding script runs, let's assume on stage 'dev'. That's working.
So far so good.
Is it possible to select a specific branch so that instead of branch XYZ, e.g. Branch FOO can be deployed on 'dev' without having to change the gitlab-ci.yml file every time?

Instead of hard-coding your branch name in .gitlab-ci.yml use $CI_COMMIT_REF_NAME. There are a bunch of pre-set environment variables you can use to get info about the current job.
The gitlab documentation has a list of them: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html


how can I import python module tkinter in gitpod I can't import pyton module tkinter in my gitpod workspace.
My .gitpod.Dockerfile contains:
FROM gitpod/workspace-full-vnc
USER root
RUN apt update && apt install -y python-tk python3-tk tk-dev

but when I run my project I get
Traceback (most recent call last):
  File "basic-screen-changer.py", line 1, in <module>
    import tkinter as tk
  File "/home/gitpod/.pyenv/versions/3.8.6/lib/python3.8/tkinter/__init__.py", line 36, in <module>
    import _tkinter # If this fails your Python may not be configured for Tk
ModuleNotFoundError: No module named '_tkinter'

python --version shows me Python 3.8.6.
Installed packaged are:
itpod /workspace/pydiag $ dpkg -l |grep ^ii |tr -s ' ' |cut -d ' ' -f2,3 |egrep '^python.*tk|^tk'
python-tk 2.7.18-1
python3-tk:amd64 3.8.5-1~20.04.1
tk 8.6.9+1
tk-dev:amd64 8.6.9+1
tk8.6 8.6.10-1
tk8.6-blt2.5 2.5.3+dfsg-4
tk8.6-dev:amd64 8.6.10-1

That problem occurs because the workspace-full-vnc image uses pyenv and the module path is different (source). To use python modules, you need to install them using python2 or python3 -m pip install ....


What is the ec2 cli command to modify the minimum number of nodes in a scaling group? I've got an ec2 autoscaling group that starts in the morning and finishes in the evening. I'd like to create a 'button' that people can click to warm up the autoscaling group to run jobs in the middle of the night (only on an on-demand basis).

My question is: What is the ec2 cli command to modify the minimum number of nodes in a scaling group?

I'd go with aws autoscaling set-desired-capacity following the documentation

But if you're up to make a button you should look into the various AWS sdk (on top right is the list of them) to call the related API instead of forking to a system command.


How to replicate a mongodb pod with its persistent storage I made these settings :

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      labels:
        name: mongo-claim0
      name: mongo-claim0
      namespace: my-app
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 100Mi
    status: {}

apiVersion: v1
kind: ReplicationController
metadata:
  labels:
    name: mongo
  name: mongo-controller
  namespace: my-app
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: mongo
    spec:
      containers:
      - image: mongo
        name: mongo
        ports:
        - name: mongo
          containerPort: 27017
        volumeMounts:
        - mountPath: /data/db
          name: mongo-claim0
      restartPolicy: Always
      volumes:
      - name: mongo-claim0
        persistentVolumeClaim:
          claimName: mongo-claim0

apiVersion: v1
kind: Service
metadata:
  name: mongo
  namespace: my-app
  labels:
    name: mongo
spec:
  ports:
  - port: 27017
    targetPort: 27017
  selector:
    name: mongo

When I try to scale this pod, Minikube UI show :

  mongo-controller-xr21r -> Waiting: CrashLoopBackOff Back-off
  restarting failed container Error syncing pod

And I got an error on the new pod :

  exception in initAndListen: 98 Unable to lock file:
  /data/db/mongod.lock Resource temporarily unavailable

Could you help me to be able to scale a pod with a persistent storage ?

https://www.mongodb.com/blog/post/running-mongodb-as-a-microservice-with-docker-and-kubernetes

  MongoDB database nodes are stateful. In the event that a container
  fails, and is rescheduled, it's undesirable for the data to be lost
  (it could be recovered from other nodes in the replica set, but that
  takes time). To solve this, features such as the Volume abstraction in
  Kubernetes can be used to map what would otherwise be an ephemeral
  MongoDB data directory in the container to a persistent location where
  the data survives container failure and rescheduling.

Like @Tensibai indicated the issue is related to the replication. According to this blog it could be solved as follows.

 apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: mongo
spec:
  serviceName: "mongo"
  replicas: 3
  template:
    metadata:
      labels:
        role: mongo
        environment: test
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: mongo
          image: mongo
          command:
            - mongod
            - "--replSet"
            - rs0
            - "--smallfiles"
            - "--noprealloc"
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: mongo-persistent-storage
              mountPath: /data/db
        - name: mongo-sidecar
          image: cvallance/mongo-k8s-sidecar
          env:
            - name: MONGO_SIDECAR_POD_LABELS
              value: "role=mongo,environment=test"
  volumeClaimTemplates:
  - metadata:
      name: mongo-persistent-storage
      annotations:
        volume.beta.kubernetes.io/storage-class: "fast"
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 100Gi


Dynamically grep with less/tail when tailing a log file, we might use:

tail -f foo.log | grep bar

if we want to change the grep argument, we have to ctrl-c and then change the argument and then restart the command.

my question is - is there a way to dynamically change what we are grepping for without having to kill the tail/grep commands?

Also looking for a way to do this with less instead of tail if possible.

I implemented this in node.js - I don't know of a unix utility that can dynamically change grep. Here is how it works:

before dygrep:

many-child-procs > logfile.log                  # in terminal 1
tail -f logfile.log | grep <expression>          # in terminal 2

after dygrep:

many-child-procs | dygrep                    # in terminal 1

then you connect to dygrep with a tcp-client and control which expressions are filtered on.

The following node.js process reads from stdin, and uses a tcp connection to listen for expressions to filter on. For each line of stdin, it only forwards that to stdout if it matches one of the current regular expressions stored in memory.

#!/usr/bin/env node
'use strict';

import * as readline from 'readline';
import * as net from 'net';
import chalk from 'chalk';
import {createParser} from "./json-parser";
import log from './logger';

let port = 4900;

const rl = readline.createInterface({
  input: process.stdin.resume()
});

const regex = new Map<string, RegExp>();

rl.on('line', l => {

  if (regex.size < 1) {
    process.stdout.write(l + '\n');
    return;
  }

  for (let [k, v] of regex) {
    if (v.test(l)) {
      process.stdout.write(chalk.magenta(' (filtered) ') + l + '\n');
      break;
    }
  }

});

interface IncomingTCPMessage {
  command: {
    add: string,
    remove: string,
    list: boolean,
    removeall: boolean
  }
}

const server = net.createServer(s => {

  const sendMessage = (m: any) => {
    s.write(JSON.stringify({message: m}) + `\n`);
  };

  s.pipe(createParser()).on('data', (d: IncomingTCPMessage) => {

    if (!d.command) {
      log.error('No "command" field was found:', d);
      return ''
    }

    const c = d.command;

    if (c.list) {
      sendMessage({regexes: Array.from(regex.keys()).map(k => ({regex: regex.get(k), str: k}))});
      log.info('Listing all regex for the client.');
      return;
    }

    if (c.removeall) {
      regex.clear();
      sendMessage(`Cleared all regex.`);
      log.info('Cleared all regex.');
      return;
    }

    if (c.add) {
      regex.set(c.add, new RegExp(c.add));
      sendMessage(`Added regex: ${c.add}.`);
      log.info('Added regex:', c.add);
      return;
    }

    if (c.remove) {
      regex.delete(c.remove);
      sendMessage(`Deleted regex: ${c.remove}.`);
      log.info('Removed regex:', c.remove);
      return;
    }

    log.error('No matching field was found:', d);
    log.info('Regex:', regex);

  });

});

server.listen(port);

to see the complete code, see:
https://github.com/ORESoftware/dygrep


SaltStack states pattern and replace for multiple files under directory and for multiple line matches I am using saltstack state pattern and replace to match a pattern and replace with a new pattern. My init.sls file executes below:

`modify_script.sh:
   file.replace:
      - name: /path/to/script.sh
      - pattern: /aaa/bbb
      - repl: /aaa/ccc
      - pattern: /xxx/yyy
      - repl: /xxx/zzz`

The above works good if I use a single pattern and replace. This fails for multiple patterns and replaces as shown above. How to achieve multiple pattern replace for two or more patterns in a single file?

Secondly, I have multiple files in a directory which requires a pattern match and replace.

modify_script.sh:
       file.replace:
          - name: /path/to/dir [having multiple files which require update]
          - pattern: /aaa/bbb
          - repl: /aaa/ccc

Can I match the patterns for all the files under directory? I wouldn't want to match and replace a single file individually.
Please advise.

You should use Jinja to loop over a list of files and pattern/replacement pairs.

{% set files = ['/a/foo.sh', '/b/bar.sh', '/b/baz.sh'] %}

{% for file in files %}
{% for p, r in [('/aaa/bbb', '/aaa/ccc'), ('/xxx/yyy', '/xxx/zzz')] %}
replace {{ p }} in {{ file }}:
   file.replace:
      - name: {{ file }}
      - pattern: {{ p }}
      - repl: {{ r }}
{% endfor %}
{% endfor %}

Keep in mind the identifiers need to be unique, that's why variables in replace {{ p }} in {{ file }}: are used.


Ansible - using facts from one host to perform actions on another host In Ansible I'm trying to generate a variable from a task on one host and then use the variable to do something to another host. Specifically, I'm trying to create a Kubernetes cluster - On the control plane node I can save the join command into a variable, but this appears to be scoped to the source host only - I can't work out how to access that variable from another host.
- name: read join command from the controlPlane
  shell: kubeadm token create --print-join-command
  when: inventory_hostname in groups['controlPlane']
  register: joinCmd

- name: join worker nodes to cluster
  command: "{{ joinCmd.stdout }}"
  when: inventory_hostname in groups['workerNodes']

The above fails on the second task (The task includes an option with an undefined variable. the error was: 'dict object' has no attribute 'stdout'). The standard approach seems to be save the line to a file locally, copy the file to the worker nodes and execute it locally which seems cumbersome (and leaves the join token lying around). How can I achieve this with variables only?
thanks,
Jim

I found a solution - the trick here is delegate_to and delegate_facts:
- hosts: all
  remote_user: ansible
  become: false

  tasks:
  - name: read join line from controlPlane
    shell: kubeadm token create --print-join-command
    when: inventory_hostname in groups['workerNodes']
    register: joinCmd
    delegate_to: "{{ item }}"
    delegate_facts: true
    with_items: "{{ groups['controlPlane'] }}"

  - name: join all worker nodes to cluster
    command: "{{ joinCmd.results[0].stdout }}"
    become: yes
    when: inventory_hostname in groups['workerNodes']

The first task kind of reads backwards to my mind - when says 'run this task on all the  workerNode hosts...' but delegate_to says '..but actually run it on a different host'. It works for me though (I've only got 1 host in controlPlane group, not sure of outcome with more hosts).


Docker stack deploy doesn't pull images I'm launching through Ansible (just SSH basically) a docker stack deploy, the issue is also true when launching directly from a manager:

docker stack deploy --prune --with-registry-auth -c /docker/docker-compose.yml "test-stack"

I have my credentials to access this registry are defined in .docker/config.json.

The problem is first seen here with a docker stack ps test-stack:

 ... Shutdown            Rejected 11 minutes ago      "No such image: myregistry.com:12345/myimage:mytag"

The workers aren't uploading the images I want. On their logs, I get the following:

Jan 31 11:22:24 myhost dockerd[1149]: time="2019-01-31T11:22:24.574302458+01:00" level=warning msg="Error getting v2 registry: Get https://myregistry.com:12345/v2/: http: server gave HTTP response to HTTPS client"
Jan 31 11:22:24 myhost dockerd[1149]: time="2019-01-31T11:22:24.574342640+01:00" level=info msg="Attempting next endpoint for pull after error: Get https://myregistry.com:12345/v2/: http: server gave HTTP response to HTTPS client"
Jan 31 11:22:24 myhost dockerd[1149]: time="2019-01-31T11:22:24.618063948+01:00" level=info msg="Attempting next endpoint for pull after error: unauthorized: authentication required"
Jan 31 11:22:24 myhost dockerd[1149]: time="2019-01-31T11:22:24.618121224+01:00" level=error msg="pulling image failed" error="unauthorized: authentication required" module=node/agent/taskmanager node.id=rarbt9afb16y7j77bfwiacby2 service.id=ej4y7wbwp539u7mmmhfashz93 task.id=c1uy2gnwdcmibp6l2p9rfxcsh
Jan 31 11:22:24 myhost dockerd[1149]: time="2019-01-31T11:22:24.618840324+01:00" level=error msg="fatal task error" error="No such image: myregistry.com:12345/myimage:mytag" module=node/agent/taskmanager node.id=rarbt9afb16y7j77bfwiacby2 service.id=ej4y7wbwp539u7mmmhfashz93 task.id=c1uy2gnwdcmibp6l2p9rfxcsh
Jan 31 11:22:29 myhost dockerd[1149]: time="2019-01-31T11:22:29.548642584+01:00" level=warning msg="failed to deactivate service binding for container test-stack_myservice.1.c1uy2gnwdcmibp6l2p9rfxcsh" error="No such container: test-stack_myservice.1.c1uy2gnwdcmibp6l2p9rfxcsh" module=node/agent node.id=rarbt9afb16y7j77bfwiacby2

The problem is that, when I do a docker pull myregistry.com:12345/myimage:mytag on the workers, it pulls the image successfully and without any problem, but not through an docker stack deploy command. Despite the fact that i have the option --with-registry-auth.

So it turned out that some of the workers didn't have the correct credentials to the docker registry. After I applied the correct ones, the command worked normally.

That's weird, since I could do it locally...
Anyway, case solved.


How to include a custom Slack message only on completion? We're running Jenkins with the Slack Notification Plugin. Is there a way to send the custom message only on completion, not on start? To be clear, I don't want to disable the build started message, I'd just like the custom text to only show on completion (success).

Use case is jobs like delete stack. We use the custom message to make it clear which stack the job deleted, but you end up with slack messages like (paraphrased)

Delete job started
  Deleted Alex stack

Delete job finished
  Deleted Alex stack

Obviously you can see where this causes confusion. I'd rather get

Delete job started

Delete job finished
  Deleted Alex stack

"It can't be done" is an acceptable answer if that's the case. I know I can do a pipelines workaround but I'd love a quick fix for the jobs we haven't converted yet.

As far as I was able to tell it's not possible.

We solved this by using Pipeline putting the custom messages in the Jenkinsfile.

The flexibility gained by having the Jenkinsfile in the git repo really helped our developers have a better understanding of the build process, and allowed them to make necessary changes to the build commands without the communications headache.

It's a nice excuse to move over to Pipeline.

It also happens to be very easy to do:

slackSend (color: colorCode, message: customMessage)


Should AWS arn values be treated as secrets? I have a configuration file that contains AWS resource arn's stored in a Github repo. This includes some arn's of AWS IAM role resources. Should these be treated as if they are a secret value, and not stored in a code repository?

Well, from a security point of view, it's never bad to give people less information.

But, as long as you have sensible security policies in AWS, there is no reason that an arn has to be considered secret.

For example, if you know the arn of the role I use to access s3 from my EC2 instances, you can't do anything with that information unless I give you permission to assume that role from one of your roles.  Between accounts, this is a very explicit and bidirectional action; so it will not happen by accident/without you knowing.

Losing the credentials to an IAM user could let people assume roles and would be terrible, but them knowing the arn alone does nothing for them.


How can I assign an IP address to a docker container different from the host's IP? Objective: Assign fixed IP address to Docker container (Unifi Controller instance).

History: I fetched the unifi controller image from Docker Hub to my Synology Docker host. The container of the unifi controller runs properly if I attach it to the network of the host (not the default bridge). That blocks though multiple ports I need for other services (in the future). Thus it would be much easier to assign that unifi container with it's own designated IP --> 192.168.2.2 (than changing all the ports).

I guess that is a cinch to deal with for most in here.

Indications of my misconception:

I created my own network in the Synology Docker GUI (with the subnet 192.168.2.0/23 and gateway 192.168.2.1)
I assigned it to a new unifi-controller container instance.
The GUI suggests to map the host ports with "auto" to the container ports, which indicates me that I do NOT understand completly the network concept of Docker. Because if I'm able to set an individual IP, why should it allow to map host ports (to container ports)?

Questions:

Why can I not assign a fixed IP to the container in the GUI?
What am I conceptually missing?
If not in the GUI, how can I accomplish it over SSH/Bash?

SOLUTION APPROACH

The approach is to create a macvlan. This will create a virtual adapter, that is allowed to lease an IP address from the subnet defined.

1) Activate Promiscous Mode

For the virtual adapter one must enable promiscuous mode in the network. For Unifi controllers, do a SSH to your gateway and set:

ifconfig [interface] promisc

Some network gateways/controllers have set promiscuous mode by default. Continue with step 2 before you're wasting time to find that out.

2) Create macvlan

Then one can create the macvlan.

$ docker network create -d macvlan \
  --subnet=192.168.2.0/23 \
  --ip-range=192.168.2.5/25 \
  --gateway=192.168.2.1 \
  --aux-address="my-router=192.168.2.10" \
  -o parent=eth0 unifinet

Limiting the IP range: use --ip-range to scope the possible IPs to lease.
Avoiding certain IPs: The --aux-address marks my Synology host, which should never ever battle for that IP address (It's marked in the unifi controller's DHCP service as a fixed IP anyways).
the -o parent is your network interface you want to attach your macvlan. In my case unifinet.

Check with docker network ls if the macvlan has been properly created.

NETWORK ID          NAME                DRIVER              SCOPE
c49094a4c914        bridge              bridge              local
b2315de1aa7e        host                host                local
c124eda0f9d2        none                null                local
a60da50f0d12        unifinet            macvlan             local

3) Start Docker Container

While you define the image (--name UnifiController jacobalberty/unifi) assign the freshly created macvlan (--network unifinet) to the container (UnifiController). It will grab the latest image respectively the latest Unifi Controller version available from Docker Hub.

docker run -dit --network unifinet --name UnifiController jacobalberty/unifi

Verify if the container has been assigned to the network in the container section of the JSON config.

sudo docker network inspect unifinet.

[
    {
        "Name": "unifinet",
        "Id": "a60da50f0d1229d1a3c76210141e0c81567c17daf6c2b49d4f1c83d5ec9f02b3",
        "Created": "2019-10-04T14:39:37.377311991+02:00",
        "Scope": "local",
        "Driver": "macvlan",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "192.168.2.0/24",
                    "Gateway": "192.168.2.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "b819bf1d6f3af459825a1a7b58f9a44e15e6e09489e7bf50653ed8e1e176fd73": {
                "Name": "UnifiController",
                "EndpointID": "41a048034ad63e48b46b58aed65661c9eaa2bf6937d3eebacefde4478ad26cce",
                "MacAddress": "02:42:c0:a8:02:02",
                "IPv4Address": "192.168.2.2/24",
                "IPv6Address": ""
            }
        },
        "Options": {
            "parent": "eth0"
        },
        "Labels": {}
    }
]

The IP 192.168.2.2 is assigned. The container workes properly and stable.

4) Optional/Reminder

If the IP was not assigned respectively one cannot connect to it for any service, the follow up on the promiscious mode setting as described in Step 1.


Jenkins deployment key is not associated with the repository Using jenkins to pull from bitbucket repository, I got this error:

git@bitbucket.org:source/source.git +refs/heads/:refs/remotes/origin/" returned status code 128:
stdout:
stderr: repository access denied. deployment key is not associated with the requested repository.
fatal: Could not read from remote repository.

I have added ssh keys in the bitbucket repo, I have run ssh -v git@bitbucket.org and this is the output:

authenticated via a deploy key.
You can use git or hg to connect to Bitbucket. Shell access is disabled.
This deploy key has read access to the following repositories:
source/source: root.staging_quc -- root@QUICKRESPONSE
debug1: channel 0: free: client-session, nchannels 1
Connection to bitbucket.org closed.
Transferred: sent 2968, received 1872 bytes, in 0.0 seconds
Bytes per second: sent 82471.7, received 52017.2
debug1: Exit status 0

I don't really know what to do, hope you guys have any clue on this one I will update this question

First make sure that you added the public key to the right repo with the right permission, usually I add it to 'Access keys' so the user has read only permission, then to make sure that is working add the private key to you system so you can use it with using git:

ssh-add private-key
git clone <your git repo>

If it's working, add those credential to you jenkins and use them later


Trigger a job and get the job log on the same page in Jenkins I would like to trigger the job in Jenkins and also get the log on the same page.

E.g. Kind of IDE where we can run the code in one side and get the output on the other side.

Are there any tools exists to fulfill this, or do we need to create it?

It looks like there exists a plugin to do exactly what you want: Display console output plugin.

However, I haven't tried this plugin myself, and it looks like this plugin is not well-maintained, so I cannot vouch for the quality of this plugin.


Does a company need DevOps if it already delivers software at acceptable speed? I mean, DevOps is, from the business view, about speed and quality, right?

What does it mean if a company is growing and fast enough for their customers without devops? Does one of the following apply?

They did, or do software development at sufficient level, no change is required.
They do DevOps but don't know it/gave it another name.
They do something totally different and better than DevOps as we know it and we really should find out what it is (before it is too late).
They are awfully wrong: their speed and quality are below their knowledge and they will know it as of next financial report (in case they manage to survive long enough to see it).

DevOps isn't just a different way of doing things, it is a better way of doing things. Perhaps it will help me to make my point if I rephrase the question in a different context - the context of the industrial age, the railroad and the steam hammer:

Does a railroad company need Steam Hammers if it already lays track at an acceptable speed?

I mean, steam hammers from the business view are about speed and quality, right?
What does it mean for a construction company contracting for the railroad that is laying track "fast enough" for their customers without steam hammers? Which of these might apply?

They lay track at a sufficient level, no change is required

They use steam hammers but don't know it or call the steam hammer something else

They use something totally different and better than a steam hammer and we really should find out what it is (before it is too late)

They are awfully wrong: their speed and quality are below their knowledge and they will know it when they begin to lose bids to other contractors.

The answer is obviously probably #4. This hypothetical company may not see the problem with the next financial report - perhaps it will take a few years - but eventually it will become clear how and why competitors outperform businesses still manually "laying track" as it were. Sure, it's possible they have come up with a diesel powered hammer that is more efficient and less prone to malfunction or breakdowns, but not likely.
Thinking that they can outperform the steam hammer is going to get this company left behind. Sure, an individual group may have their steam hammer blow up and fall apart and perhaps that one group has such trouble with their steam hammer that it would have been more efficient in that one case to do things the old fashioned way, but they are the exception to the rule and the problem likely was that their steam hammer was poorly built and/or poorly maintained.
The industrial age is coming and the railroad company can either check out the competition and then try to get their own steam hammers, or get left behind and eventually succumb to their irrelevance and antiquity.
DevOps is no different. It's just another iteration of automation that companies can either embrace or ignore at their own peril much like the steam hammer.


Error with docker-compose - image not supported? What's wrong with the following spec? (Image parameter is here to tag resulting image).

version: '3'

services:
   foo:
      build:
         image: foo/bar
         dockerfile: Dockerfile
         context: .

System output:

services.foo.build contains unsupported option: 'image'

Versions: docker-compose: 1.14.0; Docker: 17.06.0-ce

“Image” needs to be at the same level as “build”.


How to test whether a variable exists in the declarative pipeline I am using the Build User Vars Plugin which sets various variables in the pipeline. Here is what I am currently trying:

wrap([$class: 'BuildUser']) {
    def safeBuildUser = binding.hasVariable('BUILD_USER') ? BUILD_USER : "unknown"
    echo "${safeBuildUser}"
}

There are times when the plugin does not set the variable. For instance, when the build was launched by an SCM branch scan. When that happens, I receive:

groovy.lang.MissingPropertyException: No such property: BUILD_USER for class: groovy.lang.Binding

Using binding.hasVariable does not appear to work because all of the builds now show "unknown".

In addition to the above, I also tried the following with no luck:

def safeBuildUser = BUILD_USER?: "unknown"
def safeBuildUser = binding['BUILD_USER']?: "unknown"
def safeBuildUser = binding.variables['BUILD_USER']?: "unknown"

use try/catch

def safeBuildUser = "unknown"
wrap([$class: 'BuildUser']) {
  try {
     safeBuildUser = BUILD_USER
  } catch (e) {
     echo "User not in scope, probably triggered from another job"
  }
}
echo "Builduser is: ${safeBuildUser}"


Having some trouble getting service running with Docker and Terraform I'm very new to DevOps but I'm trying to learn some of it a bit.  I'm working with Terraform, trying to get a service to run on AWS using Fargate and Docker.  I've successfully set up my ECR repositories, and I walked through this tutorial on getting a service set up in Fargate.  However, I'm having a bit of a problem.  When I walked through the tutorial, everything worked more or less fine (there were some small issues but it was mostly pilot error), but when I tried to deploy my own application everything kind of exploded, and I can't figure out why.

The error I'm getting is 502 Bad Gateway.
The application is a Java Spring application with embedded Tomcat.  I haven't changed any run configurations from the default Spring config.
I am able to build and Dockerize my application locally and run it through Docker on my local machine and everything works fine.
The only changes from the tutorial code (linked above) that I made was changing the Docker image and changing the app_port from 3000 to 8080 (I also enabled Access Logging and added an S3 bucket for that purpose)
The application has some auth security enabled, and when I try to hit the LB I get a login screen, so I know at least my Docker image is being pulled correctly
One of the errors in the task logs of my ECS instances says something about failing a health check, although I'm not able to figure out anything more than that.

To be honest I'm a total newbie at DevOps anything so I have no idea what I'm doing.  Can anyone point me in some direction to help me get started?

Seems like, because the application has security enabled, the health check was not able to hit the / endpoint and the health check was failing.  When I removed the health check block I was able to access the application.  I'll add a globally-open health check endpoint and that should fix the problem I think.


Networking between Docker containers I have a docker container 'A' on a host 10.11.12.13 listening on port 0.0.0.0:8443->8443/tcp I have another container 'B' on same host how wants to talk to 'A' on 8443 via host. Thus when I run command curl https://10.11.12.13:8443 inside B it gives me error as 'No route to host' but I can ping 10.11.12.13 successfully. I am not sure what else I am missing?

Can someone help regarding the issue?

Docker actually provides DNS support for networking between containers. This means that if you define your database connection as:

http://redis:6379

This will resolve redis to the correct network address, given that you fulfill the following:

All containers are on the same network
The containers have been named

This can either be done manually by starting both instances via the cli or automated with a docker compose script, which is my recommendation. In the compose script, from version 3, you would define the both services (aka the containers):

version: "3"
services:
  web:
    build: ./web
    container_name: web
    networks:
      - backend
    command: bash -c "./start.sh"

  redis:
    image: redis:3.2
    container_name: redis
    networks:
      - backend

And at the end of the file, define the network both instances should be connected to:

networks:
  backend:


Automated docker builds with private bitbucket server I was looking to setup automated docker builds when a push is made to our bitbucket server. The only issue is that our Bitbucket is not cloud based, its an on premise private server. Is there a way to configure automated builds for the docker hub when a push is made to our private server? Obviously Docker can't see our private server, but is there a way to accomplish the same thing using web hooks in the private bitbucket repo?

I would use a Packer template, specifically with a Docker builder to do this. Triggering this build can be done with hooks (either BitBucket webhooks, or git server post-receive hooks.

Building a Docker image from a Packer template has some benefits, but requires Docker Engine installed - so, doing the build with a Dockerfile would also be an option.

The tricky part would be the management of credentials. Building the image does not require credentials, nor does tagging -- however you need to be authenticated to push the image to the registry. If the build happens on the same server that is hosting BitBucket, you could keep them in a protected file and execute the build via the post-receive hook in a shell script. This would be a lightweight option, especially if the server itself is locked down.

So, in summary :

Install Docker Engine on the BitBucket Server
Write a script which can consume registry username and password, to build the image, either from a Dockerfile or from a Packer template.
Add a server-side git hook to execute this script to build, tag and push the image.


passing args to shell module in Ansible adhoc command Is it possible to pass args, to the shell module when running it as an adhoc command?

I can pass the free_form parameter using -a, but I am trying to figure out how to specify additional arguments, such as chdir or executable.

For example, what is the adhoc equivalent to this task?

- name: Change working dir to somedir/ and run somescript.sh using bash
  shell: somescript.sh >> somelog.txt
  args:
    chdir: somedir/
    executable: /bin/bash

Append the parameters after the command. See the example below.

# ansible -m shell --args "pwd; touch test; ls -1 chdir=/scratch/tmp creates=/scratch/tmp/test" localhost
localhost | CHANGED | rc=0 >>
/scratch/tmp
test

# ansible -m shell --args "pwd; touch test; ls -1 chdir=/scratch/tmp creates=/scratch/tmp/test" localhost
localhost | SUCCESS | rc=0 >>
skipped, since /scratch/tmp/test exists


Deploy Docker Compose from different repositories I have 3 separate repositories, which contains API, web app, and Admin projects. All 3 runs on a Digital Ocean docker machine, and I deploy the 3 using docker-compose (I want to keep things simple).
Every time a push to master with a tag happen, GithubActions build each separate project and publish a docker image (with the corresponding tag), and then I manually launch docker compose up -d. The docker compose file point to the latest version of each image, so it get automatically reloaded and launched.
Is there a better way to automatically reload the docker compose from GitHub Actions? I ideally want that each time a repository build an image and push it to the registry, the docker compose automatically reload.

You can utilize watchtower to restart the specified containers each time new image of that container is avaialble.

Watchtower : with watchtower you can update the running version
of your containerized app simply by pushing a new image to the Docker
Hub or your own image registry. Watchtower will pull down your new
image, gracefully shut down your existing container and restart it
with the same options that were used when it was deployed initially.
Run the watchtower container with the following command:
$ docker run -d \
    --name watchtower \
    -v /var/run/docker.sock:/var/run/docker.sock \
    containrrr/watchtower

docker-compose integration example:
services:
  watchtower:
    command: --label-enable --cleanup --interval 300
    image: containrrr/watchtower
    labels:
      - "com.centurylinklabs.watchtower.enable=true"
    network_mode: none
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

  mariadb:
      depends_on:
        - watchtower
      env_file: .mariadb.env
      image: mariadb:10
      labels:
        - "com.centurylinklabs.watchtower.enable=true"
      networks:
        - default
      ports:
        - 3306:3306
      restart: always
      volumes:
        - ./data/volumes/mariadb:/var/lib/mysql

To check the logs of the updation you can execute docker-compose logs watchtower
docker-compose integration reference: https://chriswiegman.com/2019/12/keeping-docker-containers-updated/


Deploy Vm Vmware with simple Ansible playbook I try to test one simple playbook with ansible.
My goal is to create a new VM with it.

---
# create a new VM from a template

- name: VM from template
  hosts: localhost
  gather_facts: false
  connection: local
  vars:
    vcenter_hostname: vcenter-app
    vcenter_user: john@doe
    vcenter_pass: blabla6
    esxhost: esx-4.cbalo.fr
    datastore: VM-PROD-02-NORMAL
    vmtemplate: Centos7-template
    name: "newvm2"
    notes: Ansible Test
    dumpfacts: False
  tasks:
    - name: Create VM from template
      vmware_guest:
        validate_certs: False
        hostname: "{{ vcenter_hostname }}"
        username: "{{ vcenter_user }}"
        password: "{{ vcenter_pass }}"
        esxi_hostname: "{{ esxhost }}"
        datacenter: CD06
        folder: Test
        name: "{{ name }}"
        template: "{{ vmtemplate }}"
        disk:
          - size_gb: "{{ disk_size | default(17) }}"
            type: thin
            datastore: "{{ datastore }}"
        hardware:
          memory_mb: "{{ vm_memory | default(1024) }}"
        wait_for_ip_address: True
        state: present
      register: newvm2

The connection to my VMcenter is fine but i see that error message :

**ansible-playbook test_vmware.yml**

PLAY [Create VM from template] ******************************************************************************************************************************************************

TASK [Clone VM from template] *******************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed to create a virtual machine : Cannot complete the operation because the file or folder /vmfs/volumes/5d25e76b-5475ac54-9cdb-e4434b69d662/newvm2/newvm2.vmdk already exists"}

Even if i change the name newvm2 for an other one, i still have the same message. Someone can maybe help me with that case ?

Many Thanks

https://github.com/ansible/ansible/issues/55551

The fix is included in Ansible 2.8.3 or higher. I had the 2.8.2 version


Strategy for deploying/hosting javascript-based static websites in containers This comes up from time to time in several of our dev teams, without us having figured out the "right" way:

We use a lot of react-based webapplications that "compile" into static websites that are just a few html, js and css files.

However, the "building" of these apps take a number of variables that enable/disable feature flags, configure backend urls, etc.
This means that we cannot "build" a binary in the traditional sense and just apply a config file at deploy-time - the "build" itself needs to have these environment-specific variables set, and thus the only time we can "build" is at deploy-time.

For now we solve this by injecting the required environment variables into the Docker container and run a start cmd along the lines of

npm build && nginx run

This has a couple of disadvantages:

The build process takes a lot of cpu/memory relative to the run-time requirements of the container. That means we need to scale the container for the build process instead of the run-time requirements - which feels wrong
Build failures are hard to "track". We can use healthchecks in Kubernetes, but if a build takes 2 minutes we still have to wait for 3 mins (1 extra for safety) before we can start testing the container's healthcheck endpoint to see if its alive.
Deployments might take a long time: If we configure Kubernetes to do a "serial" deployment, it will start each pod and wait for the "initialDelay" period of 2-3 minutes before starting the next. This means that we're easily looking at a 10-min deploy time if the deployment is scaled to 3-4 pods.

This all feels very suboptimal to me. I'd be very interested in hearing how the community solves the "build at deploy-time" conundrum with modern javascript webapps.

I realize that for Kubernetes we could use "init-containers" that perform the build, places the artiacts in persistent storage and then have the app containers simply pull from persistent storage during startup, but this still feels more like "bypassing" the problem than solving the root issue.

From my standpoint the best approach would be to:

Separate build process using Jenkins that would build NodeJS project into distribution and wrap it into Docker image
Spin up Docker registry that would accumulate Docker images from Jenkins (this registry should be accessible from Kubernetes cluster)
Move environment variables to Jenkins secrets or use separate tool to collect and combine configurations from external Git repo (we use Spring Cloud Config through REST API to collect json/yml definitions for every application in every environment)

Using Jenkins you can configure continuous delivery system based on generic Pipelines. Possible flow would be:

Developers finish their work in correspondence to GitFlow (latest Pull Request merged into Release branch)
Webhook triggers Jenkins pipeline:

Stage 1: collect environment definition
Stage 2: build NodeJS app using npm
Stage 3: build Nginx Docker image with distribution
Stage 4: push Docker image to Docker registry
Stage 5: deploy/update service in Kubernetes using standard definitions

Notifications send in regard of results

This process might be visualized using Rancher. I can answer your questions in chat.


How CircleCI, Terraform and AWS should interact in CI/CD Pipeline I'm new to devops and I'm have a hard time finding resources to help me with this question. My Goal is to have an automated CI/CD pipeline to launch my project.
Basically I have a GitHub repo with a Python Web application. I've also configured a pipeline in CircleCI that runs some tests in the project. That's what I've gotten to work so far.
Now, from what I've understood, this is what I would do next (This is what I'm unsure of so please correct me):

In circleCI, generate a docker image with instructions on what kind of operating system etc and commands to pull project from GitHub repo and run database migrations etc.
CircleCI runs terraform to launch this docker file in an EC2 Instance.

So my question is basically this:

Should the docker image be generated dynamically in CircleCI or static in the repo?
How does my source code get to the EC2 instance. Configure a git pull in dockerfile?
How do I tell CircleCI/Terraform how many instances I want to launch?

I know this is a question that would take too long to explain in full here. So I'm only looking to understand the basic workflow of how GitHub->CircleCI->Terraform->AWS interactive with eachother. Or any resources that explain this well.

There can be many correct answers, depending on the strategy you are doing. My take would be:

I wouldn't create a dedicated Docker image (e.g. index.docker.com/python:3.9-slim) with the application on it, but rather use the official Python Docker image, to build the app and the same to run it, so I'd package your app separately from the image that runs it. You can still compile your code as a Docker image if that's convenient and mount it as a volume. There are many benefits of this:

your code can be mounted as read-only, lowering the attack-surface
faster build (no need to add a full Python every time), faster deployment (yes, Docker layers would still work, but if your code changes, Python has updates, you might have multiple hosts eventually don't count on layering).
separation of concerns: just because you update your Python interpreter's patch version (e.g. for security fixes), your app don't have to change. This can be an important factor if you need to have a decision log for your releases.

You can use, e.g. CoreOS and use a Systemd unit file to pull down and run it for you, or you can instruct your Docker daemon via the Terraform provider to run it and set the "restart = always" so it will be started on reboots too. I'd go with the latter as it requires no knowledge on the OS and there is a single way (Terraform) to manage the whole deployment.

Well, if you have multiple instances, high-availability, then you either

need to do this in your app with a service-discovery tool (check
e.g. Hashicorp Consul) or you can have a load-balancer, but that
costs you money. Using load-balancers is the lazy way; using service
discovery and doing the load-balancing on the client-side (app)
offers you the best outcome, lesser architectural complexity, but
much more work.
Use an orchestrator, e.g. Kubernetes, but then your whole manual deployment with Docker and Terraform is not relevant anymore because you are dealing with the deployment at a higher level. Although Kubernetes has a steep learning curve initially, it is very straightforward when it is up and running.
A runner-up solution could be a DNS-based solution. In this case, you manually set, e.g. 3 instances and then in your DNS zone, you set a round-robin.
You can't use, e.g. AWS Autoscaling Groups, because you want to manage your containers via Terraform, so if the ASG scales up an instance, somehow your Terraform needs to run. So unless you start up your containers via "user-data", ASGs doesn't make sense, but in that case, you can't manage them via Terraform because Terraform is likely aware of the container's state. Even if you manage it, it will be a fragile architecture that will bite you back when you need that high availability.

Note: You probably need at least 2 Terraform run, because the Docker Terraform provider requires access to the target Docker service via TLS/SSH or Unix socket, so your instances have to be up and ready before you run the Docker one. But that's a good idea anyway because if you add all these in your user-data/cloud-init/Ignition script, Terraform will destroy the instances, so your container updates won't be that quick.
Note 2: Do yourself a favour and if you need more than 1 instances, use an orchestrator like Kubernetes on Hashicorp Nomad.


Should I use filebeat or an app-logging module with lumberjack capabilities to collect my logs I have a pretty basic elk setup which aggregates logs from around 15 VMs that are running various services.

All of them have configured filebeat syslog forwarder with a single input in logstash.

I have app servers where I run NodeJS web apps, which output JSON logs. In order to make my logs structured I have two options :

Option A

Tell the NodeJS apps to output the logs in a /var/app/app.log rotatable file. Add the app.log to my log propspect in filebeat and push to logstash, where I setup a filter on [source] =~ app.log to parse JSON.

Option B

Tell the NodeJS app to use a module ( e.g. node-bunyan-lumberjack ) which connects independently to logstash and pushes the logs there, without using filebeat.

My question is :

Which one is a more common setup and why?

Both are common and comes with their own pro and cons.

It depends if you can accept loosing logs if logstash is dead and your app restart and on the other hand if you can write logs to disk and consume place there.

Those are the basic points to choose for one or the other, you may end up with more caveats in term of IO/caching/etc.

There's no 'best' option, there's the best option for your organization/teams which we can't answer for you as we don't know your environment.


Ansible execute task on different hosts in order Is it possible to use ansible to manage order of programs?
Im trying to find an automated solution which will do the following steps:

run a program on HostA
run a command on HostB which requires HostA program to run
run a program on HostC if 2. step succeed
run a command on HostD which requires HostC program to run
run a command on HostE which requires HostC program to run

For example:

---
  - hosts: neededhosts
    roles:
      - startprogram
      - runcommand
      - startsecprgoram
    .
    .
    .

startprogram:
- name: start hostA program
  tasks:
    - name: start program
      hosts: HostA
      shell: /etc/init.d/HostAprogram.sh start

runcommand:
- name: run command on HostB
  tasks:
    - name: run command
      hosts: HostB
      shell: command which using host A

Q: "Is it possible to use Ansible to manage the order of programs?"
A: Yes. It's possible. For example, the playbook below does the job
- hosts: HostA
  tasks:
    - include_role:
        name: startprogram

- hosts: HostB
  tasks:
    - include_role:
        name: runcommand
      when: hostvars.HostA.HostAprogram_started|default(false)

- hosts: HostC
  tasks:
    - include_role:
        name: startsecprogram
      when: hostvars.HostB.HostBcommand_passed|default(false)

- hosts: HostD,HostE
  tasks:
    - debug:
        msg: command on {{ inventory_hostname }} which requires HostC program to run
      when: hostvars.HostC.HostCprogram_started|default(false)

Example of the roles for the purpose of testing
shell> cat roles/startprogram/tasks/main.yml
- debug:
    msg: /etc/init.d/HostAprogram.sh start
- set_fact:
    HostAprogram_started: true

shell> cat roles/runcommand/tasks/main.yml
- debug:
    msg: command which using host A
- set_fact:
    HostBcommand_passed: true

shell> cat roles/startsecprogram/tasks/main.yml
- debug:
    msg: /etc/init.d/HostCprogram.sh start
- set_fact:
    HostCprogram_started: true

Example of the output
ok: [HostA] => {
    "msg": "/etc/init.d/HostAprogram.sh start"
}
ok: [HostB] => {
    "msg": "command which using host A"
}
ok: [HostC] => {
    "msg": "/etc/init.d/HostCprogram.sh start"
}
ok: [HostD] => {
    "msg": "command on HostD which requires HostC program to run"
}
ok: [HostE] => {
    "msg": "command on HostE which requires HostC program to run"
}


how to control pod termination speed in kubernetes I have a k8s deployment that I update by doing kubectl apply -f spec.yml. I use the default update strategy.

Whenever I update, new pods get created and the old ones eventually get deleted.

This is what kubectl get pods looks like a few seconds after calling apply:

NAME                      READY     STATUS        RESTARTS   AGE
server-243710380-7s7g0    1/1       Terminating   0          9m
server-243710380-ffzv1    1/1       Terminating   0          9m
server-243710380-gfs3h    1/1       Terminating   0          9m
server-3561703442-24dxm   1/1       Running       0          13s
server-3561703442-7jlds   1/1       Running       0          8s
server-3561703442-x5xw1   1/1       Running       0          11s

As you can see, the ones in Terminating status hang around, typically for about 30 seconds, before really disappearing. This happens despite the new pods reaching running status within a few seconds.

How can I control the time the old pods remain in terminating status? Specifically, I want them to terminate as quickly as possible.

There is a default grace period for pod termination of 30 seconds. See: https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods.

You can customize the grace period setting terminationGracePeriodSeconds at the pod spec level.


Git Remote Access in Azure Pipelines, possible? I'm trying to achieve the following through the build pipeline.

Default branch: SIT (from the Azure Repos repository)
Trigger: whenever a change is made in DEV branch

In a powershell task do the following:

Merge latest from DEV to the current SIT branch
Deploy the latest from SIT branch
Push the updated SIT branch to the upstream repository

Here is my code:

git checkout -b temp
git merge origin/DEV
git branch -f SIT temp
git checkout SIT
git status
git branch -d temp
git remote add neworigin $(RemoteOrigin)
git push -u origin --all

When I try to push the updated SIT branch using git push -u origin SIT and I'm getting the following error:

  remote: 0000000000aaTF401027: You need the Git 'GenericContribute'
  permission to perform this action. Details: identity
  'Build\d073f6c7-4fa2-4bda-8694-8a41fb808fff', scope 'repository'.
  remote: TF401027: You need the Git 'GenericContribute' permission to
  perform this action. Details: identity
  'Build\d073f6c7-4fa2-4bda-8694-8a41fb808fff', scope 'repository'.
  fatal: unable to access '': The requested URL returned
  error: 403

Can someone help?

I've had a similar issue with git commands in Azure Pipelines.  The reason is that any commands that access the origin server, such as git push need to done by a user that has the credentials to do so.  In your situation, your PowerShell script is running in the user context of Project Collection Build Service (yourProjectName) and it does not have the permissions to contribute to your repo by default.

See this answer, I think it will fix you up: https://stackoverflow.com/a/56542631/11767897


How to change a statefulset in OpenShift? What's the best way to change a statefulset (e.g., modify the ENV parameters, modify PVCs) in OpenShift? oc edit statefulset xyz only allows to change the scaling, nothing else.

Shameless self-answer:

So it turns out the only way to edit the statefulset, and this is by design, is to delete it and recreate it with the new values.

Doing that does not delete or stop the pods or the the PVCs - those will re-attach to the new statefulset spun up later, so you won't lose anything. N.B.: you can delete the pods, and the PVCs will still stick around. When the pods belonging to this statefulset restart later, they will re-attach to the PVCs.

There is a discussion on one of the OpenShift/Kubernetes task trackers (which I cannot find anymore) that explains the rationale: in contrast with stateless Deployments, where OpenShift can simply scale up a new set of pods, then scale down the old one, while routing traffic to the new one, this is not easily possible with statefulsets, since those have persistent PVCs, so there usually is a very defined number of pods that can/should run at a time.

So, if OpenShift would do an upgrade for you, due to an edit, it would need to decide how to do that (spawn up new pods parallel to the old one would require spinning up PVCs which may wreak havoc with your quota; replacing them in-place could wreak havoc if your pods need to talk to each other to synchronize). Hence, they require you to do this on your own. For example, you can decide to delete and recreate the statefulset, then delete one of the pods after the other; they will be automatically recreated immediately, as usual. You can then wait until the next pod has reached a state where it is synced with the other ones (in whatever way your software does it), then delete the next one etc., until you have upgraded them all.


How to find interdependencies between pods in a Kubernetes cluster? Two Pods run in a Kubernetes cluster. One is a simple Wordpress application and the other a Mysql database. The Wordpress Pod communicates with the Mysql database.

The aim is to find dependencies between pods. Is there any kubectl command or any tool like prometheus that is able to find interdependence between Pods?

Have a look to solutions implementing a service mesh.

For example ISTIO, Consul, Ambasador + Envoy...


How to make Load balancer to wait for custom program to finish running? I have a problem I am trying to solve. When the auto scaling creates a new EC2 instance, the load balancer gets a positive health check result fairly quickly. However, the problem is that we have a custom application that runs when an instance starts and we do not want the instance attached to the load balancer until it has completed. So I need to find a way to make the load balancer wait until our custom application is finished running.  What is the best way to accomplish this?

Use the health check capabilities to ensure that your instance is not in service before your custom program is ready.

If you have a classic ELB, you can point to an specific enpoint, for example, /health_check.php that performs the necessary checks in your application to ensure that your custom program is ready. Then, when a new instance is attached to the load balancer, it will perform the health check action that you have configured an expects a right behaviour of it. If not, the health check fails and will continue to perform new health checks in the configured intervals until instance become in service.

On best practice for web servers behind a load balancer, for example, is to add an specific endpoint that checks that all the necessary pieces for your application is running and in service, that is database, storage, caching processes or whatever you new or consider critical for your system.

The official AWS documentation is very useful for this.

Hope this helps


Cannot Change Docker Size on Ubuntu 16.04 Docker version 18.03.1-ce, build 9ee9f40

I am trying to increase the docker container sizing on Ubuntu 16.04.

I have created the following file at /etc/docker/daemon.json

{
    "storage-opts": ["dm.basesize=20G"]
}

However when I sudo service docker start
I get: Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.

If I remove /etc/docker/daemon.json then docker can start.

Inspecting the recommended logs yields

systemctl status docker.service
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: inactive (dead) (Result: exit-code) since Sat 2018-05-05 12:58:26 PDT; 4min 35s ago
     Docs: https://docs.docker.com
  Process: 3047 ExecStart=/usr/bin/dockerd -H fd:// (code=exited, status=1/FAILURE)
 Main PID: 3047 (code=exited, status=1/FAILURE)

May 05 12:58:26 stage systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
May 05 12:58:26 stage systemd[1]: Failed to start Docker Application Container Engine.
May 05 12:58:26 stage systemd[1]: docker.service: Unit entered failed state.
May 05 12:58:26 stage systemd[1]: docker.service: Failed with result 'exit-code'.
May 05 12:58:26 stage systemd[1]: docker.service: Service hold-off time over, scheduling restart.
May 05 12:58:26 stage systemd[1]: Stopped Docker Application Container Engine.
May 05 12:58:26 stage systemd[1]: docker.service: Start request repeated too quickly.
May 05 12:58:26 stage systemd[1]: Failed to start Docker Application Container Engine.

journalctl -xe
--
-- Unit docker.socket has begun starting up.
May 05 12:58:26 stage systemd[1]: Listening on Docker Socket for the API.
-- Subject: Unit docker.socket has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.socket has finished starting up.
--
-- The start-up result is done.
May 05 12:58:26 stage systemd[1]: docker.service: Start request repeated too quickly.
May 05 12:58:26 stage systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has failed.
--
-- The result is failed.
May 05 12:58:26 stage systemd[1]: docker.socket: Unit entered failed state.
May 05 13:01:16 stage sudo[3069]:      mjb : TTY=pts/0 ; PWD=/var/www/app ; USER=root ; COMMAND=/bin/cat /etc/docker/daemon.json
May 05 13:01:16 stage sudo[3069]: pam_unix(sudo:session): session opened for user root by mjb(uid=0)
May 05 13:01:16 stage sudo[3069]: pam_unix(sudo:session): session closed for user root

How can I increase the default dm.basesize of Docker on Ubuntu such that, when I start docker using the command sudo service docker start it starts and has an increased dm.basesize?

The only log message there is "Start request repeated too quickly." so we're missing the beginning of the error. You can run as root systemctl reset-failed and then systemctl start docker followed by journalctl -u docker --no-pager -n 50 to show the last 50 lines of logs.

Most likely, your problem is that your storage driver is not device mapper, and this option only applies to device mapper. On an 18.03 install, ubuntu will be overlay2 with a recent enough kernel, or possibly aufs on older kernels. Without using the device mapper storage driver, you should not be setting that option.


How to use in Chef different attributes values for different nodes I have a cookbook that is being used to deploy some software on a chef nodes. I use it when bootstraping a node and then remove it from the run_list since I need it only to run once. I don't know if it's proper to do thing like that in Chef or if it's an antipattern. But this cookbook needs to have some variables that change on every node. Let's say that it needs to know the name of the main network interface of that node and I need to provide to it some kind of license number. I do that by using attributes, I specify in attributes/default.rb:

default['cookbook_name']['interface'] = 'eth0'
default['cookbook_name']['license'] = 'abcxyz'

And then I upload this to chef-server and run recipe on a node that is using this attributes. But if the other person wants to deploy the same software using this cookbook on some  other node, then he needs to wait until I finish because if he will replace those values in attributes.rb with values of second node and upload it to chef-server then it will overwrite values that I need right now.

What is the proper way to do that? I feel like I am missing something about the work flow of attributes. As far as I see, using data bags will not solve the problem. Is there some way to assign "attribute" only to a specific node? Can I assign an attribute only to a given node?

You may use attributes precedence for that or a wrapper cookbook, I highly encourage you to take some time to check out https://learn.chef.io which cover those uses cases.

This is a kind of antipattern, one shot configuration are not what Chef is for, but for this use case I'd go with a common cookbook doing what is needed and a wrapper cookbook defining the attributes and including the common recipe.

A typical wrapper cookbook would need a few files:

metadata.rb as another cookbook with a line depends 'common_cookbook'
attributes\default.rb with the values for this type of server, same syntax as in common_cookbook
recipes\default.rb with the line include_recipe 'common_cookbook'

This pattern is also known as role cookbooks, more details in this blog post


How to build Java project on existing virtual machine in Azure? In my project we are using Azure DevOps in which we have a pipeline which

pulls code from code repository
downloads Ubuntu container
installs Maven
downloads all the dependencies
then tries to build code

With this approach, every time we commit code in the repository, new container gets downloaded and Maven dependencies are downloaded as well.

Downloading dependencies itself is taking 10+ minutes which is kind of frustrating.

Is there any way in which we can cut down this build time? We have couple of virtual machines in Azure, is it possible to tell Azure DevOps pipeline to use these virtual machines to build the code instead of using containers?

I am sorry if this question is too broad/generic but I am new to DevOps and Azure in general so I don't know specific terms from DevOps world.

Thanks,
Chandra

You can use a custom build container image for your pipeline with Maven and all of your other dependencies. You will need to:

Create a Dockerfile that installs Maven and all other dependencies that you have
Build the image and upload it to a custom Azure Container Registry
Modify your YAML pipeline file to reference the new build image

Sample:

resources:
   containers:
     - container: build_container
       image: demoxyz.azurecr.io/pipeline-build-image:latest
       endpoint: AzureCR
       options: '-v /usr/bin/docker:/usr/bin/docker -v /var/run/docker.sock:/var/run/docker.sock'

Here is an article that discusses the topic.


How to Chef things that don't exist yet Let us say I have some Chef code like:

require 'mixlib/shellout'
yum_package 'somepackage'
myvar =  Mixlib::ShellOut.new('/bin/somecommand').run_command.stdout.strip

Where /bin/somecommand does not exist yet because it is installed by somepackage. This will fail at recipe compile time for that reason, but will obviously work at convergence time providing the package installs successfully (and if it doesn't then obviously the recipe has failed anyway). This also fails if the package is installed be a previous recipe in the runlist since they are all compiled together upfront. How do I include things in a Chef recipe that that recipe or runlist installs itself?

I'd go with node.run_state to store a transient variable in a run and define it in a ruby_block so it happens at converge time, something like this:

yum_package 'somepackage'

ruby_block 'set myvar' do
  block do
    node.run_state['my_var'] = Mixlib::ShellOut.new('/bin/somecommand').run_command.stdout.strip
  end
end

As far as I know requiring 'mixlib/shellout' is not necessary.


Is it possible to install RPM build on Debian? I have a Docker-based Debian Sid build environment for Go. The build script of an is configured to build both .deb and .rpm.

The rpm build process of a component I  wish to build requires though the tool rpmbuild.

Is it possible to install rpmbuild on Debian or should I better try another Linux distro?

You can indeed build RPMs on Debian. You need the rpm and rpmbuild8 packages:

I tried it with a docker container:

docker run --rm -it debian bash -c 'apt-get update && \
apt-get -y install rpm librpmbuild8 && which rpmbuild'

More info on packages:

The following additional packages will be installed:
  dbus debugedit libapparmor1 libarchive13 libdbus-1-3 libdw1 libexpat1 libgdbm-compat4 libgdbm6 libicu63 liblua5.2-0 libmagic-mgc libmagic1 libnspr4
  libnss3 libperl5.28 libpopt0 librpm8 librpmio8 librpmsign8 libsqlite3-0 libxml2 netbase perl perl-modules-5.28 rpm-common rpm2cpio
Suggested packages:
  default-dbus-session-bus | dbus-session-bus rpm-i18n lrzip gdbm-l10n file sensible-utils perl-doc libterm-readline-gnu-perl
  | libterm-readline-perl-perl make libb-debug-perl liblocale-codes-perl alien python elfutils rpmlint rpm2html


What are pros and cons of Hackathons on implementing a DevOps culture? I am curious about the efficacy of using a "hackathon" to focus development early in a new project before moving a more formal DevOps process.

Does letting things start out fast and loose to get things going hamper later efforts to add process?
If so, how does one weigh the benefits of building that early momentum vs. establishing a good DevOps process?

If the project is for a startup's minimum viable product or some other sort of sanity-check/proof of concept effort - hackathon before DevOps is often recommended - basically in the "fail-fast/fail cheap" lean startup strategy.

Some might recommend hackathon before DevOps in other situations as well, for time-to-market reasons. But if DevOps doesn't followup fast enough that initial market bridgehead can easily collapse. Possible reasons for that include:

that early momentum can be ephemeral - easily lost as soon as the DevOps process starts to be established ("it's too hard, it's slowing us down")
typically hackathons cause accumulation of technical debt. The cost of managing and eliminating it can outweighs the benefits of that early momentum.


Reflect GitLab repo changes in VSTS code I have a repository in GitLab that is used by our development team. On the other hand, there is a VSTS setup (at xyz.visualstudio.com) that we use to manage our scrum board.

As of today, we have to manually import the repository in VSTS every time a feature is finalized. Once the repository is imported, it does not pull the latest commits by itself.

How do I ensure that every time a developer commits code to GitLab repository, same is displayed in VSTS automatically? Am I missing something when I use "import repository" feature in VSTS?

I'm using https://docs.gitlab.com/ee/workflow/repository_mirroring.html#pushing-to-a-remote-repository to push everything into a repository in VSTS, easily done via access token with the right permissions.


How to automatically update Swarm services when the Docker daemon restarts I'm using Docker Swarm, and I have condition: always in the restart_policy of my compose file, so my services are started automatically when the system reboots or the Docker daemon is restarted.

What I'm trying to achieve is to also have the services pull and run the latest image when the system reboots or the daemon is restarted. This is for an IoT device, so yes, I really do want to do this.

Now, if I manually take down and restart the stack, or if I use docker service update --force --image my-image:latest my-service, then the latest image is pulled and ran. But if I restart the system, it just keeps running the old image, and doesn't pull the latest one.

I could work around this by removing the restart policy and using systemd to manage lifecycle instead, but I really don't want that extra hassle or complexity, and I'm not even sure how to work that to restart services that crash.

Is there some way to achieve this with Docker Swarm alone?

You don't need a restart policy with swarm mode. Swarm mode will constantly try to reconcile any differences between the current and target state. I've seen cases where the restart policy enforced by the docker engine, and the implicit reconciliation in swarm mode have conflicted and resulted in issues. The restart policy would also restart the stopped container which would implicitly break the goal to use a new image since the image can not change for the life of a container. Swarm mode, conversely, will recreate any stopped containers with a new container.

To have swarm use the current value of a tag, rather than resolving that tag to an image digest on the initial deploy, you need docker stack deploy --resolve-image .... Note that I haven't tested if this will pull new images, I've only needed to use the feature to use locally built images that shouldn't be pulled from a registry. If Swarm mode doesn't pull new images, you'll also want an extra process to pull your image periodically.


How to automate the magento2 installation process too via docker This is what I have accomplished up to now. Written a docker file where I have done the following and for now I have not used the docker compose.

Used alpine as base Os
setup nginx
setup php-fpm
Installed magento2 file

So my docker build is ready and the container is ready and can see the setup page when opening it on the browser.

My problem
How do I automate  this Magento2 installation process from dockerfile. Like setting up database and the other installation process.

Instead of creating a dockerfile yourself you could also pull a docker image, e.g. alexcheng/magento2.


Calling a Terraform object in seperate folder I need to reference my VNET Gateway residing in an infrastructure folder, in a VPN folder in order to connect a VPN utilising a Local Network Gateway, but these could be any object in Azure for the sake of argument. These folders both use seperate state files hosted in Azure blob storage.

What would be the best way to go about calling this VNET Gateway's ID in another folder? The usual methods of using e.g "module.vnet_gateway.id" or e.g "data.azurerm_vnet_gateway.id" won't work in this scenario.

I've considered some way of referencing it by name, which I know, and pulling the id as an output, but I'm not certain this is the best method.

I've read about using remote-state, which I assume as per the guides on Google would involve creating a data-source pointed to my specific state file for "inf", and then pulling the data back, but I've yet to find a good example on Azure of how to achieve this.

Any pointers or examples StackExchange could provide to get me in the right direction?

Using a remote state file and referencing the state from a different state template would be the way I would go. Would need to add an output to resource that needs to be shared as well as configuring remote state storage, access controls, and security.
This URL has example of setting up the remote backend in azureRM.
https://www.terraform.io/docs/backends/types/azurerm.html


kubernetes: how to set "top node" and "top pod" permissions? Hoping for some pointers, I have a question: what role/permissions do I need to bind to a user that has ClusterRole=view in order to let that user use top node and top pod commands in kubectl?

My team are using kubectl version 1.15.x (client) on a 1.15.7 cluster (AKS), although I'm not sure if that's relevant information. I tried to find this in the k8s documentation, but could not find the permission-reference description, only some examples and guidelines. If you know where to get the full k8s reference, I'm equally glad and will post the solution.

Note, I also posted this on StackOverflow, but they pointed me here, which indeed looks much more suited.

Thanks in advance!

Regards, Ludo

Thanks to arghya-sadhu's comments on stackoverflow, I've got it working:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: get-pod-and-node
  labels:
    rbac.authorization.k8s.io/aggregate-to-view: "true"
rules:
- apiGroups: ["metrics.k8s.io"]
  resources: ["pods", "nodes"]
  verbs: ["get", "watch", "list"]

(I'm aggregating to the standard 'view' role)


Cleanest way to prematurely exit a Jenkins Pipeline job as a success? I have a job that will create files, unless one of the values being fed to it matches an older value. What's the cleanest way in Jenkins to abort or exit the job, without it being FAILED? It exiting is the correct behavior so I want the build marked SUCCESS.

It'll end up in an if statement like so;

stage ('Check value') {

     if( $VALUE1 == $VALUE2 ) {
       //if they do match exit as a success, else continue with the rest of the job
    }

}

I don't want to throw an error code unless that can somehow translate into it being marked a successful build.

Figured it out. Outside of any stages (otherwise this will just end the particular stage as a success) do the following;

if( $VALUE1 == $VALUE2 ) {
   currentBuild.result = 'SUCCESS'
   return
}

return will stop the stage or node you're running on which is why running it outside of a stage is important, while setting the currentBuild.result prevents it from failing.


Programmatically scrape the latest version of Tomcat for an installation script Looking for some assistance or ideas on how to grab the latest version of a Tomcat release from their website or other website.

I came across these Stack Overflow/Stack Exchange links that were relevant below:

https://stackoverflow.com/questions/22510705/get-the-latest-download-link-programmatically (Nginx, but same idea)
How to get the latest Tomcat version?

..But unfortunately both led to some dead-ends. It did give me some ideas though.

I have some installation scripts where the download URL is predictable based on the version number of the software, so if there's a consistent way of scraping for the latest version of the software I simply add a variable into the script for it to curl / grep for that version on the text and use that variable for download links, untaring, moving, etc.

Hoping to do something with Tomcat in the same vein. A installation script I won't need to keep updating (unless they change their site).

Any sort of thoughts, ideas, are appreciated.

I've also looked at http://tomcat.apache.org/whichversion.html which lists the "Latest Released Version" and this seems like maybe the best, more reliable location to grab the version, but I had trouble getting that with Curl / Grep because of the table structure. I am only just scratching the surface with learning those commands, so maybe someone more well versed could get that working, otherwise I'm definitely open to other thoughts!

curl -i https://api.github.com/repos/apache/tomcat/tags | grep '"name"' | head -1 | egrep -o "([0-9]{1,}\.)+[0-9]{1,}"

It seems to work but I am not sure if I am entirely happy with it. An additional use-case that I thought of, is it would be nice to grab the latest version of each major version instead of just the absolute latest.

Do you have suggestions on how you would parse the highest number with the link you provided, and/or how to start in one of those directories and parse the highest there? That way for example I could create a latest version install script for Tomcat 7, 8, 9, etc.

Thank you!! (Like I said I am just getting started, so please my apologies with the lack of knowledge but I do appreciate digging in and understanding how the commands work or reached their conclusion)

I've created a couple crude methods of grabbing the latest major versions of releases, as well as absolute latest.

# Grab Latest Tomcat 7
curl -i https://www-us.apache.org/dist/tomcat/tomcat-7/ | grep -Po '(?<=(v)'

# Grab Latest Tomcat 8
curl -i https://www-us.apache.org/dist/tomcat/tomcat-8/ | grep -Po '(?<=(v)'

# Grab Latest Tomcat 9
curl -i https://www-us.apache.org/dist/tomcat/tomcat-9/ | grep -Po '(?<=(v)'

# Grab Latest Tomcat
curl -i https://api.github.com/repos/apache/tomcat/tags | grep '"name"' | head -1 | egrep -o "([0-9]{1,}\.)+[0-9]{1,}"

It's a bit of a hack where I'm looking for text in-between two values on a web page to grab the version number. It's crude and I'm sure there's a more elegant way to grab this, but this what I was able to figure out with my skillset! Maybe it can help someone in the future, but if anyone has better ideas I'm all ears.