Trigger downstream pipeline job via generated variable in upstream freestyle job I have an upstream freestyle job and a downstream pipeline job.

The freestyle job is triggered via a Bitbucket hook to run whenever code is pushed to Bitbucket.

The freestyle job should capture the commit SHA and branch name and pass both to the pipeline job. It also generates a variable based on the branch author, that should also be passed.

In the freestyle job I do something like the following, which works and echos the expected parameters.

cd $WORKSPACE/prancer/
GIT_AUTHOR=$(git show --format="%an" | head -1)

echo '-------------'
echo 'git_commit is '$GIT_COMMIT
echo 'git_branch is '$GIT_BRANCH
echo 'git_author is '$GIT_AUTHOR
echo '-------------'

I have the job configured to kick off the pipeline job like so;

Downstream, the pipeline job echos the GIT_COMMIT and GIT_BRANCH as expected. However, the GIT_AUTHOR just echos the variable name, not a value.

Somehow, the git commit and branch are being passed along, but the generated variable git_author isn't. I tried piping the author to a file, archiving it in the freestyle job and using the additional Parameters from properties file step, but that still isn't passing to the pipeline job.

I can confirm that

GIT_AUTHOR has a value in the freestyle job
If I save GIT_AUTHOR to a properties file, I can view the file (with the author inside!) in the workspace of the freestyle job
The pipeline job isn't receiving the value, just the actual string $GIT_AUTHOR. If I check the parameters page for the triggered pipeline build commit and branch have values, author is the string $GIT_AUTHOR.

How do I use a generated variable's value from an upstream freestyle job when building a downstream pipeline job?

The variable GIT_AUTHOR is only visible in the build step where it's defined. You can dump the parameters for the downstream job in a properties file:

cd $WORKSPACE/prancer/
GIT_AUTHOR=$(git show --format="%an" | head -1)

echo '-------------'
echo 'git_commit is '$GIT_COMMIT
echo 'git_branch is '$GIT_BRANCH
echo 'git_author is '$GIT_AUTHOR
echo '-------------'

cat > alex_test_pipeline.properties &lt&ltEOF
GIT_COMMIT = $GIT_COMMIT
GIT_BRANCH = $GIT_BRANCH
GIT_AUTHOR = $GIT_AUTHOR
EOF

In the Trigger parameterized build... step, replace the Predefined parameters with Parameters from properties file (or something like that - I don't remember the exact wording) and select the filename used above. You might also want to check Build only if file exists.


StatefulSet or Deployment for Jira on Kubernetes We are looking into moving our Jira instance on a Kubernetes cluster and one of the concerns that were recently risen is whether we should go with a StatefulSet or a Deployment. So, are there any suggestions or lessons learned by using any of the above controllers? Any additional tips, source of information and comments by people who went through the process could be very valuable to us.

Deployments are primarily based on stateless services, where data is externalized to be stored on some other storage block or database. Using deployments for stateful services is not recommended as you will encounter scaling problems, since multiple replicas will read from the same persistent volume.

Instead, StatefulSets are the best approach to stateful services. Each replica of the StatefulSet has its own state mounted on a dedicated persistent volume, thus scaling a StatefulSet will create a new persistent volume and attach the new replica to it.


Is there any relationship between "production engineering" and DevOps? Production engineering: "Production engineering is a combination of manufacturing technology, engineering sciences with management science." (Wikipedia)

Is there any relationship to/with DevOps?

To avoid an opinionated discussion, please establish, formalize with facts and use references in your answers.

According to Facebook

  Production Engineering at Facebook is a hybrid between software and
  systems engineering; it keeps Facebook running smoothly and scaling
  efficiently. The Production Engineering team works within Facebook’s
  product and infrastructure teams to make sure our services are
  reliable and scalable. The team writes code and debugs hard problems
  in live production that impact more than 2 billion people around the
  world. From backend services like our Hadoop data warehouses, to
  frontend services like News Feed, to infrastructure components like
  our caching infrastructure, load balancing and deployment systems, the
  Production Engineering team keeps Facebook running.

as team works within product and infrastructure meaning it works along with developers and operations.


Is there a way to run the Jenkinsfile from command line? I don't want to use Jenkins UI, instead I would like to use the command line, from command line I want to execute Jenkinsfile to the Jenkins server.

Is this possible?

Let's say I have a Jenkinsfile, If I run the Jenkinsfile, It shoud execute the Jenkinsfile from Jenkins server. Assume that the job not yet configured through Jenkins UI.

I've done this before. Essentially you can use the Jenkins CLI to import/export jobs, which will allow you to create and/or update a job with the definition you want to run. Be it JenkinsFile or some groovy script or whatever. Check this blog for details.

Once you (re)define the job, you will simply remotely execute it using the same Jenkins CLI. You can check the progress of the job from CLI and even read the logs to see progress. I cannot share the code, as the copyright is owned by one of my previous employers, but it is easy to recreate.


Deploying to VSphere with Packer and/or Terraform? Scenario:

Creating multiple VMs and deploying them to VSphere. Current development uses Packer and Ansible to provision a Fusion VM, with the aim of using Terraform to deploy to VSphere.

Issue:

I've been having loads of issues with uploading Fusion VMs to VSphere and having to use work arounds such as this, and been wondering if there is a better of doing this.

Question:

I'm questioning the use of Packer in this scenario (which deploys to Fusion), and doesn't seem to currently support a Packer->ESXi (VSphere) builder unless it's an ISO.

Would it be possible to remove Packer entirely from this scenario, so instead of building a Fusion image, provisioning the image, and uploading it to VSphere, use Terraform to deploy an ESXi image (template), and provision that image in-place using Ansible as a provisioner? (such that everything is done in VSphere rather than Fusion then VSphere).

What are the advantages and disadvantages of doing this?

I'm using vagrant with vsphere plugin, starting from a 'base' image already present on vsphere.

The new machine is then fully configured (inlcluding middleware) by chef after vsphere customization.

That's one in many possibilities, unsure this question lead to actionnable answers out of everyone 'workaround'.


simulate post step for dynamically generated Jenkins pipeline stages The following simplified Jenkinsfile dynamically generates sequentially executed stages; however, I cannot create a post step for these dynamically stages, like so:

pipeline {
    agent none
    stages {
        stage('Gen Stages') {
            steps {
                script {
                    def stageNames = ["st1", "st2", "st3"]
                    stageNames.each { stageName ->
                        createStage(stageName)
                    }
                }
            }
        }
    }
    post {
        always {
            echo "post > always"
        }
        success {
            echo "post > success"
        }
    }
}

def createStage(String stageName) {
    stage(stageName) {
        echo "Stage: ${stageName}"
    }
    // I want to uncomment and use code below - or something effectively simiarl:
    // post {
    //     always {
    //         echo "${stageName} > post > always"
    //     }
    //     success {
    //         echo "${stageName} > success > always"
    //     }
    // }
}

I would like to be able to use the commented-out post {} stage, or something effectively similar.

If at all possible, I'd like to use the DSL as much as possible and avoid scripted pipelines.

Suggestions?

You can't use post because post is part of the declarative DSL, not scripted, and you're defining your stages in a script block.  So I'm afraid that as long as you're using a script block to define your stages, you're going to have to stick with the scripted method, which is the groovy try { ... } catch { ... } finally { ... } construct.  For example:

def createStage(String stageName) {
    stage(stageName) {
        try {
            echo "Stage: ${stageName}"
            echo "${stageName} > this is executed only on success"
        finally {
            echo "${stageName} > this is always executed"
        }
    }
}


How to copy HTML files to include into nginx? How do I copy an index.html file into nginx?
root@mordor:/home/nicholas/docker/nginx#
root@mordor:/home/nicholas/docker/nginx# ll
total 12
drwxrwxr-x 2 nicholas nicholas 4096 Nov 13 16:18 ./
drwxrwxr-x 3 nicholas nicholas 4096 Nov 13 16:12 ../
-rw-rw-r-- 1 nicholas nicholas   60 Nov 13 16:18 dockerfile
root@mordor:/home/nicholas/docker/nginx#
root@mordor:/home/nicholas/docker/nginx# cat dockerfile
FROM nginx
COPY /home/nicholas/html /usr/share/nginx/html

root@mordor:/home/nicholas/docker/nginx#
root@mordor:/home/nicholas/docker/nginx# docker build -t some-content-nginx .
Sending build context to Docker daemon  2.048kB
Step 1/2 : FROM nginx
 ---> c39a868aad02
Step 2/2 : COPY /home/nicholas/html /usr/share/nginx/html
COPY failed: stat /var/snap/docker/common/var-lib-docker/tmp/docker-builder726461098/home/nicholas/html: no such file or directory
root@mordor:/home/nicholas/docker/nginx#
root@mordor:/home/nicholas/docker/nginx# ls /home/nicholas/html/
index.html
root@mordor:/home/nicholas/docker/nginx#
root@mordor:/home/nicholas/docker/nginx# cat /home/nicholas/html/index.html

index
hello world

nicholas@mordor:~/html$

I'm unclear on the source and destination, which might be complicated by docker having been installed from snap.

Looks like ansible has slightly clearer, at least for me, syntax:
  tasks:
  - name: "Copying files"
    copy:
      src: files/{{ item }}
      dest: /my/folder/
    with_items:
      - file.txt

or perhaps I'm doing something incorrectly.

The docker build command builds Docker images from a Dockerfile and a “context”. A build’s context is the set of files located in the specified PATH or URL. The build process can refer to any of the files in the context. For example, your build can use a COPY instruction to reference a file in the context.
From docker build documentation.
As you run the docker build with the . path for the context, it will try to find the folder home/nicholas/html from the context root@mordor:/home/nicholas/docker/nginx
The COPY command documentation shows some examples


Is it possible/fine to run Prometheus, Loki, Grafana outside of Kubernetes? In some project there are scaling and orchestration implemented using technologies of a local cloud provider, with no Docker & Kubernetes. But the project has poor logging and monitoring, I'd like to instal Prometheus, Loki, and Grafana for metrics, logs, and visualisation respectively. Unfortunately, I've found no articles with instructions about using Prometheus without K8s.
But is it possible? If so, is it a good was? I know that Prometheus & Loki can automatically detect services in the K8s to extract metrics and logs, but will the same work for a custom orchestration system?

Yes--absolutely you can skip Kubernetes. Suggest that you still use Docker, but you could even just install the components separately.
I think it's easiest using Docker Compose (need to have package docker-compose installed on your host) to manage the containers.
Here is a docker-compose.yml that might be helpful:
version: '3'

services:
    prometheus:
        image: prom/prometheus:latest
        restart: unless-stopped
        ports:
            - "9090:9090"
        volumes:
            - /home/${USER}/opt/prometheus/data:/opt/prometheus/data:rw
            - /home/${USER}/opt/prometheus/config:/opt/prometheus/config:rw
        environment:
            SERVER_KEY: ${SERVER_KEY}
            SERVER_CERT: ${SERVER_CERT}

    grafana:
        image: grafana/grafana:latest
        restart: unless-stopped
        ports:
            - "3000:3000"
        volumes:
            - /home/${USER}/var/lib/grafana:/var/lib/grafana:rw
            - /home/${USER}/var/log/grafana:/var/log/grafana:rw
            - /home/${USER}/etc/grafana/grafana.ini:/etc/grafana/grafana.ini:rw
            - /home/${USER}/etc/grafana/ldap.toml:/etc/grafana/ldap.toml:rw
        environment:
            SERVER_KEY: ${SERVER_KEY}
            SERVER_CERT: ${SERVER_CERT}
            GF_SECURITY_ADMIN_USER: ${GF_SECURITY_ADMIN_USER}
            GF_SECURITY_ADMIN_PASSWORD: ${GF_SECURITY_ADMIN_PASSWORD}
        depends_on:
            - "prometheus"

Can also use Docker Compose for Loki and Promtail.
Docker Compose reference: https://docs.docker.com/compose/compose-file


Who is this Chaos Monkey and why did he crash my server? I had a perfect server, it was so pretty and rock solid and so I named it Petra. It was perfect in every way, everything was configured and tuned just right, it had perfect 100% service record and 753 days of uptime. I've spent a lot of time and effort making sure it run so well. No other server in the company had been this good. But last night this evil monster crashed my server for no reason.

Of course I was notified at 2am and it took me until morning to get it up and running and everything configured and tuned up, but I'm afraid it is not going to be as good as before. It might take weeks before it is back to it's former glory. Now my uptime is gone, I don't have even measly three 9s and who knows what this will do to my reputation. Who is this Chaos Monkey and why did he do that to my server and why is he trying to ruin me?

TL;DR: Chaos Monkey was developed in 2010 at Netflix and released into wild in 2012 is part of the Simian Army, wildly popular among devoted followers. Built on principles of chaos engineering, the army increases resiliency to failure by injecting constant failure to the system.

Concept

Chaos Monkey was developed specifically for AWS where it will randomly kill instances within an Auto Scaling Group. It is meant to run during the business hours when engineers are alert and can quickly react to discovered failures.

Simian Army

Members of the army would sow chaos through other means:

Latency Monkey will introduce random delays to services.
Chaos Gorilla (Kong) will simulate outage of entire availability zone.

Other Monkeys are helpful and remove the weak members of the herd:

Conformity Monkey shuts down instances not following best practices.
Security Monkey looks for known security vulnerabilities in configuration and services.
Doctor Monkey shuts down unhealthy instances not conforming to certain metrics.
Janitor Monkey looks for unused resources to reclaim.

Failure is Inevitable

Failure in the System is inevitable, something will always go wrong. You might not be able to choose what, but you can try to choose when. By introducing small errors throughout the day, you ensure that your engineers are present. By killing non-conforming services quickly, you ensure that failures happen often before deployment. By making the environment more adversarial, you ensure that it will be the developers who run into issues long before any service makes its way into production. Failures will be quickly apparent in integration phase of new services with the old ones, but that is ok, because the old production services are already resilient.

Cattle not Pets

Everyone will tell you lately: Do not treat your servers as pets. There is a power in numbers and any single point of failure will bring down the system. No matter how well you can tune and optimize your server, no matter how beefy hardware you can get, how much it can handle, it will never be a match for herd of small scalable instances. Chaos Monkey encourages you to think about removing all points of failure, because sooner or later, the Monkey will come! Everyone fails and even the Amazon S3 had an unpredictable outage.

Anti-Fragile

So what is the theory and why does it work? Nassim Nicholas Taleb in his book Antifragile describes a concept where living self-aware systems, will benefit from a small levels of randomness and actually become better in face of adversity. This is similar to annealing.

He does also describe an evolutionary way, where fragility of parts in a system is transferring into antifragility of the whole. The transfer occurs on two levels:

By a small random variations - developers making changes - the most fit for the environment will survive and propagate - pass tests and get deployed. Standard Development Life Cycle.
By failure of parts not capable to withstand a larger level of randomness in the environment, the remaining parts that were able to withstand it compose a system that is as a whole better able to deal with changing environment than before. This is essentially Chaos Monkey.

Larger levels of randomness can be withstood using the second approach.


How to trigger jenkins pipeline only on merge to develop branch from github webhook trigger? I am having a problem to trigger Jenkins pipeline job based on merge any branches to develop branch only. I also did not find any webhook for merge as like bitbucket. I also tried using pull request but that triggered the job on any changes to the PR branch which is not my requirement.

I have also shared my code below and my Jenkins Configuration.
Jenkins Configuration also please note that it is develop branch but in the image I've shown */feature I have changed it to develop but still the issue is the same.

def CONTAINER_NAME=""
def CONTAINER_TAG="${env.BUILD_NUMBER}"
def CONTAINER_PORT=""
def ECR_REPO=""
def HOST_PORT=""
def URL=""
def INSTANCE_IP=""

pipeline {
  agent any
  stages {
    stage('Image Build Develop') {
      when {
        expression {
         return env.BRANCH_NAME != 'feature_cicd'
         echo env.BRANCH_NAME
        }
      }
      steps {
        git branch: "develop", url: 'https://github.com/jenkinsci/git-parameter-plugin.git'
        imageBuildDevelop(CONTAINER_NAME, CONTAINER_TAG)
        }
    }

    stage('Image Tag') {
      steps {
        imageTag(CONTAINER_NAME, CONTAINER_TAG, ECR_REPO)
      }
    }

    stage('Image Push') {
      steps {
        imagePush(CONTAINER_NAME, CONTAINER_TAG, ECR_REPO)
      }
    }

    stage('Image Deploy') {
      steps {
        imageDeploy(HOST_PORT, env.BUILD_NUMBER, INSTANCE_IP)
      }
    }
  }
}

def imageBuildDevelop(containerName, tag) {
  echo "#------------------- Checkout Develop Branch -------------------#"
  git branch: "feature_cicd", url: 'https://github.com/jenkinsci/git-parameter-plugin.git'
  echo "#------------------- Login into ECR Repository -------------------#"
  sh "`/var/jenkins_home/.local/bin/aws ecr get-login --no-include-email`"
  echo "#------------------- Build Docker Image for Media-API -------------------#"
  sh "docker build -t $containerName:$tag -t $containerName ."
  echo "#------------------- Image Build Complete -------------------#"
}

def imageTag(containerName, tag, repo) {
  echo "#------------------- Tag media-api image -------------------#"
  sh "docker tag $containerName:$tag $repo:$containerName-$tag"
  echo "#------------------- Image Tag Complete ------------------#"
}

def imagePush(containerName, tag, repo) {
  echo "#------------------- Login into ECR Repository -------------------#"
  sh "`/var/jenkins_home/.local/bin/aws ecr get-login --no-include-email`"
  echo "#------------------- Push Image into ECR Repository -------------------#"
  sh "docker push $repo:$containerName-$tag"
  echo "#------------------- Image Push to ECR Repository Complete -------------------#"
}

def imageDeploy(hostPort, tag, ip) {
  echo "#------------------- SSH into media-api instance -------------------#"
  sh "ssh ubuntu@$ip \"BUILD_NUMBER=$tag\" \
  ' cd reactApi_test/ &&\
    echo \"TAG=${BUILD_NUMBER}\" > .env &&\
    `aws ecr get-login --no-include-email` &&\
    docker-compose up -d --build &&\
    ./test.sh '"
  echo "#------------------- Media-API container started on Port '${hostPort} (http)' -------------------#"
}

You'll want to use the CHANGE_TARGET environment variable.  If that variable exists then you are dealing with a merge request.

You can use the following code snipped to then determine the branch name and alter the job's logic depending on which branch the code is being merged into.

branchName = env.CHANGE_TARGET ? env.CHANGE_TARGET : env.BRANCH_NAME

In your code example you would replace

        expression {
         return env.BRANCH_NAME != 'feature_cicd'
         echo env.BRANCH_NAME
        }

with

        expression {
         branchName = env.CHANGE_TARGET ? env.CHANGE_TARGET : env.BRANCH_NAME
         return branchName != 'feature_cicd'
         echo branchName
        }


What is the optimal speed relation of CI/CD jobs to a (reference) local machine? Today, you could more or less setup a CI/CD toolchain by youself, even locally. Indeed I have seen teams doing it in an ITSM environment because the communication with management did not work.

Nevertherless we know CI/CD is a very important, if not most important, company production line - it produces value, even if it is not seen so by ITSM to give it operations status and priority (in short, SLA for systems which should survive project time instead of setting them up in every project and department over and over).

Let's ignore for simplicity network downloads.

Question: what should be the execution speed of CI/CD relative to a setup on a local machine (if possible):

Same is ok?
Faster - how much faster?
Slower?

You want the execution speed of the CI pipeline as fast as (practically) possible, irrespective of being a local or shared setup or the time required to set it up - this is what effectively dictates the overall development speed in the respective project.

But it should be noted that by speed here I don't mean the duration of each CI execution. The duration of such execution may be directly related to the coverage of the CI QA verification, which translates into stabiliity of the development process and the quality of the product. For example including a smoketest in the CI verification will increase the duration of the CI execution, but will also raise the product quality - it's a good thing.

But I have to mention: such private CI setups (even those SLA-backed CI setups you mention) are often an indication of what the ThoughtWorks team called the CI Theatre. Simply using CI tools doesn't make the process a CI one. Using project/feature/integration branches which still require subsequent branch merges (other than cherry-picked fixes) before release means development is still done in silos (those branches are the silos) - waterfall style.

Side note: Frankly, if I'd be part of a team in which management doesn't understand the importance of CI (let alone not being able to effectively communicate on such critical subject) I'd look for another job - I doubt they'll remain in business for very long ;)


Nexus2: How can I get a list of all artifacts from a specific repository via REST API? I want to get a list (json) of all artifacts in one specific repository via the REST API.

I've already tried:

curl -X GET http://localhost/nexus/service/local/respositories/<reponame>/content/

This works, but it deliveres me a json listing all my subdirectories in this repo. But what I want is a list with all the artifacts contained in all subdirectories.

Someone knows how to do this? Is there a way to iterate through all folders? I was thinking about the search keyword, but I wouldn't know for what to search.

Try this:

# Get status
curl http://localhost:8081/nexus/service/local/status
# List of repositories
curl http://localhost:8081/nexus/service/local/repositories

For the json parameter:

curl -i -H "Accept: application/json" <url>

More information, see this link


Applying ansible playbook to a docker container on a remote host I have automated the setup of several docker containers with ansible. This works pretty well but there are certain recurring tasks that have to be executed within the docker containers.

It is possible to implement this using the docker connection driver for ansible but this only seems to work when ansible is running on the host that is the docker runner. This is not the case and I don't want to run ansible on multiple hosts if possible. Is there a better way to use ansible to execute playbooks within containers?

// I tried it out of curiosity and it works pretty well using ansible directly on the docker runner,... anyway not my favorite solution.

With the help of @Levi (refered to Stackoverflow) I managed to find a way to connect straigt into the docker containers using ansibles docker connection driver and the remote API capability of Docker.

First of all you have to expose the API which is by default not the case. Simply add -H tcp://0.0.0.0:1337 to the ExecStart constant of the systemd script. Then reload the start script and restart the docker service. (From this blogpost)

Then you can list the containers in the inventory file like this:

[containers]
container-name ansible_connection=docker ansible_docker_extra_args="-H tcp://1.2.3.4:1337"

And execute any playbook like your used to.

ansible-playbook -i docker_inventory playbooks/my_playbook.yml

The upside of this way of implementing it this way is that you can mix up "real" hosts/vms that are maintained through SSH and Docker containers. One negative thing I have to mention is that the docker API is much slower than using SSH as a control tunnel.

This setup is of cause horribly insecure and its only purpose is to explain the concept. Before using it in production you should read into the security features and limit access to the daemon as far as possible.


What are best practices for implementing and using Elastic in an organization? At the moment there are multiple microservices that log to individual files. If there is an issue then the team is debugging like headless chickens. In order to prevent this the aim is to introduce aggregated logging, e.g. elastic. What are best practices to migrate to such an aggregated logging system?

Current idea

Deploy an elastic stack locally
navigate to http://localhost:5601
login by entering elastic and changeme
get the endpoint and configure it in every microservice? (https://www.elastic.co/guide/en/cloud/current/getting-started.html)

What endpoint should be configured?

One best practice would be to log everything to the same location. Your "Current idea" suggests that you'd be running the Elastic stack locally, which is great for a local development environment, but for anything outside that you want to be shipping the logs from all your microservices to the same location. This enables you to do correlation between logs, and have a "single pane of glass" approach to your logging.

To achieve that, you're going to have to run it on a server or use a managed service such as Elastic Cloud or AWS Elasticsearch. You can install the components through package managers e.g. yum or apt, https://www.elastic.co/start, or it's also possible to run it in Docker. The stack-docker repository is great for running locally, but it's not suitable for any sizeable deployment due to a lack of scalability.

The endpoint you ship to will depend on what method/protocol you're using to ship logs, for example Nginx running on an AWS EC2 instance might use Filebeat and ship to a load balancer that sits in front of Elasticsearch, or if you're using Docker you might use the gelf Docker logdriver to ship your logs to Logstash.

Some more best practices:

Log all your messages in JSON, with a focus on objects/fields rather than long string messages.
Exceptions should be actionable, anything else can be a warning.
Avoid multiline logs.
Use correlation IDs. If your app is passed a correlation ID it should include it in every message triggered by that call, otherwise it should generate one. The correlation ID should be included in calls to other microservices.
Err on the side of logging more rather than less, you never know when you may need that bit of data, or what trends you might find by analysing an otherwise inconspicuous piece of data.
Separate your environments logically so you can have different retention periods and archiving on your logs.


Reverse Proxy and TCP Optimization Inside the Network Need to make some optimizations and have been reading articles such as linux-tcp-tuning

In a world of load balancers and revers proxy is there any value in tuning TCP connections that are inside the network?
As I understand (please correct me if I'm wrong) there is almost zero latency and no packet loss between the load balancer and the servers.

So I wonder if changing parameters such as net.ipv4.tcp_window_scaling, net.ipv4.tcp_timestamps, net.ipv4.tcp_sack will have any effect if not set on the load balancer.
Thanks!

EDIT:

Other parameters improved my server in simulation. I set net.ipv4.tcp_fin_timeout=5, net.ipv4.tcp_keepalive_probes=5 and it served more clients (without a load balancer). I just don't know if it will have the same effect with a load balancer. There are so many TCP parameters under /proc/sys/net/ipv4/. Testing them on a single server is one thing. Testing them with a load balancer and many servers is a huge effort. I just don't know if it worth the effort of testing those parameters with many severs and a load balancer.

Tuning the parameters you have identified will certainly have an effect - to degrade performance. For example, net.ipv4.tcp_window_scaling is an on or off option. Similarly, net.ipv4.tcp_sack is either on or off. Both default to a value of on on Linux and on all load balancers. Both of these were defined in RFC1323 which was published in order to give us the ability to achieve 10/100 Ethernet networks. Without this RFC, networks were limited to 10 Mbps a second, so disabling these will result in a huge, immediate and noticable performance hit.

These performance benefits come because there is a huge difference between "almost" zero latency and zero latency. Using a sliding TCP window allows equipment to use Selective Acknowledgements (SACKs) to acknowledge the receipt of hundreds of packets simultaneously. So, if latency is 10ms on a network, without SACKs, every packet has to be acknowledged and that 10ms latency adds up very quickly - even at theoretical minimum latencies. This means that I send packet A, then have to wait for a response to packet A before sending packet B. But as it turns out I can send the rest of the alphabet and then some while waiting for the ACK to packet B. Enter SACKs - These allow the receiver to signal that it is has successfully received up to and including packet Z. Meanwhile, the sender is already sending packet AA-ZZ.

This results in something known as a "TCP Window Size" which is the maximum amount of data the client can buffer/handle. Since memory can be a concern, if the recipient is only capable of storing and processing Packets A-F, if we send A-Z some amount of data is going to be lost. Therefore, it becomes necessary for the recipient to signal their ability to receive data - that is "I can only buffer this many packets). This is really CPU dependent on how fast the recipient can empty their buffers and transmit the data up to layer 7. It is not uncommon to see TCP windows scale from empty to full (Max window size goes from 0 indicating the window is completely used to max indicating the window is completely available) and this is one way throughput bottlenecks can be identified on your network.

TCP Window Scaling offers two primary benefits:

Less unused time on the wire - we aren't sitting around waiting for an ACK and we have successfully defeated the prop. delay (latency).
Less time devoted congestion control = more time for data. Congestion control messages can eat into your transmission time and bandwidth. Even with TCP Window Scaling, Congestive Collapse can still occur in which an entire TCP window may need to be retransmitted and congestion control may be required.

Finally, the only real reason to update net.ipv4.tcp_timestamps is to provide additional security as it is otherwise based off of server uptime. Since you are proxying behind a load balancer anway, adjusting this setting is of little value unless you are using something like a Performance Layer 4 profile. I doubt adjusting this will make much difference, though again it is only likely to degrade, not improve performance.

Far and away, what is more likely to help is adjusting your send and receive buffer sizes and using Jumbo Frames. this will allow you to process more data with the same amount of overhead (a 75% reduction in processing over a standard MTU of 1500. These are the primary changes required to reach a 10 Gbps connection. Otherwise, just make sure you have enough CPU to ingest your data from the TCP stack and be aware of the pattern of your traff (eg, avoid things like the C10K problem).

EDIT:

tcp_fin_timeout doesn't actually improve performance, instead it defeats resource constraints. This setting could actually be a major player in a load balancer implementation. The problem is that for any given client and server, the server will be hosting a web server (for example) on 192.168.1.5:80. Every TCP connection must have both a source IP and source port as well as a destination port and destination IP. Source ports are chosen by the client from a pool of "ephemeral" ports. This results in a theoretical maximum of 65534 connections (the maximum number of ports) between a given client to port 80 on that server. When using load balancers, this can result in "port exhaustion" when the pool of ephemeral ports is depleted when SNAT (Secure Network Address Translation) is used on the load balancer, which results in a theoretical maximum of 65534 between the load balancer and the server. There are two basic solutions to this problem: 1) use a SNAT pool where 2 source IPs for the load balancer * 65534 = 131068 connections and 4 IPs X 65534 = 262136 max connections, and so forth or 2) don't use a SNAT. For this to work, the load balancer needs to be part of the default route back to your clients. If you are testing with a single actual client simulating multiple clients, this is a poor real-world test and you may actually support many more clients than you think because you are exhausting your ephemeral ports, where real clients would never do that because they come from unique IPs and two clients can use the same ephemeral port if you are not using SNAT on the load balancer.

Now, I keep mentioning that this is the theoretical maximum but the reality is that on Linux servers, there is a defined range of ephemeral ports and so typically you will see port exhaustion at around half of the theoretical maximum. You can see results similar to adjusting tcp_fin_timeout by adjusting the ephemeral port range in /proc/sys/net/ipv4/ip_local_port_range.

This brings me to the final piece, the TIME_WAIT state governed by tcp_fin_timeout and defaults to 60 seconds. This gives every port used a 60-second "cooling off period" before it is returned to the ephemeral port pool. This is why you saw a performance boost in your testing. This setting is supposed to be 2 times your maximum segment lifetime as defined by the TCP protocol, as it is theoretically possible that for a FIN packet with this setting below below 1 X maximum segment lifetime (30 seconds) to still be in transit. In reality, this is very unlikely, but technically you are breaking RFC compliance by adjusting this setting. If you adjust this setting downward however, you would want to be sure to adjust it downward on the Load Balancer's TCP profile (so tcp_fin_timeout corresponds to Time Wait, not FIN Wait on an F5, for example).

Ultimately, this is a fantastic example of where testing with a load balancer and many servers would result in a huge performance difference - because your testing probably doesn't reflect the real-world architechure. It isn't a good simulation of the customer experience.

However, tweaking both ephemeral port settings and using keepalives are two divergent traffic management strategies. In fact, if you use a setting like OneConnect, there simply is no longer any need to tweak your ephemeral port ranges and timeouts because connections are recycled and multiple clients can be funneled over the same connection (note, this only works for HTTP traffic, but similar can be achieved with tcp_keepalive_probes if you want to configure it that way because clients no longer have to wait for the three-way-handshake - they can simply start transmitting.)

Conclusion

There is no pat answer here. You are asking "Is it worth doing all this work?" - I don't know. Is it? What is the acceptable level of business risk? What does it cost your business if you get it wrong and bottleneck?

Ultimately, I feel you are basically asking two things 1) what do all these settings do and what is the effect of tweaking them? And 2) are there any shortcuts I can take?

Unfortunately, you have not provided enough information to answer either question (which is why your last question was closed and this one will be downvoted.) First, I don't have the time to explain what every TCP setting in proc does (nor do I know off the top of my head, I'm actually googling these on the fly.) but secondly (and more importantly) it is impossible to determine their effect without knowing how your service is architected. There is probably performance that can be squeezed out of your system, but kernel TCP parameters are pretty much tuned to the best possible configuration that can be safely made without breaking the TCP/IP RFC. What you are doing is pushing into territory that can be safe, but only if you understand the architecture and the TCP/IP protocol. And it would behoove you to learn it. If you want to be the smartest guy in the room, learn TCP/IP. Pick up a copy of TCP/IP Illustrated and/or Internetworking with TCP/IP and commit the basics to memory. Learn how HTTP requests and SSL handshakes work. Learn how to read a pcap in wireshark and tell the story of what is happening according to these protocols. This will take you very far in life whether you go in to System Administration, Networking, or Programming (I have used this knowledge in all three realms). Simply put, there is no substitute for knowing what the hell you are doing and this just isn't something you are going to be able to answer off of Stack Exchange. This is why you pay consultants - because if they are good they know the ins-and-outs of this. Once you understand 1) your architecture and 2) what the settings actually do and what effect they will have you will better be able to tell if you can take any shortcuts (that is, I don't need to tweak this setting; it is irrelevant. This testing will be worthless because it doesn't simulate the customer experience.


How to set an existing IAM role to a new instance while spinning in terraform? I am trying to attach an existing role created in AWS, but I am not able to add it in Terraform Code. I tried to add the role in instance profile but it didn't work either for me.

Is there any direct way to add it in the resource in Terraform Code ?

iam_instance_profile  = "my-role"

  Note: my-role is having the full access of ec2.

you can add role like below

role = "${aws_iam_role.ec2_role_name.name}"

Please check - https://www.terraform.io/docs/providers/aws/r/iam_instance_profile.html


What could be a valid definition of DevOps to introduce it to a novice? I've done / created lots of SCM related presentations, and now I'm trying to "upgrade" to a DevOps successor of it.

What I always try to do in my presentations, is to come up with an introduction slide which somehow includes the message I want to deliver (and which I then elaborate on in the rest of my presentation). When doing so, I try to answer my own question like "What would me 1 to 3 phrases be that I'd want to use if I got like 10 to 20 secs (only!) to explain it to somebody new to it?*".

I thought I knew what DevOps actually means, and what it is about. But I've seen some bizarre usages/contexts of DevOps (even on DevOps.SE ...). It makes me wonder if maybe what I think DevOps is, is completely wrong.

So what is generally agreed to be the definition of DevOps?

DevOps in a nutshell

From Wikipedia:

  DevOps (a clipped compound of "software DEVelopment" and "information technology OPerationS") is a term used to refer to a set of practices that emphasize the collaboration and communication of both software developers and information technology (IT) professionals while automating the process of software delivery and infrastructure changes.

  It aims at establishing a culture and environment where building, testing, and releasing software can happen rapidly, frequently, and more reliably.

From Overview:

  Venn diagram showing DevOps as the intersection of development
  (software engineering), operations and quality assurance
  (QA)

While there is no single "tool" for DevOps, but rather a set of tools, also known as a DevOps toolchain:

  Illustration showing stages in a DevOps toolchain

Illustrations of DevOps

Below are a few quotes from some of the questions on DevOps.SE, which all seem to somehow fit / confirm part of the DevOps description above:

From 'What is the difference between SRE and DevOps?':

  ... DevOps is about maintaining code ...

From (an answer to) 'How do I hire a good DevOps, fitting my company?':

  ... It is a common misbelief that “DevOps” is a role in a company. The word merely denotes an organisational shift in software companies ...

From 'Does my organization need adopt Agile Soft. Dev. before adopting DevOps?':

  ... there are still organisations which are not practicing Agile in their software development, who might be interested in adopting DevOps ...

DevOps is NOT a role

Below are a few quotes from some of the questions on DevOps.SE, which all seem to illustrate that DevOps is NOT a role:

From (an answer to) 'What is the difference between Sysadmin and DevOps Engineer?':

  ... DevOps is not a role (when used as such it's more a buzzword than a real role).

From (an answer to) 'How do I hire a good DevOps, fitting my company?':

  ... It is a common misbelief that “DevOps” is a role in a company. The word merely denotes an organisational shift in software companies ...


Combining CD and GitFlow Firstly, I apologize if the title is not descriptive enough or does not describe my problem well enough. I'm struggling to figure out how to describe my issue in brief! It really seems like this should be a problem that has been solved already, but I'm struggling to find the answer. I might be missing something glaringly obvious!!

So - we currently have a Git workflow on GitLab using 3 core branches (master, preprod and develop). We branch features off from develop, then back into develop for initial testing. Merging into develop triggers a pipeline that tests, builds and deploys to a dev server, so that Project Owners can review code in progress. Once signed off, the develop branch gets merged to preprod, which also triggers a pipeline to deploy to our UAT environment for... UAT. Once signed off, preprop gets merged to master which has a manually triggered pipeline to deploy to live.

Bunch of issues with this approach, but the biggest is the bottle neck. If we have a feature in preprod that we don't want to release yet, but another feature is coming up behind it, it's tricky to bypass the feature we don't yet want. So we are considering moving to a different branching strategy such as GitFlow.

Now on to the bit I can't figure out! Currently, in order to deploy a feature to an environment where it can be viewed and tested, we have pipelines triggered by a merge to specific branches. However, I would like to do away with this approach when testing features and hotfixes. Essentially, I need a way to be able to run a pipeline that will run automated tests, build and deploy any branch at any time. So if I am working on FeatureA, I can push that branch to GitLab and deploy it somewhere so that it can be reviewed in isolation from FeatureB that is being worked on by a colleague. Does that make sense? Almost like a dynamic pipeline with a dynamic environment that spins up when needed and gets destroyed again when no longer needed.

If anyone can point me in the right direction here, it would be appreciated!

Should also add that we are starting to use docker on our local machines for development, so a solution that uses docker for spinning up dynamic test environments is not out of the question.

Thanks in advance...

Conflating deployment environments and branches often leads to problems like you describe.  It does seem like a natural fit though and easily maps to mental model of an ideal solution but, real life is anything but ideal.

Basically, git is not a deployment tool, one should not clone to deploy a version but fetch a package (a tarball, zip file, docker container, NPM package etc).

In practice these two concepts are orthogonal.  Branching strategies (whichever it may be, gitflow, trunk based etc) is a means to assist developers isolating and sharing their work.  It ties naturally to the versioning strategy and is what forms the CI half of your CI/CD system.  Here you must force yourself to keep CI (continuous integration) separate from CD (continuous deployment) as you will see if you read-on.

Deployment however consumes the outcome of versions that were created and integrates various parts of the system and makes a whole that is usable, testable.

So if you view your branches and the code therein as a means to assemble a version which the build system will build and package and then store it somewhere.  Can be as simple as a network share or FTP site or it could tie in to an existing package manager with a appropriate back end, whatever suits your team best really.  This choice will depends greatly on the technology stack you are using and the underlying platforms you will be deploying to (static environments, docker, kubernetes, standalone copy-and-run etc).

Then another set of tasks can pull in any of these versions and assemble a working deployment.  Here this works best if your deployment environment are dynamic but it can be made to work with static environments as well.  This will form the CD part of your continuous system.

Once decoupled you can spin up an environment that would contain any versions that has been previously built.

In term you could even have multiple production environments with slight variations of a user interface tweaks and pit them against each-other with techniques like A/B testing and clever load balancing / distribution to end-users.

Going back to your particular situation it will simplify your gitflow back to a more traditional master/develop permanent branches and a set of features/hotfix and release branches.

Using semver you can keep official versions strictly for master merges (from release branches or hotfix branches) and pre-release versions for all other branches.  for your pre-release branches simply add a -SOMETHING where something is the name of the branch for example and possibly a timestamp to differentiate successive deployments.

for example,

release branches would get the x.y.0-RCi (i th release candidate). x.y is the version that will be released where the -RC is simply dropped when merged in master.  0 as the patch because release branches are used for new features and product evolution
hotfix branch follow the same principle as the release branch except the patch number is set to a non-zero value.
development branch could get x.y.z-dev.timestamp
feature branch x.y.z-JiraID-timestamp.  Where JiraID is the ticket number of the system you use to track new features (like Jira, Bugzilla, etc)

I propose semantic versioning here because I am familiar with it but the same broad principle of segregating branches through versioning can be applied with pretty much any versioning scheme.

Nowhere in the source control do you see anything that relates to deployment environment.  Only your versioning strategy is what links them, they are otherwise completely isolated and independent.

Your devs will feel more comfortable in this new environment and your deployments will feel less constrained and awkward.

Hope this helps


On Premise CI / CD recommendations for Docker images We have TFS building Docker images, and loading them into a Private Registry on one network, our Development Network. Everything works well, and we are generally happy with the CI process in that environment. We have an Air-gapped Integration Network that we need to export these Docker images to, and then have something that finds them, puts them into a registry there, and then restarts a Docker swarm.

I am looking for a solution that can:

Watch a folder for new files
When a new Docker Tar file is found, load it into Docker Registry.
Move the Docker image that was loaded to an "archive" folder
Restart Docker swarm to use new images in the Registry

It seems like step 2 might not be possible directly, I might need to load it into a local Docker instance, tag it, and then push it, but if so, that's fine.

Is this something I should Jenkins for? I could just write a script in Bash or Python, but I would like to use a proper CI tool, if I can.

Jenkins Option

If you're comfortable with Jenkins and want to use a CI tool, it is a decent option.

It has multiple plugins for watching for files arriving in a file system, and at the end of the day, you can even just wrap your bash scripts into it so you have a central GUI and recorded logs.  So, you're not limited in any way by it.

Inotify Option

I personally might just have a CI tool like Jenkins or GitLab-CI deploy a bash script though.  Then you could use inotify to properly watch a folder and execute your script exactly when things arrive.  See: https://www.howtogeek.com/405468/how-to-perform-a-task-when-a-new-file-is-added-to-a-directory-in-linux/ for an example.

Why iNotify?

In many tools, I believe you generally have to "poll", which means run over and over just to see if you have anything to do. Inotify lets you genuinely "watch" and be pushed events only when things happen.  This is cleaner and more efficient as you won't have hundreds or thousands of tasks running a day that literally do nothing.

Monitoring

Note: One plus side to the pure Jenkins approachis built in monitoring.  Every  job run that fails can easily email/alert you.   You can pull this off with the iNotify script as well, but  if you don't have the right tooling for monitoring/etc in your environment, you may favor the Jenkins approach.


How can I use a local file for a nested Cloudformation stack? How and is it possible to use a local file for a CloudFormation stack besides an S3 bucket?

I'm afraid you can't. The nested stack files must be in S3 as far as I know. You have a couple of options:

Have some automation in place (e.g. aws-cli + shell script, or better ansible) that when executed will upload all the templates to a designated S3 bucket and then execute the the CloudFormation deployment.
Use some templating engine like Ansible + Jinja or similar that can build a big CFN template from little snippets saved in separate files. With Jinja you can also use loops, lookup variables, etc, to build your final template file. Then again you can use the same Ansible playbook to deploy it. Or do it manually as it will only be a single file.

Hope that helps :)


Using docker-compose, what's the recommended way to handle different sets of containers for dev and prod? I'm creating a webapp with a python backend, Nginx as the server, and Vue as my frontend web framework.  When deploying to production, I build an nginx container that, on build, compiles all my frontend code into minified static files it can serve (takes ~2minutes), and on run, serves those static files and routes non-static-file HTTP requests to a different API Container (running the python app).  (So two containers: nginx, and api server).

The problem I'm facing is that during development, I want to use a third container to run a hot-reloading webpack-dev-server for the frontend files, instead of having to wait two whole minutes for webpack to rebuild the frontend bundle every time I make a change. For reference, building the frontend code into servable static files takes around two minutes, whereas the webpack-dev-server can reflect changes in <3 seconds)

My idea is to have a production nginx config that handles the production case of serving static content directly from the nginx container, and a dev nginx config that handles the dev case of routing those static requests to the webpack-dev-server container.  But having different docker-compose files for different environments seems to go against docker philosophy.  Am I approaching this incorrectly?

docker-compose.override.yml, docker-compose.prod.yml and extend

An outline answer would be to have a docker-compose.yml file for your common base configuration, a docker-compose.prod.yml for production-specific sets of containers and for other specific environments, docker-compose.override.yml. It is possible to use the docker-compose extend keyword/command/statement in a docker compose yml file to inherit from another docker compose yml file.

The above approach is on the basis that one would first consider only the things common across all environments as the pure, base setup and then dev and prod environments would be based off of this (inherited).

Your dev-specific packer setup could be configured in the docker-compose.override.yml file, which inherits from the base docker-compose.yml. Configuration that is common across all environments would reside in docker-compose.yml. Your production setup could be in docker-compose.prod.yml, which again can inherit from the base docker-compose.yml.

References

This seems to be the best practice approach, based on the following references:

The inheritance/nesting approach is explored here: Inheritance or
nesting with docker compose
The official documentation here - Share Compose configurations
between files and projects - describes the docker compose
extend keyword.

Some things you may or may not need to consider for your specific case

There may or may not be some bespoke config code required in
addition i.e. in your deployment itself, e.g. bash scripts etc., should the above approach provide most of what you need but not quite everything. These scripts can be run from docker compose.
Research what can be overridden when writing docker-compose.override.yml
when "inheriting" from docker-compose.yml. Your particular project may require a configuration in each of the dev and prod environments that override the same thing in the base environment, thus the need to override what is in the "parent"/base docker-compose.yml environment. Consider this issue: https://github.com/docker/compose/issues/3729 - which suggests there are limitations as to what can be overridden, particularly if you want to move a configuration item that was present in the parent docker-compose.yml. A better way could be to only have items common across all environments in docker-compose.yml and have the specifics in files for specific enviroments, such as docker-compose.prod.yml


Increase concurrent users by limiting container resources with docker-swarm Consider the scenario where I have:

4 replicas of an app
4 cores
On load, the app consumes 100% for a single core

Can I tell each replica to only use 50% of the core and if so, would this allow at most 8 simultaneous connections to the app? I have just dipped my toe into this stuff so an feedback would be appreciated.

In general: No you can't. At least not on the Docker level.
You can maybe code your app in a way that it monitors such metrics and adjusts itself but Docker isn't able to.
What Docker CAN do is limit the app to 50% and kill it if it uses more than that by utilizing resource limits:
docs.docker.com - containers/resource_constraints
docs.docker.com - docker-compose resource limits


Scope for one click deploy So one of the goals of a software company is to have one click deploy.  I think this is a major responsibility of the dev ops team.  Whether or not it is should be reserved for another conversation.  My question is that our company has multiple individual artifacts that make up a release.  Should the developers be able to one click deploy all these artifacts at once, or should each artifact be deployed individually?

The answer is: it depends.

Before I answer, I need to provide definitions for a couple of terms to prevent ambiguity.

1) Deploy is defined as the act of changing executables in an environment. This might mean that new source is copied over, and some process is restarted. Or a new version of a docker container is placed into an orchestrator - doesn't really matter.

2) Release is defined as the act of making new functionality available to users of the system.

Do note that while in most organizations release and deploy happen together, they definitely don't have to be.

  Every released change (usually) requires new code to be deployed. But not every deployment of code means new changes were released to be used.

Let us first look at the best case possible. Where you don't have any button that requires pushing, to get a deployment to happen. Every change sent to your version control system - automatically gets tested for correctness and is automatically deployed to your system.

Now your question is optionally several different questions:

Do I have a way to release a new feature and is that way manual? (button)
Do I release ALL available (multiple) features at the same time when that button is pushed?
Do my feature releases require changes to happen in multiple decoupled parts of the system?

Since releasing new functionality requires some business approval. For example, the UX and correctness of the new functionality must actually bring value to your customers. It makes sense to have a "dark" release that only some friendly customers can see and provide feedback for. Then when these customers are happy with the value they get, your business can decide to make this new functionality generally available (GA).

Should this feature break the system when released, it is possible to turn off the switch that made it released in the first place. All of which does not require any new code (or old code) to be deployed to the system, because the code is already there.

When you consider that deploy and release are separate, it makes the decision to deploy everything together or separately simpler.

The manufacturing world and Toyota Production System specifically have taught us that having small batches is much more beneficial than batching work items together. TPS even goes as far as to strive for single-piece-flow, where each work item is delivered to the next work center alone and not in a batch.

Agile methodologies have taken a cue from TPS and also describe the benefits of having small-batches. These would be described as a "sprint" with a definite deployed deliverable at the end and striving to keep the sprints as short as possible. This allows to run short experiments and receive feedback regarding the value of the changes created as soon as it is possible to do something about them.

Continuous Delivery is taking this even further by saying that any finished code, should be deployed to your systems as soon as it was auto-tested. And a release can happen at any time when someone decided to do so. And experiments can be validated as part of the development cycle. Moving the definition of "done" for a developer away from "I finished writing my code" and towards "I received feedback from customers that they received value". The shorter the feedback cycle is - minutes, seconds, etc... the better. And batching is a deterrent to short feedback.

By having a button that by definition batches things together, would mean that risk of problems is higher. Learning from TPS, Agile and CD means that single-piece is far superior and has many benefits.

The answer "it depends" is correct because all of what I wrote above highly depends on the maturity of your organisation. The amount of automation you have. The amount of communication you have. And many other factors.

Separating release from deployment requires continuous integration and deployment. As well as a system that enable to select which features to release. For example https://github.com/intuit/wasabi or https://www.infoq.com/presentations/etsy-deploy


How to make an AWS S3 bucket accessible over HTTP? Does AWS S3 provide a way to make bucket's files available over HTTP preferably protected with BASIC_AUTH? I mean listing the files (index) and make it possible to download them over HTTP.

Download - yes. Simply set each Object ACL to be publicly readable.

List - sort of. If you set the Bucket ACL to publicly readable anyone can list the content. However it's a rather human-unfriendly XML listing.
You can generate your directory listings to some index.html files through an external process and configure the bucket to serve those.

Basic auth - nope. You can look at something like S3 pre-signed URLs to protect the content.

Hope that helps :)


Which AMI was my EC2 instance created from? Is there any way to tell which AMI my EC2 instance was created from?

Is there any way to tell which AMI my EC2 instance was created from?

Short answer: yes.

Longer answer: yes - how would you like to get it?

The AMI which the EC2 instance was launched with is part of the EC2 instance metadata

Every instance publishes its metadata, which you can retrieve via the HTTP interface:

curl http://<instance.ip.address>/latest/meta-data/ami-id

(lifted directly from the AWS docs)

Many people prefer the Python library for the AWS API (boto3). In this particular case you would need to use the instance.image_id method.


Ansible - Variable expansion inside hostvars declaration? I have a hardcoded hostvars line, like so:
node1_hostname={{ hostvars['192.168.0.162']['node1_hostname'] }}
I have a set_fact variable from a previous play that I'd like to expand in the host section of the hostvars declaration, instead of the IP address. Can hostvars take expansion inside those brackets and single quotes?
Thanks.

No, you cannot nest jinja2 expansion i.e. the following won't work:
# Warning ! Those will fail !
some_var: "{{ some_dict[{{ some_dynamic_key }}] }}"
other_var: "{{ {{ dynamicaly_prefixed }}_var }}"
# Did I forget to say the above examples will fail ?

But you can use your var without any problem, just as in almost any other programming language dealing with list and indexes. There is actually an ansible FAQ entry on this particular subject
some_inventory_hostname: 192.168.0.162
node1_hostname: "{{ hostvars[some_inventory_hostname].node1_hostname }}"


How to fail stack when EC2 spot is refused? I'm using AWS Cloudformation to create EC2 spot instance (with block duration). In order to do this in cloudformation I'm using a Launch Template where spot configuration is set. Then my cloudformation template create an EC2 resource which use the Launch Template.

It work fine except when AWS refuse to create a spot instance. In that case, the cloudformation stack is still running.

I'm wondering if there is a way to fail the stack when the spot request is refused ?

Thanks

You should use TimeoutDurationMinutes in your code while requesting for spot instances via cloudformation.

For more details: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-instancefleetconfig-spotprovisioningspecification.html


How to see when an AWS IAM role was last used? In AWS in the GUI you can select a role and then click on delete, then when you are prompted to approve or deny the deletion it shows you when the role was last used.

I would like to be able to know when a role was last used without pretend deleting it via the GUI.Ideally if I could get that data from the AWS CLI or a Boto3 script that would be great. Is there a way to accomplish that?

Amazon CloudTrail tracks all API use

In CloudTrail create a new trail

You can configure each trail to send log events to CloudWatch: Edit the trail and choose to send Logs to CloudWatch. It will offer you to create a Role for this

https://docs.aws.amazon.com/awscloudtrail/latest/userguide/monitor-cloudtrail-log-files-with-cloudwatch-logs.html

Then, run some calls on your role and wait 5 minutes.

In Cloudwatch, go to your log trail and search for "AssumedRole". Your events will be shown here

There is also documentation on searching Cloudwatch logs with filters through the api https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SearchDataFilterPattern.html


Why does Docker hub allow non-official Docker images and why do they exist? Note: also posted in Security StackExchange as there is a security aspect to this question (sorry for cross-posting).

I'm evaluating Docker images for my development and production setups.

When I research how to do something in Docker, quite often I find answers that include the author's own Docker image. For me, this is not suitable as this hinders the transparency of the solution and uncertainty of reliance on future maintenance by that author. Both of these potential issues have security implications.

While the Docker Hub, can provide a convenient centralised and well-known outlet for publishing one's own Docker images, derived from officials, what would be the advantage of using these unofficial images over the originals.

I am aware of reverse engineering tools, as described in answers on Stack Overflow and that at least some of the unofficial Docker images have an associated GitHub repo that apparently shows how the Docker image was built. But this is an inconvenient level of "indirection" or friction to being really sure (if at all possible) as to the contents of the Docker image.

As with most elements of Information Security, there is a balancing act between usability and security; an increase in usability often results in a correlated decrease in security and visa versa.

As a thought experiment...

  I can make any server 100% secure by wiping the hard drives, putting it through a shredder and then melting down all of the components parts.  It also makes it 0% usable.

Docker Hub is no different, the desire is to make these images accessible for the common good, at the cost of traceability and provenance of the images.  Fortunately, there are several tools such as JFrog Artifactory and Sonatype Nexus Lifecycle which enable you to apply governance to any Open Source repository (i.e. Docker Hub, Nuget, NPM, etc).

One aspect that I think Docker Hub gets wrong, that other sites get right, namely NPM, is presenting clear information about what Source Repository an image is generated from, that I suspect is due to the decoupled nature of Docker Images from the source code that generates them.


Why does development oppose operations? I'm still a student, but I'm not knowledgeable about operations, and my English is still bad.

My question is: Why does development oppose operations? When does developing oppose operations?

The point of DevOps, is that development shouldn't oppose operations, instead they should support each other.

Traditionally, due to waterfall deployments and large scale updates, development would cause operations a variety of problems when deploying due to inadequate testing, changing server environments, the list goes on and on. Essentially, the updates were too large for the operations team to be able to effectively deploy them without some problems arising in the process. These problems might be why you believe that development opposes operations.

On the other hand, DevOps works to reduce update size, decrease rigid environments, and generally improve the handoff of the application between development and operations by increasing the amount of times the handoff occurs each year. With the increased number of deployments comes less headaches for operations, because they have either automated a large amount of work required to update the products, or they better anticipate and prepare for the updates.

Tldr: DevOps aims to nullify the theory that development opposes operations by creating a mindset where operations and development work together to frequently deploy products in a timely and easily reproducible way.


difference between packer and matchbox I have this setup to standup a baremetal machine:

Terraform-> Matchbox
         \-> Chef-> Docker

My colleague has a setup to standup VMs in AWS:

Terraform-> Packer-> Chef-> Docker

Is Packer to AWS as Matchbox is to Baremetal, in terms of image provisioning?  I know there not 1:1 in features since packer can provision chef and stuff (it seems, and so can terraform) but for the general purpose of standing up a machine can this be considered.

If not what is the difference between the two.  I am having trouble understanding exactly what packer is for (compared to matchbox), if it doesn't fit that relation.

You may use matchbox on a VM as you do on a baremetal machine, you won't be able to use packer on a baremetal machine ont he other hand as it doesn't handle any PXE boot option.

That said, leveraging vSphere/AWS/ API/cli to create a new machine from a template is usually quicker and more effective than using the API/cli to create the VM and then make it boot in PXE.

Packer will create machines, matchbox won't, it will allow you to match a profile with a machine but it won't rack your machine nor plug the networks and power cables, it won't do the machine creation neither for a VM, that not its goal.

So the main difference is the "before provisioning (chef)" step, as the machine creation, Packer will handle the machine creation whereas matchbox won't, matchbox will handle booting a blank machine and push the OS where packer usually won't.


How to build windows image on azure dev ops hosted agent The host agent pool on azure-devops to build windows container, Hosted Windows Container is running Windows Server version 1803.

But I want to deploy on Windows Server 2019, so instead of using Microsoft :1803 images, I'm building from images tagged :ltsc2019.

How can I build my :ltsc2019 images on azure-devops?

EDIT
Microsoft added a new hosted agent: Hosted Windows 2019 with VS 2019.
So you should use this image to build for windows-1809 or windows-2019
Source : https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml#use-a-microsoft-hosted-agent

Old answer:
FWIW, I ended up creating a windows server 2019 vm on azure to act as a build agent.
Could not manage to build 2019 image in any of the hosted agent pools.


Using a ubuntu base image vs an application specific image (node, python, etc) What are the benefits of using an application specific Docker image vs a generic OS? I'll use an angular app here as a contextual example, but this should apply to other apps as well.

Example 1 | Use application specific images:

FROM node as build
RUN npm install
COPY package-lock.json
RUN ng build --prod

FROM nginx
# ... move files from build to nginx/html
EXPOSE port

Example 2 | Use a linux base image:

FROM ubuntu
RUN apt-get install npm
RUN apt-get install nginx
COPY . .
RUN npm install
# ... build Angular app and mv files to nginx/html

Example 3 | Build angular app outside of docker first and:

FROM nginx
COPY . .
# mv files to nginx/html
EXPOSE port

What are the advantages / disadvantages to doing it one of these 3 ways? Are there image size differences? Does the orchestration environment (kubernetes) need access to the base images or is everything needed included when it's pushed? Which way is considered best practice?

They're all functionally equivalent.  When you docker push or docker pull an image (or mention it in a Kubernetes manifest), all of its layers are pushed or pulled, so you always get a complete image with all of its dependencies at the point where it was built.  (It turns out the versioned ubuntu images are re-released pretty routinely to include current security patches.)

Building your application outside of Docker was pretty common before there were multi-stage builds.  There's nothing wrong with doing it, if it makes sense.  It makes more sense if your built application artifact is portable (in your example, Webpack JavaScript/HTML/CSS artifacts; Java .jar files work well here too).  This is also pretty easy to plug into an existing CI system that produces the artifacts, and then you just need to wrap them up in an image.

Using prebuilt images like nginx or node, or building your own on top of ubuntu or alpine, is mostly a matter of taste and organizational requirements.  Building an image FROM node will be easier than building your own Node image and can get you running faster; if you start FROM ubuntu then you know exactly who built it and what security updates and packages are in it.

If you're pinning to specific versions of software, the other important thing to be aware of is that the Docker Hub packages eventually stop getting updates.  If you're using node:8 that image isn't being automatically built any more; if you pin to a more specific version, node:12.13 or node:12.14.0 aren't getting updates either.  (As of this writing the Docker Hub node page lists 12.14.1 as the current LTS build.)  That could be an issue if you're forced to take a language runtime update to get an OS-level security fix.


Separate Jenkins config from build logs My JENKINS_HOME is (something like) /var/lib/jenkins

In there, there is jobs directory, in it a subdirectory for each job. In each job subdirectory, you have config.xml plus the builds subdirectory.

I would like to somehow separate my config.xml from the same file structure that contains builds so that I can check them in my version control but to not clutter its view by having all the build logs that each execution writes. If I change a job config, the corresponding config.xml is changed and I can (actually in a Jenkins job) check it in my version control automatically or if I change it manually in a text editor, it gets checked out and deployed. But I don't want the builds in the version management.

Is there a way to separate just the job configs from the build logs in the filesystem?

If you're interested in only backing up configs, try installing the SCM Sync Configuration Plugin. This plugin syncs with a repo each time you change either a job or the Jenkins configs, but does not keep builds.

SCM Sync Configuration Jenkins/Hudson plugin is aimed at 2 main features :
Keep sync'ed your config.xml (and other ressources) jenkins/hudson files with a SCM repository
Track changes (and author) made on every file with commit messages

As a bonus you can set it to take a commit message each time you change a job, for easier restoration if necessary.


Error when creating new recource using backend AWS s3 When I try create a new resource using terraform apply (Terraform init was success) received an error:

data.terraform_remote_state.environment_data: Refreshing state...

Error: Unable to find remote state

  on backend.tf line 15, in data "terraform_remote_state" "project_data":
  15: data "terraform_remote_state" "project_data" {

No stored state was found for the given workspace in the given backend.

Error: Unable to find remote state

  on backend.tf line 25, in data "terraform_remote_state" "environment_data":
  25: data "terraform_remote_state" "environment_data" {

No stored state was found for the given workspace in the given backend.

And here is my point, it's a new resource the Terraform should be create the new tfstate in using my backend configuration in this case AWS S3.

I have this same recipe in use to other resources running...

Anyone can clarify me?

Guys I discovered what happened.
It was necessary to apply these two resources that I informed (project_data and environment_data), but they were in the above directories and I was not aware.

Something like this:

Application
|
|-Environment
  |-environment_data.tf
|
|- project_data.tf


How efficiently does GlusterFS deal with failures? What happens when one of the storage bricks goes down (HDD failure), how does GlusterFS deal with data recover? Is the stored data still safe?

Gluster has built in data "translators" that automatically replicate data across all of your bricks. The particular type of translator you're interested in is called AFR for automatic file replication. The AFR translator also uses the DHT (distributed hash table) translator. It's important that you have at least two master bricks since if you only have one, you have a single point of failure. As long as you have at least two bricks, an auto-healing process is triggered by GlusterFS's daemon (it's automatically installed when you invoke gluster the first time) using the other master server's replicated/translated data if anything goes wrong. Actual disaster recovery that the auto healing can't fix requires going through a careful process documented here.

Other wisdom on the topic (for example, Redhat's documentation) recommends having at least six bricks in two sets, actually. This way, "even if we lose two bricks from each set, there is no data loss". Basically, the more bricks you have, the more redundancy you have, and the less chance of data loss you have (however, it'll become cost-prohibitive and time-prohibitive after a while).


Should developer wait for CI pipeline to complete or start next task after pushing My company is integrating CI/CD, so far we have implemented CI from what I understand. Currently when a developer pushes code to our git repo, the CI pipeline runs.

Currently our CI pipeline include building the project and the doing static code analysis to make sure it meets our coding standards. We will be implementing testing next. The build and static code analysis take about 3mins right now. From what I have read fixing issues right away is critical to CI/CD. I expect when we add in unit tests that the pipeline could take around 10mins to run.

So my question is when a developer make a pull/merge request should they wait for the CI pipeline to complete or just move on to the next task and come back to fix pipeline issues if they exists? Or should they sit and watch the pipeline run?

Sit and watch the pipeline run?

No, that is not how you work efficiently.

Developers push their commits to the source control repository and then the CI/CD pipeline is triggered.

Developers may post a well-written pull request anytime they want. There is usually a visual mark representing an "on-going build"/"failed build"/"successful build".

Typically a brach can be merged whenever all these criteria is fulfilled:

at least one approver / or as many that is required to approve has approved.
a "successful build"
no unresolved merge conflicts

If any of these criteria not is fulfilled, they need to be fixed, but the developer does it whenever he has time or priority to do this task.


What are the advantages of using AWS nameservers as opposed to servers provided by other registrars? I am transfering a domain name from namecheap to aws route 53 service. AWS provides you with three options when transfering:

Continue to use the name servers provided by the current registrar or
DNS service.
Import name servers from a Route 53 hosted zone that has
the same name as the domain.
Specify new name servers to replace the
current registrar's name servers (not recommended)

what is the advantage of using option 2 over the others. What is the difference in pros and cons between the options?

There are two parts to owning a domain. The registration, and the hosting of the DNS entries. As you are transferring the domain registration, AWS needs to know whose service do you want to use for managing your DNS host names and other settings.
As a sensible default, select option (1) as this will keep everything managed with the current registrar / hosting company and so nothing will break (at this point in time). When your hosting expires with this company, its quite likely they will drop all of your DNS settings as well, in which case you will need to have migrated somewhere else before this point (options 2 and 3)
However, if you are using AWS services you really should be managing the DNS settings in AWS and create a Route 53 hosted zone. This provides better integration with the AWS environment, and allows you to use some of the "AWS aware" aspects of Route53 for your infrastructure.  Assuming that you initially selected option 1, you should then set up a parallel zone in Route53, verify all of the DNS records match (except NS settings for the domain) and then update your Domain registration to use option 2. Changing this setting can take 72 hours to roll across the internet. In my experience, usually 90% of the internet knows in under 6 hours.
Option 3 means you (or someone else) is managing the Domain DNS master server for your domain. There may be situations where this makes sense, but they are rare.


ansible error: skipping: no hosts matched I've tried a few variations on running ansible locally as a test case:
nicholas $
nicholas $ ls
ansible.cfg  ansible.cfg.orig  first_playbook.yml  inventory.txt  playbook.yml
nicholas $
nicholas $ cat ansible.cfg
[defaults]
transport = local

nicholas $
nicholas $ cat playbook.yml
---

- name: Network Getting Started First Playbook
  connection: network_cli
  gather_facts: false
  hosts: all
  tasks:

    - name: Get config for VyOS devices
      vyos_facts:
        gather_subset: all

    - name: Display the config
      debug:
        msg: "The hostname is {{ ansible_net_hostname }} and the OS is {{ ansible_net_version }}"

nicholas $
nicholas $ ansible --version
ansible 2.9.6
  config file = /home/nicholas/ansible/ansible.cfg
  configured module search path = ['/home/nicholas/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.8.2 (default, Jul 16 2020, 14:00:26) [GCC 9.3.0]
nicholas $
nicholas $ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04 LTS
Release:    20.04
Codename:   focal
nicholas $
nicholas $ ansible-playbook --connection=local  playbook.yml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [Network Getting Started First Playbook] **************************************************************************************
skipping: no hosts matched

PLAY RECAP *************************************************************************************************************************

nicholas $

perhaps I need an additional configuration for the localhost loopback?
Although I can ping localhost as:
nicholas $
nicholas $ ansible localhost -m ping -u root
127.0.0.1 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}
nicholas $

The answer is in the warning you get:
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

When you use hosts: all in your playbook, localhost is not matched.
If you want to run the playbook on localhost, you can do one of the following:

Change your playbook to hosts: localhost.
Explicitly provide an inventory file that has localhost.

You can create an inventory file - inventory.yml with the contents:
all:
  hosts:
    localhost

and then pass this inventory file to ansible explicitly like this:
$ ansible-playbook --connection=local -i inventory.yml playbook.yml
You can find more information about inventory in the Ansible documentation.


Git Files Encryption for Business I am using git and have the following problem:
In the company where I work you now want to protect the source code from theft.
So if for some reason a developer lets the source code of a project go along and sells it, for example.
To prevent this, I thought about it for a long time and came to the conclusion that encrypting the source code is the best choice.
Like git-crypt does, that's a very good idea.
Only with one mistake:
If I add a user's GPG key, he can decrypt all files, and the protection measure would be worthless.
The files of the developer should be decrypted locally and only remotely. (This way the developer does not always have to encrypt, push, decrypt the files)

I tried to re-program git-crypt in Python (because of cross-platform) but then failed due to the git attributes/smudge, clean filter drivers.

Do you know a possibility or an application that takes up exactly my problem?

There really is no way to take on your problem with git, which is to deny a developer with access to the code (required to work with it) the possibility to steal the code. This is because as soon as your developer has the code in a readable format (to program on it), he can do with it whatever he wants, anyways.

git-crypt has a totally different use, and is not related to your problem at all.

To solve your problem, there is basically only one possibility: lock the network in which the code resides down completely - make it so that there is no physical way to transfer the code to the outside. This could mean:

No direct internet connection, obviously.
No http/https proxy either, no mail clients etc.
No way to add new devices to the network (unless through official ways). E.g., no unused network ports to plug anything in; no guest DHCP; MAC address filtering; no WIFI, smartphones; etc.
No way to plug an USB device or other transfer mechanism into any devices.

This may sound harsh, but really is not. It could be as simple as keeping your code strictly within a set of VMs, accessed by something like Vmware Horizon or something like that, with cut&paste and file transfer disabled, of course. Be nice to your developers and allow them to transfer stuff into the environment (their favourite editor, for example).

The measures so far do not require any physical changes, i.e. your developers still can work remotely or in their usual unsecure office. Obviously, they still can go to extreme measures (screenshots on their client PC...) to get specific parts of the code page-by-page.

To go a step further, in increasing severity (although we are going into James Bond territory here...):

Strict policy with severe punishment. Setup a specific contract which each emplyee which contains clauses as harsh as the laws of your country allow.
Switching access to the limited network to a physical area in your office.
Security measures like locked doors accessible to only specific personnel, video cameras with 24/7 human supervision and so on.
2-factor access (either 2 persons required to checkout code, or at least a physical TAN generator or something along those lines).

Note that in your case you want to make it hard for a thief who knows that he is doing something illegal and already has committed to doing a crime. He will not respect any "weak" protection (like rules/laws) anyways.


How to inject credentials to python virtual env in jenkins? I am trying to inject my credentials
to a Jenkins Freestyle project.

The project is building a python project in virtual environment
with "behave" created with ShiningPanda plugin

can someone help me ?
thanks

In order to pass masked credentials to your Jenkins job, do the following:

In Jenkins, install Credentials Binding plugin.
In your Jenkins, go to Credentials and click the global domain.
In the left pane, click "Add credential".
From the "Kind" dropdown, choose "Username with password".
Enter the username and password in the reflective boxes and choose an ID (you will use this ID when you want to access the credentials) and click OK.
Then, in your job, under "Build environment", check "Use secret text(s) or file(s)" and it will add a "Binding" section and an "Add" button, click the "Add" button and "username and password separated", choose a variable name to hold the username and another to hold the password and choose the credentials you created earlier from the credentials dropdown.
Then, in the "Build" section of the job, you can access the credentials using your chosen username and password variables.

I wrote an article about this subject in the past so feel free to check it for more detailed explanation and screen shots.


Fluentd unable to read logs from all namespaces I have a cluster installed on Openshift and used the Cluster Logging Operator and Elasticsearch Operator to enable logging on the cluster.
On Kibana, I am only able to see the logs from certain namespaces and not all the namespaces I have. How do I configure Fluentd to read logs from all pods in all namespaces?
I do not have a Fluentd service account (It was not created with the installation of the operator).
Also, the namespaces from which I don't see any logs have a node selector mentioned in them. Could that be a reason why the logs are not available?

Million dollar question - are you running your fluentD as a DaemonSet?  If not, there's a chance that fluentd simply isn't running on the node where some containers live.
Otherwise, my best guess is, it's missing permissions to read from all namespaces, especially since you mentioned you haven't attached a ServiceAccount to it.  If you used a public FluentD Helm chart such as this one, try making sure you're deploying it with rbac enabled, i.e.:
rbac:
  create: true

serviceAccount:
  create: true

It'll also help if you post some logs from your fluentd container.


Configuration Management Tools and Rollbacks / Approaching Immutable Infrastructure I am interested in whether there are any configuration management tools out there that make it easy to roll back.

So far it seems that with Ansible, Chef, and Puppet you have to write the roll back tasks/recipes/whatever to get back to your previous state.

Coming from a bit of personal interest and experience with NixOS I have seen that with pure Immutable Infrastructure rollback is easy.

I am in Enterprise environments where NixOS is not happening. How can Enterprises begin to approach Rollbacks better with tools that work with Windows and Enterprise Linux Distros?

You hint in the title about immutable infrastructure, so it sounds like you already know the solution:  don't change existing servers, but bring up new ones with your changes, switch over to them, and switch back to the old ones if necessary.

Theoretically any of the standard config management tools can roll back by simply checking out a previous commit and rerunning the config.  However, practically speaking this often isn't the case, as you may have introduced a new config rule (and so whatever it is doing is simply ignored in older versions) or you've written a rule that isn't easily reversible (for instance, does some destructive data operation).  If you were very careful in how you always write your playbooks, you could avoid this, but a few less error-prone solution is just to use an immutable infrastructure approach.


How can I check if there is a new build the same jenkins pipeline I have a github multibranch pipeline, in declarative format. It is configured to trigger on pushes to pull request branches (ie, branches which are included in pull requests to a target branch, say main.
If a new commit comes in before the execution of a previous commit is finished, this will trigger a concurrent build of the pipeline. I want to maintain this behaviour, but need to know if there is a new pipeline running in one of the stages.
Ie I want a groovy script step like this pseudo code:
if (build_number + 1) is_currently_running
  do one thing
else
  do something else

Is there any way to ask Jenkins if build n+1 is running?

This should be fairly straightforward to do in a script block.  I don't think there's a declarative-native way to do this.
I use this script block to cancel builds for earlier (i.e. outdated) revisions of PRs:
script {
  // currentBuild is a Pipeline-builtin variable for accessing the Build object
  // https://www.jenkins.io/doc/book/pipeline/getting-started/#global-variable-reference
  def build = currentBuild

  while(true) {
    build = build.getPreviousBuild()

    // exit the loop if we have finished iterating over builds
    // (groovy doesn't have a do...while feature)
    if (build == null) break

    def rawBuild = build.rawBuild

    if (rawBuild.isBuilding()) {
      // abort the build
      rawBuild.doTerm()
    }
  }
}

As you can hopefully see, this iterates over all previous builds and uses the isBuilding() method to find ones that are currently running.
If you want to check later builds instead of previous ones, you can use getNextBuild() instead (I believe the API doc is here, it's been a while since I wrote this so this might be the wrong class).
And if you want to check only one directly adjacent build instead of all earlier/later builds, you can just omit the loop and only call getPreviousBuild()/getNextBuild() once.


How trigger a build by a commit in a branch prefix? On the Azure DevOps, in we git we have a schema of branches where we create a branch per sprint, for sample:

sprint01
sprint02
sprint03
...

We keep the 3 last sprint branches, and in this case when I create the sprint4 branch, we will drop the sprint01. Each sprint  We publish a version of our product every week. On the build pipeline definition on our project, we have defined the trigger to run over a push on the sprint* branch because we don't want to change the definition on each new version.

It works but it runs a build for each sprint branch and a release for each one. I would like to have just for that branch (with prefix sprint) that was pushed on the git. Is there any way to configure it?

My current configuration:

It's recommended that you use a convention of prefixing your grouped references (like branches) with "folder" prefixes.

For example, when you create sprint branches, create them with the names:

sprints/sprint01
sprints/sprint02
sprints/sprint03

Then, configure your build trigger Include filters with this prefix:

sprints/*

... and then an Exclude filter for master, since by default it is also included.

Once configured, this should build only the sprint branch (topic branch) that was committed to.

Reference: https://docs.microsoft.com/en-us/azure/devops/pipelines/build/ci-build-git?view=azure-devops&tabs=designer


How do we encrypt credentials in Jenkins console output I can see credentials in Jenkins console output.  How can we encrypt them? The "Mask Passwords plugin" is considered as not safe in my case.

Here's an example of using them in a pipeline safely.  In this case, they are injected as environment variables and their value should never have to be shown unless you explicitly choose to print out the variable content for some reason or other.

Environment variables are generally seen as the best way to store secrets at this point and are heavily relied on by many modern deployments (e.g. often used in kubernetes apps).

https://www.jenkins.io/doc/book/pipeline/jenkinsfile/#handling-credentials

Jenkinsfile (Declarative Pipeline)
pipeline {
    agent {
        // Define agent details here
    }
    environment {
        AWS_ACCESS_KEY_ID     = credentials('jenkins-aws-secret-key-id')
        AWS_SECRET_ACCESS_KEY = credentials('jenkins-aws-secret-access-key')
    }
    stages {
        stage('Example stage 1') {
            steps {
                //
            }
        }
        stage('Example stage 2') {
            steps {
                //
            }
        }
    }
}

You can also avoid Jenkins altogether by having your pipeline/etc retrieve them from some other tool like HashiCorp Vault, AWS SSM, Azure Vault, etc.


Getting a single revision from Git Having the full Git revision history has a lot of benefits as part of the development process.

But our product is the source code, we are using scripted languages that don't need compilation or processing, and then the Git history becomes a burden on deployment- in our example we deploy a clean virtual environment following every change, having several deployments on a single machine.

There are some ways to reduce the amount of history, for example shallow clones whose efficiency depends on how deep the revision is in the branch, doing a fetch instead of clone but then you still get history from the revision and back, or getting the full repo once then pull when needed but this is wasteful in terms of disk space and tends to be less reliable.

Is there a way to get a single revision from Git without it's history ?

Shallow clone

You can indeed get a shallow clone out of Git using:

git clone --depth=1 <url>

This will still clone the repo and create a .git folder with the objects, only smaller in size (difference depending on your total file size vs. history size).

Git archive

You can also use git-archive to extract an archive of the repo:

  Creates an archive of the specified format containing the tree structure for the named tree, and writes it out to the standard output. If  is specified it is prepended to the filenames in the archive.

In the examples it shows for instance:

  git archive --format=tar --prefix=git-1.4.0/ v1.4.0 | gzip >git-1.4.0.tar.gz

    Create a compressed tarball for v1.4.0 release.

Hosted Git, archive API

If you are hosting your repo on GitHub, then you can use their archive API:

https://api.github.com/repos/<username>/<repository>/zipball/<commit_hash>

Bitbucket.org has a same functionality for this:

https://bitbucket.org/<username>/<repository>/get/<branch_name|commit_hash|tag>.zip


What percentage of microservice architectures employ polyglot programming models? One of the tenets of microservices is the polyglot, autonomous services (developed in different programming languages). In my experience (online reading and YouTube and discussions with developers) it seems this is not really applied in practice except the big companies like NetFlix, AWS , Google. Will this assumption be true ? I  will appreciate information on statistics about this e.g. reports, surveys.

A good source for statistics could be stackshare.io.

In general, polyglot services i.e. teams delivering different stacks could create silos, and additional overhead for diverse toolchains; nevertheless I consider that larger organizations have the polyglot situation organically because this type of thing is not always an issue of top-management.

Arguments against polyglot approach:
"Why You Should Think Twice Before Even Considering Polyglot Microservices"

Worth to mention is the polyglot toolkit Eclipse vert.x (though I am not sure how is this different to say "it has bindings to many languages").