Using Docker images as vehicles for binary deployment artifacts Let's say we're in an environment where we have a modern CI/CD pipeline for a number of containerized apps. We also have a number legacy apps which haven't been containerized yet.
I'm wondering how sound/unusual/crazy the following idea is: even for apps which don't run in containers in production, use container images and a container registry as a vehicle to deploy binary build artifacts.
Here's a very simple example to illustrate the idea: let's say we have a .NET app which consists of two projects. One that rarely changes and contains a large number of static assets (ChangesRarely), and another small, lightweight one which changes often (ChangesOften).
Now, let's say we're using the following Dockerfile to build the app in a multi-stage build:
FROM mcr.microsoft.com/dotnet/sdk:3.1 AS builder
ADD src src
RUN dotnet publish -o buildOutput/ChangesRarely src/ChangesRarely
RUN dotnet publish -o buildOutput/ChangesOften src/ChangesOften

FROM alpine
COPY --from=builder buildOutput/ChangesRarely /buildOutput/ChangesRarely
COPY --from=builder buildOutput/ChangesOften /buildOutput/ChangesOften

Note that because of the two separate COPY instructions, the resulting image will contain two separate layers. This will be important in just a second.
When our CI/CD build system (e.g. GitHub Actions) needs to create a new build, it'll basically run docker build -t myapp . and push the resulting image to a container registry. So far, pretty standard.
Now let's say we want to deploy a new version to production. Again, "production" isn't some sort of container orchestrator (yet) - assume it's a set of plain old servers in a data center. So here's where the approach might become a tad unusual:
Each production server runs a Docker engine- but only to pull the container image and subsequently copy the build artifacts out of it, deploying them to their final destination (some local folder):
id=$(docker create myapp)                 # pulls image from container registry, if necessary
docker cp $id:/buildOutput - | tar zxf -  # copy build artifacts
docker rm -v $id                          # destroy temp container

I've confirmed that this really works. But, why would you do that? Three main reasons:

The container registry acts as a unified store for all build artifacts (for containerized and non-containerized apps). No need to manage two different artifact stores.

Image layers which are identical across builds (such as the large ChangesRarely layer in the example above) only need to be stored once in the container registry, preserving storage space

They also will be cached by the production server's Docker engine, preserving network bandwidth and reducing deployment times.

How usual or unusual is this approach?
Any problems with the approach that I might be missing?

I've seen it done before but wouldn't call it common. There are other artifact servers, and many git repositories have the concept of a release with artifacts that makes more sense.
I suspect Docker Hub is one of many artifact servers to introduce rate limits and other usage caps, so if you do this, make sure you control (or pay for) the registry server. Otherwise the risk is you'll exceed limits and lose access to your artifacts.
If you only want to copy binaries, then I'd skip the base image and use FROM scratch.
It's also possible to bypass the docker engine entirely and pull the artifacts from the registry server directly to the filesystem. I have a regclient project that ships a regctl command with regctl image manifest and regctl layer pull commands. E.g.:
$ regctl image manifest regclient/regctl
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
  "config": {
    "mediaType": "application/vnd.docker.container.image.v1+json",
    "size": 3019,
    "digest": "sha256:e3abac7c7ce78aaa5efddb5abedd54f8ed3077d9d5b03b5a66ecf9f4c3502454"
  },
  "layers": [
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 930,
      "digest": "sha256:41437cd6a6e3d3098329474c62ccc5bd01fcf6b41bf35f0e8c89db96110b23ac"
    },
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 122413,
      "digest": "sha256:69d6afda22ccf4f223c016bfd3a108fe0a1ddd15c72978bd238b19201e0e1973"
    },
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 161,
      "digest": "sha256:57dd9736a25ab0b55d9d364dc20ee0946a844a8ac72d7f986ef902826f933e89"
    },
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 3963283,
      "digest": "sha256:282c5a3b4b6c8bf0b7038204919bbfc26c8b120cd87f4a1f573c12c808774c1c"
    }
  ]
}

$ regctl layer pull regclient/regctl sha256:282c5a3b4b6c8bf0b7038204919bbfc26c8b120cd87f4a1f573c12c808774c1c | tar -tzvf -
-rwxr-xr-x root/root  10235904 2020-11-05 20:52 regctl

Note that the sha's are unique and persistent, so if that hash is unchanged there's no need to pull the binary again. And the above example has a few extra layers since the command is expected to run in the image as a non-root user (/etc/passwd, ca-certificates, and a home directory). For a non-running image, that could be a single layer. E.g.:
# change image to your scratch-based source
image=regclient/regctl
store=/tmp
sha=$(regctl image manifest $image --format '{{ (index .Layers 0).Digest}}')
if [ ! -d "$store/$sha" ]; then
  mkdir -p "$store/$sha"
  regctl layer pull $image $sha | tar -xzvf - -C "$store/$sha"
fi
echo "result in $store/$sha"


Grafana Status Panel Plugin - genereate status box for each entry from a Prometheus querry I am trying to create a dashboad out of process_uptime_seconds{namespace="$namespace"} querry from Spring Actuator. and Prometheus as datasource I am trying to build a single stat box menus that will show the status, but for  each value from the request. So instead of singlestat I have tried using Status Panel. However I keep getting a single box generated. How can I create dynamically a separate box for each entry from the returned object?

So far I have upgraded from 6.6 to 7 and switched to the new Stat panel that managed to visualize properly. The limitation I was trying to avoid is that I can't  control fonts.


Where do you store your configurations? Dockerfiles, gitlab-ci, application.properties, deploy scripts Let´s suppose that our repository contents is something that will be shared. Is strange that we include the deploy scripts there, explaining how to deploy in our machines

If by "Shared" you mean that your team has access to the files then it should be stored in Source Control so that everyone on the team is using the same scripts.  You can consider a secrets manager so that team members do not share sensitive information (e.g., Docker Secrets, AWS Secrets Manager, Azure Key Vault).

If this will be publicly available you can still use a secrets manager to publish these files with the sensitive data removed.


AWS RDS- Determine what Database Tables aren't interacted with? Is it possible to have AWS-RDS (or another aws service) generate a report on databases that haven't been interacted with (write/read) after a certain period of time?

My background is in QA and I'm relatively new to the DevOps game, sorry if this is a basic question, couldn't find it in search.

I don't think any database services "internal traffic" is available to AWS services directly.

That being said, you should be able to dump all sql queries to a log file on S3 and also extract all database/table combinations to a text file as well.

It then becomes a text processing exercise that you can possibly solve with AWS Athena


Where/what is the `auto-devops` script in gitlab's "auto devops" staging stage? https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml
staging:
  extends: .auto-deploy
  stage: staging
  script:
    - auto-deploy check_kube_domain
    - auto-deploy download_chart
    - auto-deploy ensure_namespace
    - auto-deploy initialize_tiller
    - auto-deploy create_secret
    - auto-deploy deploy
  environment:
    name: staging
    url: http://$CI_PROJECT_PATH_SLUG-staging.$KUBE_INGRESS_BASE_DOMAIN
  rules:
    - if: '$CI_KUBERNETES_ACTIVE == null || $CI_KUBERNETES_ACTIVE == ""'
      when: never
    - if: '$CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH'
      when: never
    - if: '$STAGING_ENABLED'

What does auto-deploy refer to? Is it a script? Where is it located? Where can I find out more info about it (because the staging stage is currently failing, and I'd like to debug what it's doing)?
The failing job logs:
To get the password for "user" run:
    export POSTGRES_PASSWORD=$(kubectl get secret --namespace initial-24739737-staging staging-postgresql -o jsonpath="{.data.postgresql-password}" | base64 --decode)
To connect to your database run the following command:
    kubectl run staging-postgresql-client --rm --tty -i --restart='Never' --namespace initial-24739737-staging --image docker.io/bitnami/postgresql:9.6.16 --env="PGPASSWORD=$POSTGRES_PASSWORD" --command -- psql --host staging-postgresql -U user -d staging -p 5432
To connect to your database from outside the cluster execute the following commands:
    kubectl port-forward --namespace initial-24739737-staging svc/staging-postgresql 5432:5432 &
    PGPASSWORD="$POSTGRES_PASSWORD" psql --host 127.0.0.1 -U user -d staging -p 5432
WARNING: Rolling tag detected (bitnami/postgresql:9.6.16), please note that it is strongly recommended to avoid using rolling tags in a production environment.
+info https://docs.bitnami.com/containers/how-to/understand-rolling-tags-containers/
Validating chart version...
Fetching the previously deployed chart version...
Fetching the deploying chart version... v1.0.7
secret "staging-secret" deleted
secret/staging-secret replaced
No helm values file found at '.gitlab/auto-deploy-values.yaml'
Deploying new stable release...
Release "staging" does not exist. Installing it now.
INSTALL FAILED
PURGING CHART
Error: release staging failed: timed out waiting for the condition
Successfully purged a chart!
Error: release staging failed: timed out waiting for the condition
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 1


Simple service to check infrastructure status I have a very simple setup here (solo IT guy): we have a on-prem server which runs some services. They should produce e.g. a daily logfile. I'd like to get a notification if this doesn't happen for some reason. I make a script, activated from cron, that checks this and sends me an email if it doesn't happen. However, this obviously doesn't work if e.g. the entire server goes down, my email script has a bug, etc, so I'd like to have the converse: have a daily check and get an email if it did NOT ran. I do have this with www.duplicati-monitoring.com: I get an email even if my on-prem server is down. I'd like to extend this to more infrastructure tests.
How is such a system called? Are there websites (preferrably free / open source) offering this? I looked at eg nagios, but it seems far too big, and it seems that I need to host it myself.

This is conventionally called a watchdog, deadman, or health check.  Bruce's comment is good, including the part about a raspberry pi.   You could also spin up something simple like monit in a container someplace like GCE or AWS.
Other than setting up an external service or server, there are also a bunch of phone apps that will do the pinging and/or http fetch and HTML parsing for you, right from your phone, providing you with a notification and/or audio alert when a condition isn't met. Since you're a one-man shop, using your own phone isn't substantially worse in terms of being a single point of failure, though you'll want to have a migration plan for when and if you do grow.
The thing I would probably do myself is keep the more complex monit configs on-site, summarize those on a simpler monit config from a stable IP at GCE, with a simple config on my (dynamic IP) phone watching the GCE monit.  That lets firewall rules keep things relatively private other than that summary query from the phone, and when you add a new team member they just need to add that phone app with the one query.  If you're comfortable with leaving the firewall rules more open, then you could skip the GCE step and just have your phone hit a summary statistic on your own server directly.
Back in the depths of time I used to use SMTP mail for this sort of thing a lot; these days I'd prefer the phone app for a more timely notification.


I trying on Windows 10, Trying to connect / login to work interactively on container but I'm getting this kind of error I trying on Windows 10, Trying to connect / login to work interactively on container but I'm getting this kind of error

docker run --rm -p 8080:80 laravel-docker
 Error response from daemon: driver failed programming external connectivity on endpoint wizardly_diffie (0c95bf862e583179e0a988eae0b07606e41184548c494ddd523df95c1430422f): Error starting userland proxy: mkdir /port/tcp:0.0.0.0:8080:tcp:172.17.0.8:80: input/output error.

How to resolve this.

my docker version

$ docker --version
Docker version 18.09.0, build 4d60db4

Restart docker, it falls into this state after the OS reboots.

The below thread mentions running it a swarm as preventing the issue from reoccurring and several ways to resolve the underlying OS issue with the software that may, or may not be workable for you.

https://github.com/docker/for-win/issues/1038


Custom Layer 4 Load Balancer What are my best options for deploying a load balancer on AWS with the following requirments:

Add auto scaling that will start my own EC2 instances.
    Deploy 3rd party SW on the load balancer.
    Have a readable shared memory (can be S3).

Found the tutorial "Create an External Load Balancer" about Kubernetes but not sure about it.

EDIT:

deploy 3rd party [software] on the load balancer" -- to do what: Want to start new instances with parameters and for routing rules to the instances that were created with the auto scaling.

how is "readable shared memory" used, and for what purpose? shared memory between the load balancer and the instances. I can do it with sockets.

Why is the balancer layer 4 as opposed to a different layer? Need TCP parametrs, layer 7 info can also be useful

Where is TLS handled? I havn't thought about this.

Historically, F5's Application Delivery Controller has been the enterprise-grade industry solution for doing this. With this solution you can:

Use iRules LX to communicate with other software to auto-scale on an as-needed basis
Load balance at Layer 4 (to achieve better throughput/acceleration) or Layer 7
Share memory between Virtual Servers (there may be a better way using iRules LX instead of just iRules)
Bridge or offload/terminate TLS on the Load Balancer
Have vendor support and consulting if needed
Do all of this in Amazon's cloud  (or anyone else's; you can also run it in-house)
Scale to millions more connections than Kubernetes, nginx, or any other load balancer solution that doesn't use a microkernel is capable of.


Terraform 0.12 - For each subnet-id in array, get cidr_block I have a list of subnets IDs and I want to get an array of the cidr_blocks associated to each subnet.

variable "aws_subnet_ids" {
  type = list(string)
  default = ["subnet-aaaaaaaa","subnet-bbbbbbbb"]
}

desired_output = ['10.0.1.0/24', '10.0.2.0/24']

I've been fiddling around with for loops and count data for a quite a while for what should be a really simple problem. But I'm having a hard time finding an appropriate way to actually do this.

This seemed like a decent example to work off of, but you can't actually use a for_each in a data source off a list of strings. https://www.terraform.io/docs/providers/aws/d/subnet_ids.html

data "aws_subnet" "example" {
  for_each = var.aws_subnet_ids
  cidr_blocks       = each.cidr_block
}

output "subnet_cidr_blocks" {
  value = [for s in data.aws_subnet.example : s.cidr_block]
}

data "aws_subnet" "subnets" {
  for_each = toset(var.aws_subnet_ids)
  id       = each.value
}

locals {
  cidr_blocks = [
    for subnet in data.aws_subnet.subnets :
      subnet.cidr_block
    ]

  cidr_blocks_rendered = [
    for cidr_block in local.cidr_blocks :
     {
       from_port   = 6443,
       to_port     = 6443,
       protocol    = "tcp",
       cidr_blocks = cidr_block
     }
  ]
}

output "output1-1" {
  value = local.cidr_blocks_rendered
}

This is what ultimately I needed.


Is it possible to consolidate notification handlers when they are very similar? I have an Ansible role which uses handlers to restart services via the systemd module.

- name: restart ntp
  systemd:
    name: ntp
    enabled: yes
    state: restarted
  become: true

- name: restart web server
  systemd:
    name: nginx
    enabled: yes
    state: restarted
  become: true

- name: restart grafana
  systemd:
    name: grafana
    enabled: yes
    state: restarted
  become: true

Is there a way so I could parameterize the three separate handlers into something like this?:

- name: restart {{svcname}}
  systemd:
    name: '{{svcname}}'
    enabled: yes
    state: restarted
  become: true

In ansible 2.7 and earlier versions, variables in handlers name are not populated.
This bug was reported and a correction was lately pushed to devel version.

You need to run ansible from devel, and hope they publish this fix in the next release.


How to probe all running aws ec2 instances and their sec groups, etc to generate templates? Typically I spin up all aws ec2 instances together with all its accompanying security groups, volumes, etc ... by executing a series of aws cli commands.

However when I manually use the aws console in a browser to spin up instances, I am wondering if there is a tool I can run to exhaustively probe this set of running ec2 instances and output runable aws commands to recreate the currently running instances?

Does such a tool exist ?

I have never looked at Terraform, so I wonder if that has such a utility?

Terraform would be the best resource to create these EC2 instances repeatedly.

I would highly recommend you to create a variables.tf or terraform.tfvars file template allowing you to create a new instance of these resources and store that file and the state file securely away for future management of the resources. A versioned S3 bucket would do quite well for this.

Please note that once you create your Terraform code you can instruct Terraform which state file(s) to work with, etc. Allowing you to reuse the same code over and over without duplication.


Github plugin triggering to push event only for changes in specific branches I am using Jenkins2 and GitHub plugin for triggering Push events.

I have the next service (Jenkins Github): http://myjenkinsmaster/github-webhook/

I also have 3 branches: dev, stage, master which need to be built.

But I require only the branch with changes to be built.

For example if changes were pushed to dev, only dev branch will be built, and others ignored.

However, Jenkins builds all 3 branches which I specified in branch field in job config.

How can I achieve a Jenkins build that builds only the branch where changes were pushed?

Finally I have resolved it with the help of "strategy of choosing what to build" with "Default behavior" for each branch.


Rollup status from task to story to feature in Azure Devops? Are there any extensions that let you rollup issue status across a hierarchy?

For example, if I have a story where all it's tasks go to Resolved, then the story is set automatically to Resolved?

Likewise, if I have a story where at least one of the tasks go to Active, the story is set to Active, then the story is set to Active?

Ideally, it would do this across a hierarchy, from task, to story, to feature, etc.

Currently, I don't see any extensions that do this.

You may try TFS Aggregator (Web Service). If you or your team have experience with C#, you may try the console project that I`ve developed: Automation of state changing.


Problem with Connecting Jenkins to SonarQube I have a Jenkins pipeline that periodically pull from gitlab and build different repos, build a multi-component platform, run and test it. Now I installed a sonarqube server on the same machine (Ubuntu 18.04) and I want to connect my Jenkins to sonarqube.

In Jenkins:

I set up the sonarqube scanner at Global Tool Configuration as below:

I generated a token in sonarqube and in Jenkins at configuration I set up the server as below BUT I couldn't find any place to insert the token (and I think this is the problem):

In the jenkins pipeline this is how I added a stage for sonarqube:

stage('SonarQube analysis') {
    steps{
        script {
            scannerHome = tool 'SonarQube';
        }
        withSonarQubeEnv('SonarQube') {
            sh "${scannerHome}/bin/sonar-scanner"
        }
    }
}

But this fails with below logs and ERROR: script returned exit code 127:

[Pipeline] { (SonarQube analysis)
[Pipeline] script
[Pipeline] {
[Pipeline] tool
Invalid tool ID
[Pipeline] }
[Pipeline] // script
[Pipeline] withSonarQubeEnv
Injecting SonarQube environment variables using the configuration: SonarQube
[Pipeline] {
[Pipeline] sh
+ /var/lib/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/SonarQube/bin/sonar-scanner
/var/lib/jenkins/workspace/wws-full-test@tmp/durable-2c68acd1/script.sh: 1: /var/lib/jenkins/workspace/wws-full-test@tmp/durable-2c68acd1/script.sh: /var/lib/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/SonarQube/bin/sonar-scanner: not found
[Pipeline] }
WARN: Unable to locate 'report-task.txt' in the workspace. Did the SonarScanner succeeded?
[Pipeline] // withSonarQubeEnv
[Pipeline] }
[Pipeline] // stage

And when I check my jenkinstools on the disk sonnar plugin is not there:

$ ls /var/lib/jenkins/tools/
jenkins.plugins.nodejs.tools.NodeJSInstallation

Can someone please let me know how I can connect Jenkins to sonarqube?

Create and add token to be able to connect to SonarQube.
You have create project in SonarQube and use it as a parameter:

sh """
   ${scannerHome}/bin/sonar-scanner \
   -Dsonar.projectKey=your_project_key_created_in_sonarqube_as_project \
   -Dsonar.sources=. \
"""

Copied from [StackOverflow] Problem with Connecting Jenkins to SonarQube.


Where to install Prometheus for Kubernetes monitoring I'm considering using Prometheus to monitor and analyze different metrics from my AWS EKS Kubernetes cluster running in Fargate, as it seems to be the standard tool during these times. However, something that really impressed me was that installing Prometheus itself in the very same k8s cluster (as another running set of pods) is a normal thing. Am I right or just got a wrong idea from my research? Considering good practices, shouldn't the monitoring tool be outside of the system that you actually intend to monitor? Also, if I set up multiple Kubernetes clusters, isn't it more cumbersome to have one Prometheus installation per cluster? Is it possible to set up a single Prometheus installation and centralize all k8s clusters metrics on it?
Same question for Grafana's installation... Should it be installed inside or outside the cluster?
On a different topic, Prometheus history retention period is set to 15 days by default (storage.tsdb.retention.time). Even though cluster metrics insight should be useful for a short period of time so you can troubleshoot issues, I wonder if there may be scenarios in which a longer history period might help. Any comments on this matter?
Thanks!

This is more of an opinion based question but let me try to answer it.

installing Prometheus itself in the very same k8s cluster (as another running set of pods) is a normal thing.Am I right or just got a wrong idea from my research?

Considering good practices, shouldn't the monitoring tool be outside of the system that you actually intend to monitor?

This is a correct practice to have. Monitoring systems are usually setup to look into the object being monitored. (outside looking in). But with self healing systems like k8s, installing monitoring within the system can be done with minimum issues. There are a few cases you do need to look into for proper stability.

Pods get deleted :- Run Prometheus in a HA mode (2 replicas minimum)
so as not to lose any data.
Pods are replaced due to resource crunch:- Add priority classes
to prevent pod form being evicted.
Use Persistent volumes:- Look into volume node affinity
conflict. Also configure the data retention period to make
optimised use of storage.
Cluster goes down. See below.

Also, if I set up multiple Kubernetes clusters, isn't it more cumbersome to have one Prometheus installation per cluster?

Not at all, using a combination of IaC(Terraform/Chef),Config management(Ansible),CI/CD (Spinnaker),GitOps and Helm charts. Configuring/maintaining  Prometheus instances across multiple instances will not be an issue.

Is it possible to set up a single Prometheus installation and centralise all k8s clusters metrics on it?

Prometheus although highly optimised, will not be able to handle all the metrics thrown at it from multiple clusters.Also Prometheus does not scale horizontally on its own, therefore it would need huge resources. Centralising all metrics to one place will cause other issues as well.

Network Connectivity/Latency:- Running a centralised Prometheus
instance receiving data from over the public internet may cause data
loss/false alerts due to network drops and latency.
Cost:- Running tens of exporter all sending data over the internet
will cause high internet costs for cloud deployments.
Cluster managing:- You may face issues with exporter/job
configuration of such a large number of exporters.

Its a good choice to run Prometheus per cluster. wherein you can have all the data locally till the time is required.
Grafana does not consume much resources so its safe to run inside the cluster. However it faces a different set of issues:-

User/Permission Management :- Each local instance would need to be
maintained separately which is a huge task.
Charts Management:- Same as above.
L1/DevOps will have to keep n number of tabs open to monitor your
infra.

Other than this, setting up alerts on all the clusters separately would also be a never-ending task.
My advice would be to setup Prometheus clusters locally (inside the cluster or right next to it) and connect all of them via Thanos or Cortex
and set up a HA mode Grafana centrally on top of Thanos along with alerts and external storage.This will alert if the local Prometheus instance goes down and will help with the outside looking in factor

scenarios in which a longer history period might help. Any comments on this matter?

Depends upon the metrics being stored, A quarterly SLI/SLO and incident reports seem like a good use case to store data for longer than 15 days.


Ansible - stop when host is success, exit with IP Basically, my use case is that I want to run on list of hosts (group from inventory).

It will run one by one, until it finds machine answer my standards,
when it find the one with free RAM as I say e.g 40GB free,
stop looping host, continue the next role/task with the IP address of this machine.

The thing that makes this complicated is that ansible registers variables and facts locally, on the target remote host - and then has a different set of variables and facts once it moves to the next host. So it would require a bit of fiddling to do this exactly the way you want.

One way to approach this would be to just use run_once, but it doesn't work with serial: 1 which you are asking for (check one host at a time);

---
  - name: test
    hosts: all
    vars:
      mem_to_look_for: 62950
    tasks:
      - name: do things
        debug:
          msg: "hello"
        run_once: true
        when: ansible_memory_mb.real.total==mem_to_look_for

If you really want it to run serially, we could borrow some of the knowledge out of this article https://stackoverflow.com/questions/42376123/how-to-terminate-an-ansible-playbook-run-when-it-is-successful-on-one-host and tweak it a little.. and it works great!

---
  - name: "test"
    hosts: all
    gather_facts: no
    serial: 1
    vars:
      mem_to_look_for: 62950
    tasks:
      - meta: end_play
        when: "{% for item in hostvars.values() %}{% if item.done|default(False) %}True{% endif %}{% endfor %}"

      - name: gather facts
        setup:

      - name: ensure host has the memory we're looking for
        set_fact:
          done: true
        when: ansible_memory_mb.real.total==mem_to_look_for

      - name: skip this host if it doesn't have the memory we're looking for
        meta: end_play
        when: ansible_memory_mb.real.total!=mem_to_look_for

      - name: run this thing only on one host
        debug:
          msg: "running the thing"

I set the playbook to gather_facts: no because we don't want to continue gathering facts once we found the host we're looking for. It doesn't abort the playbook unfortunately, but just kind of runs a no-op on the remainder of hosts. You could also probably speed things up by tweaking the setup: module by specifying which subset of facts you want to collect, but I left it to collect all the facts in this example. https://docs.ansible.com/ansible/latest/modules/setup_module.html you may also want to just move the host selection to the setup module by adding failed_when: ansible_memory_mb.real.total!=mem_to_look_for at the end of the setup module, but then the hosts would be marked as failed. I'm not sure if that's what you're looking for, but it's really up to you on if you want them to be failed or just skipped.

Additionally, if you want it to completely stop checking hosts as soon as its done I think the any_errors_fatal property will do that. https://docs.ansible.com/ansible/latest/user_guide/playbooks_error_handling.html#aborting-the-play


'find' command suddenly not working in windows hosted agents In the YAML file defining our ADO build pipeline we have the following section

- bash: |
    pip install pylint
    find preprocessing/ subtyping/ visualisations/ -name '*.py' | grep -v hmm | grep -v /test_ | sort | xargs pylint
  displayName: 'Run linter'

At some point in the last week this stopped working. It still works on an ubuntu-latest hosted agent, but on Windows (either windows-latest or vs2017-win2016) it now fails with the message

  FIND: Parameter format not correct

We have not changed anything, but our build process now fails.

What has changed to cause this failure?
Why does it only fail on Windows?
How might I diagnose the problem?
Is there anything wrong with our call to bash's 'find' command?

Find is a command in Windows that does very different thing to find in a posix shell.

And that would be the error you would get if you ran the Windows find (usually Windows find is located at C:\Windows\System32\find.exe


Tying playbooks together Say, i have written bunch of ansible roles and playbooks, configured  my playbooks structure and now i'm trying to tie them to one playbook which will do its thing depending on tags or inventory file provided. Should i strive for that? Or is it better to have bunch of playbooks and call them depending on the tasks/hosts needed? What is the best way to go about this? I can't seem to find any example of tying it all together in one playbook or bunch of them which will do its thing to ones infrastructure depending on arguments provided.

Best practice in ansible docs covers organization of playbook structure but not organization of playbook use on infrastructure. Is there any examples of organizing playbooks use on infrastructure?

What I have done before goes on the same direction as Zeitounator commented; you can have one or more "meta" or master playbooks that import the ones you need to run your entire automation. To add to Zaito's comment; you can use the when clause on import_playbook or include to control the execution of those "inner" playbooks; something like this:
Say you want to execute PlatformB playbooks only for an stg or prod environment using a defined env ansible variable:
- include: platformA/do_something_on_platformA.yml
- include: platformB/do_something_on_platformB.yml
  when: env == 'stg' or env == 'prod'

you could change the value of env using --extra-vars on execution time or something.
hope it helps, cheers.


Azure monitor and deployment slots My question is actually quite simple: if I have a webapp with a deployment slot and a monitor that checks the health of webapp in some way (connections, cpu time or the like) , if I do a swap, what is the monitor watching? The new webapp or the old?

I would like to think it is always watching the current production slot, but I can't find it in the docs.

Deployment slots share the same resources. The percentage of CPU used is what is taken by all slots combined.
After a swap, it should still monitor the production slot.


Count of currently active jenkins agents Is there an easy way to get a count of how many agents are currently up? I know I can go to www.myjenkins.com/computer/ to see the agents, but counting them manually is tedious.

Is there an API call or something I can run in the script console that just outputs the count for me?

You can sufix computer/ with api/json and run some script on this response. Count the computers array or group by some one of his attributes, like idle or offline.

https://www.myjenkins.com/computer/api/json


is a secrets-provider-container a good idea? I am looking for a way to provide secrets to a container running on Google Cloud Run responding to the following criteria:

secure
centralized & pulled based
free
simple
working with a team, with various trust scopes
host agnostic: changing 1 config will be the same on aws, gcp, azure

When starting a new project with containers, I always come to the point where I need to manage secrets, and I often find it difficult.  My current solution is to store my secrets encrypted on a public git repository, and decrypt them inside the container, in the entry point script.
But I don't like so much installing gpg along with Django and it's dependencies. I am now thinking of running another container on the same network, which sole responsibility will be serving the config to the first container.
Is it secure ? Is there a better way ?

There is probably a better way if you are working with Google Cloud Run.  I have not tried this but I see that they have "Secret Manager" that you can use.  They also mention a project Berglas that can be used to store secrets in GCS cloud storage.
In my personal experience, it find it best to leverage widely adopted solutions and avoid creating a unique solution for security.  Getting it right is non trivial and it is less code you have to support.
This FAQ has doc links to the solutions I mentioned.  https://github.com/ahmetb/cloud-run-faq#how-to-configure-secrets-for-cloud-run-applications


AWS ECS/EKS: EC2 vs Fargate mode Amazon offers the option of launching their container management services ECS and EKS in two modes: EC2 and Fargate.

In the former, normal VMs (EC2 instances) are used and ECS or EKS (depending on the selection) clusters are created.

In the later, there is no concept of vm/instance present; you just have to deal with the containers and plan their capacity so that the system knows what memory/cpu needs you have.

Given that Fargate apparently accounts for a (far?) more cost-efficient solution given that you are charged only for the resources' consumption of your containers, I would like to know what are the benefits (or example use cases) where anyone would like to go with the EC2 - based solution.

Update 12/05/2020: Fargate prices have been greatly reduced over the last months and are now comparable with EC2 prices for the same CPU / RAM configurations. That makes Fargate a valid option for 24x7 workloads.

Originally written in 2018 - no longer correct!

Fargate is more expensive than EC2 for the same vCPU/RAM amount.

For example:

m5.large (2 vCPU, 8 GB RAM) costs $0.096/hr which is ca $69.12/month
Fargate container with 2vCPU and 8 GB RAM costs $0.2028/hr or $146.02/month

If you want to run your container 24x7 you'll be much better off running it on an EC2 instance.

However if your containers only run briefly to complete a task and then exit, or if they scale up and down based on demand it will be much easier for you to run them in Fargate - you won't need to scale up and down the underlying EC2 cluster to support the load.

With Fargate you pay premium for the flexibility.

In many cases it works out better to run on Fargate even if it's more expensive - we spin up batches of hundreds of containers at a time a couple times per day for some processing and each container runs for only about 10 minutes. If we had to scale up the EC2/ECS cluster before each run, wait for it to settle, deal with the failures, then run our batch job and then scale down again the overhead would be quite high and our batch processing would take much longer.

Here Fargate works great for us. I wouldn't use it for an always-on service though.

Hope that helps :)


Update ECS task definition with new image from ECR I have an ECS cluster/service/task definition using an image from my ECR repository.

Let's say I update the image with a new version, will ECS pick the changes automatically?

Do I need to modify the task definition?
Do I need to somehow reload/restart the task(s)?

My goal is to automate the deployment to ECS. I'm not sure if using CodeBuild to build and push the new image to ECR is enough, or if I need something else (codepipeline?) to deploy changes to ECS.

Note: task definition is using "latest" image tag, new images update that tag in ECR.

Options are:

Use CodeBuild to push the new image. Make sure the task definition is using the "latest" tag. You'll need to force the deployment of the task definition to pick the new image.
Use CodePipeline to update ECS. It will automatically generate a new task definition revision with the new image and deploy it.


Cost-efficient hosting for CI/CD? I'm paying over $300/month for Kubernetes to have CI/CD in GitLab. And it only covers web apps, Android apps and deployment to Google Cloud. Plus some staging environments, that do not require many resources and that I'm using rarely. It doesn't cover CD to Apple App Store though.

I'm curious if there is a way to optimize this? Maybe I should buy some hardware or optimize Google Cloud somehow?

Given that you mentioned rarely using some k8s resources, you could explore some alternatives:

With gcloud SDK you could create scripts to be executed for the periods you want to reduce GKE nodes (even to zero) during periods you're not using it
With Google Cloud Functions (GCF) you can create some python/Node.js/Go script to access GKE APIs and also execute the operations to stop/reduce your instances. These GCF could be triggered by schedulers configured by Cloud Scheduler. The cost here for such executions will be really low or even zero.


How to properly manage mupltiple environmental variables with Gitlab CI and Docker Swarm? In our project we are using Docker and we have multiple services with a pile of environmental variables (mostly credentials to various dbs, mail, samba, 30+ entries in total). It all worked well with docker_compose.yml and docker-compose up -d --build. The advantage of this method was an extensive usage of env files. For example, I had mupltiple env files and I could use them in docker-compose.yml:

env_file:
  - ./envs/common.env
  - ./envs/mail.env
  - ./envs/mysql.env

We set up a Gitlab CI and are planning to use Docker Swarm, so no more manual creation of production env files, use CI/CD. And the primary way in Gitlab is to create variables one by one.  We could create file type variable and put the contents of env file in it, but it will be hard to manage those inner-variables.

So how do I have to work with env vars now?

Create a ton of variables in Gitlab UI and aggregate them in env files?
Use file type variables?
Pass them to services in docker-compose.yml file like this:

environment:

VAR1
VAR2
...
VAR20

Or should we also use Docker secrets or some configuration management tool?

There is stages in gitlab ci and on 'docker build' stage I don't need ALL env variables. Usually its only a REACT_APP_SERVER_URI variable.
Later I use env file in compose.yml like you are.

env_file:
  - ./envs/common.env


How to Integrate Amazon API Gateway with EKS using Application Load Balancer (ALB)? According to the AWS documentation, it is possible to integrate Amazon API Gateway with EKS using ALB. However, by far most of this documentation explains how to do this with NLB instead of ALB. So hopefully, someone can explain how to do this with ALB.
For example, do I need to create ingress to enable me calling my services? Or is there another way?  How to configure my ALB and my API gateway?

From the same page:
The steps to integrate ALB and NLB with API Gateway are identical.

Which follows by a sentence with a link to a documentation for an ALB ingress, including prerequisites for VPC subnets and EKS resources.
Yes, you have to create one unless you'd want to configure ALB manually instead of using Ingress manifests inside k8s.


Terraform conditionally create VPC I am in the process of creating a terraform template, part of which can create a VPC.

The user has the option to specify the parameter create-vpc. This will be passed to the module https://github.com/terraform-aws-modules/terraform-aws-vpc which will create the VPC based on this parameter i.e.

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  create_vpc = var.create-vpc
  # ... omitted
}

I am now trying to use a data source to share the VPC with the rest of my template i.e.

data "aws_vpc" "myvpc" {
  id = "${module.vpc.vpc_id}"
}

How can I update this to use the variable vpc-id instead of the module output if the VPC hasn't been created?

Ideally I'd do something like:

data "aws_vpc" "myvpc" {
  count = "${var.vpc-id != "" ? 1 : 0}"
  id = "${module.vpc.vpc_id}"
}

data "aws_vpc" "myvpc" {
  count = "${var.vpc-id != "" ? 0 : 1}"
  id = var.vpc-id
}

This won't work as resource names need to be unique.

Other resources in the template reference this VPC.

The way I'm understanding your question is that you're writing a module that takes an optional vpc_id as an argument, and either uses it if specified or creates a new VPC itself if not specified. You then need to use the VPC id elsewhere in your configuration, regardless of whether it was provided by the caller directly or if it was created by the VPC module.

If I understood that correctly, here's one way to get that done.

First, we'll declare the vpc_id variable for your own module that the rest of this will use to make its decisions.

variable "vpc_id" {
  type    = string
  default = null # optional with no default value
}

From this we will derive our decision about whether to create the VPC, which I'll write as a local value here mainly for exposition purposes but you might choose to inline it at the point of use in practice depending on what feels most readable for you:

locals {
  create_vpc = var.vpc_id != null ? true : false
}

We can now use that value both with the module and with a data resource so that we'll either create a new VPC or retrieve data about the existing one:

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  create_vpc = local.create_vpc
  # ...
}

data "aws_vpc" "selected" {
  count = local.create_vpc ? 0 : 1

  id = var.vpc_id
}

We can then write another local value that chooses which of those two locations to derive other VPC-related values, such as the id and cidr_block from:

locals {
  vpc = (
    local.create_vpc ?
    {
      id         = module.vpc.vpc_id
      cidr_block = module.vpc.vpc_cidr_block
    } :
    {
      id         = data.aws_vpc.selected.id
      cidr_block = data.aws_vpc.selected.cidr_block
    }
  )
}

You can then use local.vpc.id and local.vpc.cidr_block elsewhere in the configuration to get the appropriate values to use regardless of whether this module created its own VPC or not.

The above is the most direct answer to your question. I also wanted to share a different approach that is based on the patterns described in the Module Composition section of Terraform's docs, which might be appropriate for you if you're building a shared module intended to be called by another Terraform module.

A different way to approach this problem using composition is to say that your module never creates its own VPC, and always expects one to be provided to it. This is an example of the dependency inversion principle. Here's how that might look for your particular use-case...

In your module, you can declare an input variable called vpc that has the same structure as the final local.vpc that we created in the previous approach:

variable "vpc" {
  type = object({
    id         = string
    cidr_block = string
  })
  # This time the VPC is required. The caller decides how to obtain it.
}

Your own module's expressions would then get the VPC id as var.vpc.id instead of local.vpc.id as in the previous approach.

The calling module can then decide between several different ways of providing the value for that variable:

# Using the terraform-aws-modules/vpc/aws module
module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  # ...
}

module "your_module" {
  source = "./modules/your-module"

  vpc = {
    id         = module.vpc.vpc_id
    cidr_block = module.vpc.vpc_cidr_block
  }
}

# Directly using the aws_vpc resource type

resource "aws_vpc" "example" {
  # ...
}

module "your_module" {
  source = "./modules/your-module"

  vpc = aws_vpc.example
}

# Using the aws_vpc data source to find an existing VPC

data "aws_vpc" "example" {
  tags = {
    Environment = "STAGE"
  }
}

module "your_module" {
  source = "./modules/your-module"

  vpc = data.aws_vpc.example
}

In other words, this lets your new module focus on whatever new problem it is trying to solve while being decoupled from the problem of creating a VPC, allowing the calling module to obtain that VPC information in whatever way is appropriate. The appropriate way to obtain the VPC might change over time, and this composition approach means that callers can have more control over how they manage the transition between different approaches, independent of the implementation of your new module.

Module composition isn't always the best option, but I wanted to point it out because the Terraform documentation specifically recommends considering this approach and recommends against having many levels of nested module calls.


How to stop a container dying at the end of an execution? I'm running git-all-secrets over our organisation to ensure we have nothing exposed. I'm doing this using:

sudo docker run --rm -it abhartiya/tools_gitallsecrets -org <org> -output /root/output.txt -thogEntropy -token <githubtoken>

I've monitored the events for this repository, and after the command completes, the following events fire:

2019-02-21T13:01:36.386539368Z container die cc479e7ac878f39c1e555eea1cbc9105dc06e21a526639975cd15f031297ab23 (exitCode=0, image=abhartiya/tools_gitallsecrets, name=inspiring_bell)
2019-02-21T13:01:36.602243215Z network disconnect 26b5a225be28b13153e587c4d1464361957ac899bc1a4152c6ae5aacc3229a0f (container=cc479e7ac878f39c1e555eea1cbc9105dc06e21a526639975cd15f031297ab23, name=bridge, type=bridge)
2019-02-21T13:01:37.235130059Z container destroy cc479e7ac878f39c1e555eea1cbc9105dc06e21a526639975cd15f031297ab23 (image=abhartiya/tools_gitallsecrets, name=inspiring_bell)

Since the container is  then destroyed, I can't then copy the file locally after exection by using:

a@ubuntu-s-2v-2-01:~$ sudo docker cp cc479e7ac878:/root/output.txt ./output.txt                                                                                │ 0 -rw-r--r-- 1 codingo codingo     0 Dec 19 04:49 .sudo_as_admin_successful
Error: No such container:path: cc479e7ac878:/root/output.txt

I'm still new to docker, so I'm not sure what the solution to my problem is - but how can I have  the docker contain

Option --rm tells Docker explicitly to remove the container after the (main) command finished. Just leave it out.

Note that none of the examples contain this option; they only use it for -help.

As an alternative (which lets you keep --rm), you can tell Docker to mount a volume at /root. Volumes persist after the containers mounting them go away, so you can access the data written there. However, for your purposes here, a simple bind mount is quite enough; add

-v .:/root

to have it write the output file to the current (host) directory. You may have to adjust this if /root is used heavily; you probably want to use a previously empty directory on both sides.


What's the impact of the General Data Protection Regulation (GDPR) on DevOps? The General Data Protection Regulation (GDPR) is a set of rules to improve the protection of data about European citizens. Quote from this link:

  The EU General Data Protection Regulation (GDPR) is the most important change in data privacy regulation in 20 years.

Refer to GDPR Key Changes for a summary of what it is all about, especially the Data Subject Rights, such as:

Breach Notification.
Right to Access.
Right to be Forgotten.
Data Portability.
Privacy by Design.
...

Some GDPR facts:

GDPR will be enforced as of 25 May 2018 (that's pretty close).
Quote from GDPR Key Changes (about Territorial Scope):

  ... will apply to the processing of personal data by controllers and processors in the EU, regardless of whether the processing takes place in the EU or not. The GDPR will also apply to the processing of personal data of data subjects in the EU by a controller or processor not established in the EU, where the activities relate to: offering goods or services to EU citizens (irrespective of whether payment is required) and the monitoring of behaviour that takes place within the EU. Non-Eu businesses processing the data of EU citizens will also have to appoint a representative in the EU.

For anybody who is old enough to remember the Y2K-hype back in the 90s (i.e. the impact of it on IT systems): IMHO addressing all those Y2K issues was a piece of cake as compared to the challenge ahead with this GDPR thing.

Question(s): What is the impact of GDPR on DevOps, more specifically I wonder about:

What kind of changes will be needed in the DevOps toolchain to make them GDPR-compliant?
How can DevOps-practises help (facilitate, simplify, etc) a company to become GDPR-compliant, similar to how SCM tools helped in the 90s to become Y2K compliant?

"Enforcing" means lawyers going after possible culprits. Frankly, in my opinion (and those of some folks around me which are more a mix between IT-guy and lawyer), nobody knows how that will work out exactly.

Those iLawyers expect a lot of very interesting court cases in the first months to tests out the water, and see what precedence cases are going to come up. There will probably be plenty of law firms fishing for nice fat fish to fry.

For me personally, I don't really see that much impact specifically regarding DevOps. The applications themselves need to be adapted as necessary, obviously. Also, the actual data storage, be it RDBMS, elastic search indexes or whatever, will have to be looked at.

Aside from that, one benefit of DevOps, specifically if you work with some kind of container system and infrastructure-as-code is that you should generally know exactly where your data, log files etc. is stored, and what kind of them you have; much more so than in classical servers, where your data and especially logs might be strewn all over the place. It should also be very easy to regularly roll your data as applicable, i.e., get lost of old stuff.

So it can help to, for example, take your catalogue of i-a-c files and walk through them diligently - you can then be pretty sure to have found everything you need to be aware of.


Build image from inside a pod or container It seems pretty easy to build image when I just mount the docker.sock in a container. It's same as what I do on hosts machine. Simply docker build . -t image/name:sometag.
But when I need to do the same from a pod/container in K3s installation, I don't find a way of doing it. Can someone please guide me to something? It has to happen without mounting docker.sock.

If you simply need to build a docker image, I recommend using Kaniko. It allows you to build a docker image without having to expose the docker.sock or running your pods in privileged mode, which introduces potential security vulnerabilities. Per the Kaniko documentation, your spec file would look similar to:
apiVersion: v1
kind: Pod
metadata:
  name: kaniko
spec:
  containers:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:latest
    args:
    - "--dockerfile=<path to Dockerfile within the build context>"
  restartPolicy: Never


Docker NGINX issue -- unable to edit nginx.conf I'm having trouble figuring out how to edit nginx.conf inside NGINX open source Docker container.

I'm trying to follow the directions here:

https://docs.nginx.com/nginx/admin-guide/installing-nginx/installing-nginx-docker/#maintaining-content-and-configuration-files-in-the-container

I've gotten through all 4 steps succcessfully. I have SSH access using the helper mynginx4_files container.

This part confuses me, though:

  the /bin/bash argument means that the bash shell runs in the helper
  container, presenting a shell prompt that you can use to modify files
  as needed

This sounds good! Unfortunately, I don't see any way I can modify nginx.conf. There's no text editor that I can see. I don't see vi or emacs or anything else. (I searched through all 4 'bin' directories, and whereis and find didn't turn up anything for vi.) I tried to install vi using apt-get but was unsuccessful. I even tried ubuntu instead of debian, but even with ubuntu there is no text editor. I thought vi is standard on all Linux distributions. I don't know what's up, but I feel like I'm missing something big here. Any help greatly appreciated.

I suggest you mount your conf file as a volume:

docker run -v nginx.conf:/etc/nginx/nginx.conf ....

This way you can easily change the file outside the container and then just restart the container. If you change your config file inside the container and then you should have to restart nginx to pick-up the changes. At that point your container will stop because you stopped the main process.

Your local nginx.conf should already have the initial configuration which you can copy from inside the container (cat /etc/nginx/nginx.conf).


Is there an established metric to describe optimal/needed ressources for a specific computational task? Let's say I have a piece of software which does a computational task that is pretty demanding to either CPU/GPU/RAM/Disk-IO or a combination of those. Then it would be optimal if one has some metric to describe what the optimal configuration of a machine would be to run this software.

Guess that metric should include: 1) "minimal requirements" and more importantly 2) the "optimal ratio" of the above parameters, so that there is no idling CPU/RAM/GPU so that I get the best computation per money/electricity/computation time.

This is especially true, if I want to run your job on AWS/GoogleCloud/OtherProvider or on an HPC environment where you can pick those parameters.

I am aware that this is probably a pretty hard problem, since it is not easy to compare different kinds of CPUs (Cache, clock speed, etc.) or different kinds of RAM.

But still, does there exist and "standard" or approach for this? If yes, what are the methods to establish such a metric?

PS: Is this the right place to post this question?

As far as I'm aware, there is no universal standard amongst cloud providers. One reason for this is because most cloud providers like to abstract how their backend infrastructure, that their instances run on, is managed. How your instance is ran could change minute by minute depending on the cloud provider. The user doesn't care as long as they are getting the desired computing power that they want. Cloud providers also may not want to divulge their exact infrastructure metrics for proprietary reasons as well.

Despite these reasons, some cloud providers offer a standard amongst their own virtual instances. These standards are relative to the computing power of each instance type that the provider offers.

For example, Amazon Web Services uses EC2 Computing Units (ECU) as a metric to compare the approximate computing power in relation to each instance. Comparing two instances types such as an m5.large and an m5.xlarge for example, the m5.large has an ECU of 8 vs the m5.xlarge which has a ECU of 16. The m5.xlarge has roughly double the compute power and therefore double the cost.


What Key Performance Indicators (KPIs) are used to measure DevOps? I am trying to drive good behaviors within a DevOps transformation programme, to support this I am looking at identifying actionable metrics around the operations disciplines:

Problem and Incident Management
Capacity Management
Change and Release Management

To be absolutely clear, these are functions that used to belong to the operations organization and are now owned by the Agile/DevOps organization.  There are existing KPIs that drive bad behaviors are:

Time to Root Cause Analysis completed:

Drives incomplete RCAs just to get them into the system on time.

Test execution duration:

Disables long running tests, regardless of their business value.

Average utilization of cloud services:

Encourages over-commitment of compute resources, resulting in slow response times

What Key Performance Indicators can be used to encourage good behaviour in a DevOps Programme?

I don't think there are any "universal" DevOps KPIs.  For example, velocity is great, unless it's not a key driver for your business.  Amazon cares a lot about velocity because they have a massive retail operation.  That's less important for a small app with 100 users.

This begs the question: how do you select the best KPIs relevant to your business?  That's a research and discovery process that involves your entire Enterprise.

What do you care about?

Quality
Reliability
Maintainability
Velocity
Process Improvement
Service Levels

What keeps your business stakeholders up at night?  What determines whether you make money this quarter or not?  The list above might include some of those things, or it might not.  Make your list, then figure out how to align incentives across every department to achieve them.

Incentives drive behavior, so decide collaboratively on SMART goals.  Pick two or three items off your brainstormed list, and start a measure/fix feedback cycle for each.  Don't pick too many at once- you're more likely to succeed by focusing hard on two or three things.


ansible task -easy_install with requirements.txt file I want to run easy_install task in order to install many modules in virtual environment.

Is there an option to install all modules in requirements.txt file? as I understand it accept only one module at a time.

Another option is to do it with with_items but how can I make Ansible
iterate over both the name and the version of each module

- name: Install things
  easy_install:
   name: pip
   state: latest

I want certain versions obviously...

I can not work with Pip task, only easy_install.
Thanks

You can loop with dictionaries using loop. It doesn't look like the easy_install module handles versions with a module attribute, so you might have t pass the version on the name attribute.

- name: install things with easy_install
  easy_install:
    name: "{{ item.name }}=={{item.version}}"
    state: present
  loop:
    - { name: 'pip', version: '18.1' }


AWS Automation Document Run Steps Against Different Servers Targeted By Tag As the title describes I want to execute a step against a resource, but I want to specify that resource not by resource id but by tags, I have tried with the below example, but it doesn't work. What is the proper syntax and is that possible at all?

---
description: "Automation Document in YAML - version 3.0. Script to fully start instances in order respecting components dependencies. Created by - xxxx"
schemaVersion: "0.3"
# assumeRole: "{{ AutomationAssumeRole }}"
# parameters:
#   InstanceId:
#     type: "StringList"
#     description: "(Required) EC2 Instance(s) to start"
#   AutomationAssumeRole:
#     type: "String"
#     description: "(Optional) The ARN of the role that allows Automation to perform the actions on your behalf."
#     default: ""
mainSteps:
## STEP 1
- name: startSomeProductComponent1Test
  action: aws:runCommand
  maxAttempts: 2
  inputs:
    Tags:
    - Key: Product
      Value: SomeProduct
    - Key: Name
      Value: SomeProduct-Component1-Test
    DocumentName: AWS-RunPowerShellScript
    Parameters:
      commands:
      - Start-Service -Name "Component1"
      - Start-Sleep -s 30
  nextStep: ...

The issues were a lot, but the main one was not using Targets properly and then some YAML syntax problems.

---
description: "Example"
schemaVersion: "0.3"
assumeRole: "arn:aws:iam::{ID}:role/Example"
# parameters:
#   InstanceId:
#     type: "StringList"
#     description: "(Required) EC2 Instance(s) to start"
#   AutomationAssumeRole:
#     type: "String"
#     description: "(Optional) The ARN of the role that allows Automation to perform the actions on your behalf."
#     default: ""
mainSteps:
## STEP 1
- name: StartExampleServiceOnServer1
  action: aws:runCommand
  maxAttempts: 2
  inputs:
    DocumentName: AWS-RunPowerShellScript
    Targets:
    - Key: tag:Name
      Values:
      - ExampleOne-SomeService-Dev
    Parameters:
      commands:
      # Start service for SomeService
      - Start-Service -Name "ExampleOne"
      - Start-Sleep -s 30
  nextStep: ...


Jenkins deploy static web with Apache2 I have a static website that I would like to deploy to my Apache2 server using Jenkins. I think of 2 solutions:

Should I change the workspace dir of Jenkins to /var/www/?
Or should I copy all files in workspace to /var/www in post build?

As mentioned, there are many ways to accomplish the final goal of exposing a site built from a Jenkins job to public traffic.

Both ideas could work, but here are some tweaks.

Instead of changing the job workspace directory, build the site to a sub directory in the workspace and symlink the directory with root directory of your site. This allows control over what is part of live site.
To put a boundary between the build and deploy, rsync would be more efficient and easier to manage than copying. Follow the same pattern of creating a sub directory and rsync to the site directory.

The decision depends on the requirements of the site. If it is static with no promise of up-time, a symlink will work fine, however if the Jenkins job breaks and breaks the workspace version of the site, the actual site will be down. I would recommend using rsync and adding a validation step between the build and rsync to ensure the site stays up.

Final note, using git or other scm provides backups which can be tagged. If the current build breaks, rolling back would be easy. However, if there is no scm (not recommended), keep some segregation between the build and live site to ensure it is not broken for an extended amount of time.


Error 502 before execute Ansible playbook I execute an Ansible playbook with  an ansible user. The playbook executes properly, with no errors in the application log, no errors in the Ansible log. This .war usually deploys with user "tomcat" when the deploy is manual.
Could this be the cause of the problem? Execute playbook with different users
This is my playbook (syntax is correct):
- name: Deploy war
  hosts: grupo1
  tasks:
    - name: Stop Tomcat
      command: systemctl stop tomcat

    - name: Delete old war1
      command: rm -rf /opt/tomcat/catalina_base/webapps.old/portal-admin
      args:
        warn: false

    - name: Delete old war
      command: rm -rf /opt/tomcat/catalina_base/webapps.old/portal-client
      args:
        warn: false

    - name: Copy the old war
      copy:
        src: /opt/tomcat/catalina_base/webapps/portal-client
        remote_src: true
        dest: /opt/tomcat/catalina_base/webapps.old/

    - name: Copy the old war2
      copy:
        src: /opt/tomcat/catalina_base/webapps/portal-admin
        remote_src: true
        dest: /opt/tomcat/catalina_base/webapps.old/

    - name: delte the war fileclient
      shell: /bin/rm -rf /opt/tomcat/catalina_base/webapps/portal-client/*
      args:
        warn: false

    - name: deploy warclient
      command: unzip /tmp/portal-client.war -d /opt/tomcat/catalina_base/webapps/portal-client/
        args:
          warn: false

    - name: delete the war fileadmin
      shell: /bin/rm -rf /opt/tomcat/catalina_base/webapps/portal-admin/*
      args:
        warn: false

    - name: deploy waradmin
      command: unzip /tmp/portal-admin.war -d /opt/tomcat/catalina_base/webapps/portal-admin/
      args:
        warn: false

    - name: Start tomcat
      command: systemctl start tomcat

The error when I execute the playbook is
Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /portal-client/cm/misalud/.

Reason: Error reading from remote server

First - @caxcaxcoatl is completly right - your playbook is not Ansible, it is a bash script inside Ansible and that isn't Ansible.
Your error doesn't have something to do with your playbook, but with that, what the playbook is doing. You're redeploying an application and then the proxy in front of the Tomcat cannot read the response from your Tomcat. So your deployment is wrong, has some failures, what ever. You should have a look at your Tomcat HTTP response (without the Proxy in front) direct on the machine (with curl, telnet or whatever). Maybe you should also check the logs of Tomcat to get the error. The proxy sends the error, because the Tomcat is answering with a 500 HTTP error code, when the proxy hits /portal-client/cm/misalud/.
To give you a hint, how a playbook could look like (of course this wasn't the question, and it could be done much better then this with includes for a unspecific WAR and just call include_tasks on all available WARs in the main playbook, or with a Ansible module written in Python just giving the params of the new WAR etc.):
- name: Deploy war
  hosts: grupo1
  vars:
    webapps_cur "/opt/tomcat/catalina_base/webapps"
    webapps_old: "/opt/tomcat/catalina_base/webapps.old"
  tasks:
    - name: "Stop Tomcat"
      systemd:
        name: tomcat
        state: stopped

    - name: "Delete old Backup WARs"
      file:
        path: "{{ webapps_old }}/{{ item }}"
        state: absent
      with_items:
        - "portal-admin"
        - "portal-client"

    - name: "Backup the current WARs"
      copy:
        src: "{{ webapps_cur }}/{{ item }}"
        dest: "{{ webapps_old }}/{{ item }}"
        remote_src: true
      with_items:
        - "portal-admin"
        - "portal-client"

    - name: "Delete the current WARs"
      file:
        path: "{{ webapps_cur }}/{{ item }}"
        state: absent
      with_items:
        - "portal-admin"
        - "portal-client"

    - name: "Deploy new WARs"
      unarchive:
        src: "/tmp/{{ item }}.war"
        dest: "{{ webapps_cur }}/{{ item }}"
        user: "tomcat"
        group: "tomcat"
      with_items:
        - "portal-admin"
        - "portal-client"

    - name: "Start Tomcat"
      systemd:
        name: tomcat
        state: started


Best AWS EC2 instance class for general Kubernetes cluster Hi We are deploying a fairly standard multiple purpose (data replication workers and some nginx web proxies) in an EKS cluster, and wondering what EC2 instance class people recommend for this workload. Probably would look at a base of 3 x (M5|T3|..).large instances with the ability to scale it up. I wonder if others have experience with comparing the compute types.

I think the instance class is going to vary based on the kind of workloads you run. You might start out with X but next thing you know your running a bunch of random stuff. You can always run a cluster of different instance classes and then have the scheduler schedule your workloads to the right instance.

M5 and t3 are Nitro based instances. They have a big list of benefits https://aws.amazon.com/ec2/nitro/ . My favorite being that I can log into those instances using my AWS account with no SSH keys.

My personal workloads are more memory-hungry. Since you can't throttle memory like cpu you should factor that in when sizing your instances. I think fewer denser instances are probably more favorable than lots of small instances.

I'm not sure what your use case is for running nginx rever proxies but you may want to look into kubernetes ingress.


Ansible Remote provisioning not working in Docker container build I have a Packer template to start with a Docker container and provision it.  The shell script provisioners seem to be working, but, while the playbook runs and seems to succeed, unlike the shell provisioners, it seems to not be applying to the Docker container.

packer-template.json

{
  "builders": [
        {
            "type": "docker",
            "image": "chenjr0719/ubuntu-unity-novnc",
            "commit": "true"
        }
    ],
   "provisioners": [
        {
            "type": "shell",
            "inline": ["sudo apt-get update"]
        },
        {
            "type": "shell",
            "script": "setup_ansible.sh"
        },
        {
            "type": "shell",
            "script": "add_ansible_roles.sh"
        },
        {
            "type": "ansible",
            "playbook_file": "./kubeadm.yml"
        }
   ]
}

setup_ansible.sh

sudo apt-get install software-properties-common -y
sudo apt-add-repository ppa:ansible/ansible -y
sudo apt-get update
sudo apt-get install ansible -y

add_ansible_roles.sh

sudo ansible-galaxy install djx339.k8s-kubeadm-master

kubeadm.yml

- name: install kubeadm
  hosts: localhost
  roles:
    - { role: djx339.k8s-kubeadm-install }

Output from the playbook

docker: PLAY [install kubeadm] *********************************************************
docker:
docker: TASK [Gathering Facts] *********************************************************
docker: ok: [localhost]
docker:
docker: TASK [djx339.k8s-kubeadm-install : include_tasks] ******************************
docker: included: /home/dw/.ansible/roles/djx339.k8s-kubeadm-install/tasks/setup-Debian.yml for localhost
docker:
docker: TASK [djx339.k8s-kubeadm-install : Sysctl] *************************************
docker: ok: [localhost] => (item={u'name': u'net.bridge.bridge-nf-call-iptables', u'value': 1})
docker:
docker: TASK [djx339.k8s-kubeadm-install : Add kuebeadm apt key] ***********************
docker: ok: [localhost]
docker:
docker: TASK [djx339.k8s-kubeadm-install : Add kubeadm apt repo] ***********************
docker: ok: [localhost]
docker:
docker: TASK [djx339.k8s-kubeadm-install : Install kubeadm] ****************************
docker: ok: [localhost] => (item=[u'kubelet', u'kubeadm', u'kubectl', u'kubernetes-cni'])
docker:
docker: PLAY RECAP *********************************************************************
docker: localhost                  : ok=6    changed=0    unreachable=0    failed=0

Running kubeadm in the docker container throws an error.

Why isn't the playbook applying? It works when I run it locally.

The error which I'm getting is:

  kubeadm not found

So basically it's not installing kubeadm.

Note: just noticed the mismatch between the role I'm adding in the add_ansible_roles.sh script and the playbook.

It appears that you are trying to run ansible locally inside the container but instead you are running it locally on the machine you use to run packer on(the provisioning one).

To run ansible in local mode you need to update the ansible part of your packer file to:

    {
        "type": "ansible-local",
        "playbook_file": "./kubeadm.yml"
    }

Also, I would suggest to add connection: local to your ansible playbook, to avoid unnecessary attempts for ssh connections.

- name: install kubeadm
  hosts: localhost
  connection: local
  roles:
    - { role: djx339.k8s-kubeadm-install }


Specifying branch name for jenkins job through AnyStatus I am wondering if anyone else has integrated before a parameterized Jenkins build with AnyStatus.

When removing the parameter above, I am able to run builds using the branch specified previously from the browser, but that's not the idea. There is a post explaining that I can use a script, but that wouldn't be integrated with AnyStatus in the existing Jenkin's plugin.

Oh, I figured it out. I just went to the job configuration, activated the "build is parameterized" checkbox, added a string parameter called BRANCH, and in the "branches to build" section added ${BRANCH}. Pretty much what is explained here:

https://stackoverflow.com/questions/20359810/how-to-trigger-jenkins-builds-remotely-and-to-pass-parameters
https://stackoverflow.com/questions/32108380/jenkins-how-to-build-a-specific-branch


Discovering extra args needed for running ansible playbooks I've got a number of ansible playbooks that I come back to from time to time, some have required and optional arguments that I forget about from time to time.

I know there's a list-tags, list-hosts and list-tasks option baked into the ansible-playbook command, there's no list-args, which would be super useful, but maybe it means I'm using ansible wrong?

So what strategies do devops engineers use to manage "api's" for ansible playbooks? Maybe just a recursive grep command to list non-standard args that aren't registered variables like this

grep -or "{{[a-zA-Z_ ]\+}}" roles/.  | grep -v "ansible" | grep -v "item"

only better.

For instance, I've got a program I want to put on a device called "showstopper", but I specify the version with extra_args and if I don't specify the version, it doesn't get installed.

 - name: download showstopper from artifactory
   local_action: get_url url=https://example.net/artifactory/generic-release-local/com/cdw/mans/silo/showstopper/showstopper-{{ showstopper_version }}.noarch.rpm dest=/tmp validate_certs=no
   run_once: true
   when: showstopper_version is defined

 - name: copy showstopper to silo
   copy: src=/tmp/showstopper-{{ showstopper_version }}.noarch.rpm dest=/tmp
   when: showstopper_version is defined

 - name: Install showstopper
   yum: name=/tmp/showstopper-{{ showstopper_version }}.noarch.rpm state=present
   notify:
    - start cron
   when: showstopper_version is defined

This works good when I run it the way I intended to run it, but in 2 months when I want to run it on another machine, I forgot to specify showstopper_version and the last few machines I've built went out without that configured on it.

If an extra variable is truly required, you can add assert statements at the top of the role to stop it from executing if they're not defined:

┌─[jamesph@geror] - [~/temp] - [Sat Jan 13, 09:40]
└─[$]> cat assert.yml
- hosts: localhost
  vars:
    foo: False
  tasks:
    - assert:
        that:
          - "foo is defined"
          - "bar is defined"
┌─[jamesph@geror] - [~/temp] - [Sat Jan 13, 09:40]
└─[$]> ansible-playbook assert.yml
 [WARNING]: Unable to parse /etc/ansible/hosts as an inventory source

 [WARNING]: No inventory was parsed, only implicit localhost is available

 [WARNING]: Could not match supplied host pattern, ignoring: all

 [WARNING]: provided hosts list is empty, only localhost is available

PLAY [localhost] *************************************************************************************

TASK [Gathering Facts] *******************************************************************************
ok: [localhost]

TASK [assert] ****************************************************************************************
fatal: [localhost]: FAILED! => {
    "assertion": "bar is defined",
    "changed": false,
    "evaluated_to": false
}
    to retry, use: --limit @/Users/jamesph/temp/assert.retry

PLAY RECAP *******************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1


Install and remove a program in the same Puppet declaration The program I want on my VM requires 7-zip as part of the installation process but not after the install.  I want to leave the VM as clean as possible by removing 7-zip after the install is done.  In puppet it seems like it would be easy enough to declare once to ensure present and again later to ensure absent.
I am currently stuck with a Duplicate declaration error. In my case 7-zip but this could appear for any package.

Duplicate declaration: Package[7-Zip 19.00 (x64 edition)] is already
declared

I have the code split into multiple classes but here is what is looks like in a single manifest.
$installer_location = 'puppet:///resources/'
$installer_name = "7z1900-${::architecture}.msi"
$install_directory = 'C:/temp/'

#Create directory for installer
file { 'install_dir':
  ensure => directory,
  path   => $install_directory,
}

# Copy installer local
file { 'Copy_7zip_installer':
  ensure => present,
  path   => "${install_directory}${installer_name}",
  source => "${installer_location}${installer_name}",
}

# install 7zip
package { 'Install 7-zip':
  ensure   => present,
  provider => 'windows',
  source   => "${install_directory}${installer_name}",
}

#Use 7-zip to unpack archive and do pipelines agent install
String $package_path = lookup('azure_pipelines_agent::package_source')
String $package_name = lookup('azure_pipelines_agent::package_name')
String $install_path = lookup('azure_pipelines_agent::install_path')

# Create the install directory
file { 'install-path':
ensure => directory,
  path   => $install_path,
}

# Copy Azure pipeline package
# Due to a bug the archive forge module requires 7-zip
# instead of defaulting to powershell for .zip files
archive { 'extract_azure_package':
  source       => $package_path,
  path         => "${install_path}${package_name}",
  extract      => true,
  extract_path => $install_path,
  cleanup      => true,
}

# uninstall 7zip
# The title (or name) of the package must match the value of the package’s DisplayName property in the registry
# https://puppet.com/docs/puppet/6.17/resources_package_windows.html
package { '7-Zip 19.00 (x64 edition)':
  ensure   => absent,
  provider => 'windows',
  source   => "${install_directory}${installer_name}",
}

# ... Configuration and setup of Azure Pipeline Agent

Is there a simple means of working around what puppet sees as a duplicate declaration?

I think that the simplest solution is to have the lifecycle of the 7-Zip package managed by exec resources rather than as package resources.  Imagine an exec resource that installs 7-zip, lets call it install-7-zip.  There is also a second puppet exec resource that uninstalls 7-zip, lets call it uninstall-7-zip.
If all of your other puppet resources that need to use 7-zip then require 'install-7-zip' and notify 'uninstall-7-zip' then they will all be actioned whilst the executable is available.
however its not a very puppet way to solve the problem. It means that even if there are no changes required, you will still be running these exec resources, and puppet will think it has changed the platform.
is repackaging the azure pipeline package a possible alternative? perhaps re-zipping it with a different compression setting? or generating a self extracting 7-zip exe instead?


AWS CodePipeline staged deployments I am implementing a CICD pipeline using CodePipeline, CodeCommit, CodeDeploy and CloudFormation (I skipped the build part of the pipeline and am going to commit built code for the moment).

I already have a CloudFormation template that creates the required infrastructure to host the application, a CodeDeploy Application (in a nested stack) and a CodePipeline that gets the source code from CodeCommit, does In-Place deployments via CodeDeploy and finally sends an approval email (in a nested stack).

I would like to implement the following:

In order to create the required infrastructure to host the application for the first time, I launch a CloudFormation template that creates (I already have this):

The required infrastructure to host the application (AutoScalingGroup and Launch Config, TargetGroup and ListenerRules, IAM roles)
CodeDeploy Application and DeploymentGroup (this will be in a nested stack)
CodePipeline pipeline pre-configured to carry out the next steps (this will be in a nested stack)

When source code is committed to CodeCommit, CodePipeline launches a new CloudFormation stack containing the required infrastructure to host the application, then deploys the new version of the code to this stack, without affecting the existing stack for the old version of this application. The new stack will have a listener rule with a different host name, so I can test the application while users keep using the old version;
Once the code has been deployed, CodePipeline sends an approval email (through an SNS topic and I already have this);
If the email is approved, the old stack (created in step 1. ) is deleted
I will manually swap the listener rules for the two stacks to direct users to the new stack created in step 2, then delete the old stack.

I am struggling to implement step 2 and currently trying to use a Deploy action with a CloudFormation deployment provider to carry out step 2. above. I am struggling to pass the output artifacts (new source code) from the previous stage to the new stack.

Can you provide some guidelines on how to implement step 2 ?

I managed to setup an environment that does what I described above. Here are the full details:

Launch a template that creates a CodeCommit repository for the
application. This repository will be used to store both the source
code and the CloudFormation (CF) templates to create the required
infrastructure for the application;
Launch a template that creates a CodePipeline pipeline for the
application. The pipeline will deploy a stack for the infrastructure
and one for deploying the code through CodeDeploy (the CodeDeploy
stack will will create a new CodeDeploy Application and deployment
group, and will deploy the latest code to the infrastructure created
previously)

The pipeline has a paramater "StackBlueeGreen" that is used to determine whether the stack is the Green or the Blue version.

When the pipeline runs successfully, it will create a separate infrastructure

  (NameOfTheApplication-B or NameOfTheApplication-G)

and deploy the code there. The second infrastructure will have its own load balancer listener rule setup with a staging URL

  (something like URLOfTheApplication-staging.somedomain.co.uk)

This allows developers to test the new version of the application withuot affecting the previous version. Once developers are happy with the new version, the listener rules for the old and new infrastructure need to be changed manually so that users are now directed to the new infrastructure. If there is an issue with the new version, rolling back is as simple as swapping the load balancer listener rules again.

Finally, the pipeline stack need to be updated: the StackBlueGreen parameter needs to be switched to G/B depending on how it is currently set. If this is not done, next time there is a change, the new version will be deployed on the same infrastructure.

[The setup described above works very well and meets all the requirements outlined in the OP.
An improvement that I would really like to implement is automating the switching of the load balancer listener rule, deletion of the old infrastructure and updating the pipeline stack. I will add to this question if I manage to implement this improvement.
Meanwhile, any input would be much appreciated!]


How securely to keep google cloud service account key? How securely to keep google cloud service account key?

I want to ask about how we should securely keep service account key. The scenario looks like:

Our product is connecting to BigQuery.
We have project which is used only by developers as playground.
We need to run integration tests in pipelines and they needs service account to run correctly.
Developer also should be able to run integration tests locally.

Where should I put the service account key?

I can add it to our repository, but I'm not sure if this solution is ok. (even it is only key with read-only permission to developer version of BigQuery).

The second solution is create a few keys. And every developer should has own. Then every developer have to care about key himself (and we can easy remove key when developer i.e quit job)

I would recommend you use their Google Cloud Secrets Engine. Documentation is here: https://www.vaultproject.io/docs/secrets/gcp/index.html

Managing such sensitive secrets by any means other than a Vault service is dangerous.


java.lang.UnsupportedOperationException: no known implementation of class jenkins.tasks.SimpleBuildWrapper is named BuildUser in jenkins I preparing script in Jenkins as below where I getting error while build job. This job is send email to user for input for next step.
 [Pipeline] End of Pipeline
 java.lang.UnsupportedOperationException: no known implementation of class jenkins.tasks.SimpleBuildWrapper is named BuildUser
    at org.jenkinsci.plugins.structs.describable.DescribableModel.resolveClass(DescribableModel.java:549)
    at org.jenkinsci.plugins.structs.describable.DescribableModel.coerce(DescribableModel.java:473)
    ...
    ...

Version :
$ java --version
 openjdk 11.0.11 2021-04-20

jenkins : 2.277.3

Pipeline code:
def user
node {
wrap([$class: 'BuildUser']) {
user = env.BUILD_USER_ID
}

emailext mimeType: 'text/html',
             subject: "[Jenkins]${currentBuild.fullDisplayName}",
             to: "user@xxx.com",
             body: '''click to approve'''
}

pipeline {
agent any
stages {
    stage('deploy') {
        input {
            message "Should we continue?"
            ok "Yes"
        }
        when {
            expression { user == 'hardCodeApproverJenkinsId'}
        }
        steps {
            sh "echo 'describe your deployment' "
        }
    }
   }
}

Can anyone please review this, please ?

Have you installed the Build User Vars Plugin? This plugin provides the "BuildUser" class.


How to exclude storage.bucket.list permission on public gcp storage bucket Google Cloud Platform Storage Buckets can be directly exposed via HTTP. The recommended configuration for this is to have allUsers with the Storage Object Viewer role.

The Storage Object Viewer role has got storage.bucket.list permission. I tried creating a custom role based on this role and removed the storage.bucket.list permission.

I updated the IAM permissions by getting permissions with gcutil, editing, then I tried to update.

When I tried updating I used the following command

gsutil iam set permissions.txt gs://example.com/

But I got the following error

  BadRequestException: 400 Role roles/CustomStorageObjectViewer is not
  supported for this resource.

Is there any way to configure a public gcp storage bucket without the storage.bucket.list permission?

I am not entirely sure what configs you were using, but here is a process using CLI (gcloud and gsutil). The end product is a new custom role replacing the current standard roles/storage.objectViewer on allUsers for a GCP Bucket.
Recommend using GCP Cloud Cloud Shell in your Project: https://cloud.google.com/shell

Create a new Role with the Custom Permissions you would like.

Create yaml file to hold the new config:
vi custom_role.yaml

Add role configurations to new custom_role.yaml:
title: "storageobjectviewer.nolist"
description: "Storage Object Viewer Role without source objects list"
stage: "ALPHA"
includedPermissions:
- resourcemanager.projects.get
- storage.objects.get

Create new Role in GCP:
gcloud iam roles create storageobjectviewer.nolist --project=[YOUR PROJECT] --file=custom_role.yaml

Pull current permissions for [YOUR BUCKET] locally:

gsutil iam get gs://[YOUR BUCKET] > perms.txt

Update Permissions with the new role.

Should only be this section:

{
  "members": [
    "allUsers"
  ],
  "role": "projects/[YOUR PROJECT]/roles/storageobjectviewer.nolist"
}

Update the allUsers to the new role on [YOUR BUCKET]

gsutil iam set perms.txt  gs://[YOUR BUCKET]

The new configuration should be viewable in CLI:
gsutil iam get gs://[YOUR BUCKET]

Also in the UI:
Bucket Details -> Permissions

and IAM -> Roles

Hopefully this helps!
Additional Info:

https://cloud.google.com/iam/docs/creating-custom-roles#iam-custom-roles-create-gcloud
https://cloud.google.com/storage/docs/gsutil/commands/iam
https://cloud.google.com/storage/docs/access-control/making-data-public#gsutil
https://cloud.google.com/storage/docs/access-control/iam-roles


Saving time on an EC2 instance I have started playing around with AWS (privately). I have one free tier Amazon Linux 2 VM running right now, which serves a "fat" service (in a docker container) to myself and one other person. It has an Elastic IP associated with it.

We use the service 1-2 hours every few days.

Is there, in the rather large AWS zoo of services, a smart way to have the VM shut down when not in use, and come up when we need it? "Need it" could be measured by counting network traffic, i.e. I could in principle write a little script which uses tcpdump or something like that to watch for 15 minutes of no traffic on the port of the service and then somehow trigger a shutdown; but how would we get it up again?

I am happy to change it in any way necessary; but the Docker image has to be considered a black box for all intents and purposes. There is one docker volume (realised as a directory in the home of the ec2-user right now) if that makes a difference. Timing is not critical.

So far I'm doing shutdowns/startups through the EC2 Management Console, but would like to end that.

AWS has something built in called cloudwatch alarms that can handle the basics.
You could also use a Lambda Function. Give it access to the ASG, and then have cloudwatch configured so once the min amount of traffic starts coming in it sends an Event to the Lambda.
{
  "ASG": "MyASGName"
  "Action": "Start"
}

And once the traffic goes below the min.
{
  "ASG": "MyASGName"
  "Action": "Stop"
}

You could also build on this over time to control Vertical and Horizontal scaling.
EDIT: I actually prefer the Lambda. I always seem to run into these weird use cases where we can't shut down any of the instances, we have to do some math or investigation on which instances to shut down. In these situations a python lambda running boto3 is really helpful.


How do I enable Progress report for Test Plans Sprint 156 update rolled out Progress report in preview (should be an option between Test plans* and Parameters) but I can't seem to find where to enable it. I have turned on all my previews and still don't see it.

According https://docs.microsoft.com/en-us/azure/devops/test/track-test-status?view=azure-devops it should be available by clicking on Progress report under Test Plans.


Why does Docker define invalid IP address of service? I have web-app with PHP and database in docker-compose file:

version: '2'

networks:
  default:
    driver: bridge

services:

  yii2-app:
    build:
      context: ./web
    ports:
      - "80:80"
    networks:
      - default
    depends_on:
      - yii2-db
      - queue
    environment:
      DB_NAME: ${WEB_DB_NAME}
      DB_USERNAME: ${WEB_DB_USERNAME}
      DB_PASSWORD: ${WEB_DB_PASSWORD}

  yii2-db:
    image: mysql:5.7.16
    command: --default-authentication-plugin=mysql_native_password
    networks:
      - default
    expose:
      - "3306"
    environment:
      MYSQL_ROOT_PASSWORD: ${WEB_DB_PASSWORD}
      MYSQL_DATABASE: ${WEB_DB_NAME}
    volumes:
      - "./web/db:/var/lib/mysql"
  ...

When I start services via docker-compose up my PHP-app makes db migrations. But this error occured in database connection:

Exception 'yii\db\Exception' with message 'SQLSTATE[HY000] [1045] Access denied for user 'root'@'172.21.0.8' (using password: YES)'

It's strange, because in my PHP-app I use name of db service yii2-db and docker by itself to define the correct IP.

I connect to my PHP-app container via

docker exec -it passportrecognition_yii2-app_1 bash

and check mysql connection via mysql-client

mysql -h yii2-db -u root -p
ERROR 1045 (28000): Access denied for user 'root'@'172.21.0.8' (using password: YES)

But 172.21.0.8 is not the IP of my db, it's the IP of my PHP-app. That is PHP-app make db-connect to itself.

Database IP is 172.21.0.4 as shown with this ping command:

ping yii2-db
PING yii2-db (172.21.0.4): 56 data bytes
64 bytes from 172.21.0.4: seq=0 ttl=64 time=0.145 ms

And another strange moment: when I connect directly on database IP 172.21.0.4, the following error occurs:

bash-4.4# mysql -h 172.21.0.4 -u root -p
Enter password:
ERROR 1045 (28000): Access denied for user 'root'@'172.21.0.8' (using password: YES)

Why can't I connect to my database? Why does Docker define an invalid IP for the db service?

Ok, I have found the reason:

I tested starting the services with different environment variables for db. And after that I shut the services down, changed the db name, username and password, and started the services again.

But I forgot that db folder /var/lib/mysql is mounted in a volume, and it remembered my previous data; everything was mixed up. So it's not problem in network, it's mixed data in mysql (different passwords for the same user).


How can I push multiple containers, created with docker-compose, to a registry I am using Jenkins pipeline and run a shell command to bring up my linked containers for testing. Once the testing is complete, I want to push the containers to a registry (I would prefer AWS ERC, but at this point I will take any working examples). What is the proper way to do this? Should each image go to a different repository? How can I get all of the containers created by the compose file pushed up? I have been attempting to get this done with the docker-compose push command, but I can't seem to get it working properly or find any decent examples.

Jenkinsfile

node('ec2-build-slave') {
    stage('Docker Build') {
        /* This builds the actual image; synonymous to
        * docker build on the command line */

        sh("docker network create aa_backend_network")
        sh("docker-compose up -d --build")
        //docker.build("$IMAGE")
    }
    stage ('Docker Push'){
        // /* This pushes the previously built image to the Amazon
        // * ECS Container Registry */
        sh("eval \$(aws ecr get-login --region us-west-2 --no-include-email | sed 's|https://||')")
        // docker.withRegistry(ECR_URL, ECR_CRED) {
        // docker.image(IMAGE).push()
        // }

        sh("docker-compose build")
        sh("docker-compose push")
    }
}
catch (exc) {
    currentBuild.result = "FAILED"
    throw exc
}
finally {
    // Success or failure, always send notifications
    notifyBuild(currentBuild.result)
    sh("docker-compose down -v")
    sh("docker network prune -f")
}
}
}

docker-compose

version: "2"
services:
  db_setup:
    networks:
      - default
    build: .
    depends_on:
      - db
    container_name: db_setup
    image: db_setup
    command: python asteri_analytics_db.py
    working_dir: /asteri_analytics_db/asteri_analytics_db
  db:
    networks:
      - default
      - outside
    image: mongo:3.6.3
    container_name: mongo_db
    restart: always
    environment:
      MONGO_DATA_DIR: /data/db
      MONGO_LOG_DIR: /dev/null
      MONGO_INITDB_ROOT_USERNAME: ###edited####
      MONGO_INITDB_ROOT_PASSWORD: ###edited####
    ports:
      - 27017:27017

networks:
  outside:
    external:
      name: aa_backend_network

Just make sure the images you build all have the right names in terms of registry prefix and so on. If not you need to rename them accordingly.

Otherwise this task is pretty easy. To push all Images you build using a docker-compose file just use docker-compose push.

Make sure you are authorised to push to the registry (logged in etc.)


Ansible equivalent of below sed command How can i use ansible lineinfile module to simulate working of below sed command.

sed -i _bkp '/(ADDRESS = (PROTOCOL = TCP)(HOST = 10.0.0.1)(PORT = 1501))/d;s/(ADDRESS = (PROTOCOL = TCP)(HOST = 10.0.0.2)(PORT = 1501))/(ADDRESS = (PROTOCOL = TCP)(HOST = my-site.com)(PORT = 1501))/;/(BALANCE = yes))/d' myconffile

To replace all lines in the file replace module is needed. This is the equivalent (backup differs).

  tasks:
    - lineinfile:
        path: myconffile
        backup: yes
        state: absent
        regexp: '\(ADDRESS = \(PROTOCOL = TCP\)\(HOST = 10.0.0.1\)\(PORT = 1501\)\)'
    - replace:
        path: myconffile
        backup: yes
        regexp: '\(ADDRESS = \(PROTOCOL = TCP\)\(HOST = 10.0.0.2\)\(PORT = 1501\)\)'
        replace: '(ADDRESS = (PROTOCOL = TCP)(HOST = my-site.com)(PORT = 1501))'
    - lineinfile:
        path: myconffile
        backup: yes
        state: absent
        regexp: '\(BALANCE = yes\)'

The simplified version is below.

  tasks:
    - lineinfile:
        path: myconffile
        backup: yes
        state: absent
        regexp: "{{ item }}"
      loop:
        - 'AAA'
        - 'DDD'
    - replace:
        path: myconffile
        backup: yes
        regexp: 'BBB'
        replace: 'CCC'


What are the Devops-related conferences happening around the world? This is a similar question like the Statistics Conferences question at CrossValidated

What are the Devops or Devops-related conferences happening around the world?

Rules:

Include a link to the conference
Please include links for the talks, e.g. youtube, the conference site or some other video streaming site

Velocity Conference is one of the most popular Devops conferences, and also is an O'Reilly conference.

From it's website:

  At Velocity, systems, site reliability, and systems engineers, systems
  architects, and application developers learn to build and manage fast,
  resilient, and highly available websites and apps.

The videos of the conference are present on O'Reilly's youtube channel.