Volumes Not Mounting with Docker-Compose I'm trying to run a SQL Server Database in a container and have been unable to mount a volume using docker-compose. I am able to persist data using docker run, however.

SETUP

docker volume ls

DRIVER              VOLUME NAME
local               db-data-system
local               db-data-user local
docker-containers_db-data-system local
docker-containers_db-data-user

RUNNING DOCKER FROM COMMAND LINE

I'm able to run a container using docker run that persists data (see testdatabase and arbq_app).

sudo docker run -e 'ACCEPT_EULA=Y' -e 'P@ssw0rd' \
  --name 'arbq-db' -e 'MSSQL_PID=Developer' -p 1401:1433 \
  -v db-data-system:/var/opt/mssql -v db-data-user:/var/opt/sqlserver \
  -d mcr.microsoft.com/mssql/server:2017-latest

In dBeaver: SELECT name FROM sys.databases

master
tempdb
model
msdb
testdatabase
arbq_app

WITH DOCKER COMPOSE

docker-compose.yml:

version: '3.6'

services:

  # Another Container
  another-container:
    content: some-content

  # SQL Server
  arbq-db:
    container_name: arbq-db
    image: mcr.microsoft.com/mssql/server:2017-latest
    ports:
     - "1401:1433"
    environment:
      ACCEPT_EULA: Y
      SA_PASSWORD: P@ssw0rd
      MSSQL_PID: Developer
    volumes:
      - db-data-user:/var/opt/sqlserver
      - db-data-system:/var/opt/mssql

volumes:
  db-data-user:
  db-data-system:

I run docker-compose up --build

In dBeaver: SELECT name FROM sys.databases

master
tempdb
model
msdb

docker inspect -f '{{ .Mounts }}' e02bd89ccc38

[{volume docker-containers_db-data-user /var/lib/docker/volumes/docker-containers_db-data-user/_data /var/opt/sqlserver local rw true } {volume docker-containers_db-data-system /var/lib/docker/volumes/docker-containers_db-data-system/_data /var/opt/mssql local rw true }]

I honestly don't know what docker-containers_db-data-user and docker-containers_db-data-system are, but they don't contain the db data I need persisted!

What can I do to ensure that the testdatabase and arbq_app databases persist in the container when using docker-compose?

Thanks in advance.

If you let docker-compose manage the volumes for you, the volumes names in docker-compose.yml will not be the final ones created/used in docker. docker-compose will prepend the volumes names with the compose project name, which is by default the name of the folder holding the compose file. This is to allow separate projects to use the same names without conflict. For information, you will get the same naming convention for networks.

To get your data back correctly in your compose project, you have two choices.

Prefered in your case: Move data to automated volumes.

This will preserve the default compose volume management in your future runs.

stop any running container using any of those volumes
launch a dummy container not having mysql (e.g. busybox) and mount all 4 volumes in separate identifiable paths
empty all volumes created by docker-compose
copy all data from "docker only" volumes to "compose" volumes.
stop and recycle your dummy container.
docker-compose up -d and enjoy.
if all went good and smooth, delete the other unneeded volumes.

If you know what your doing: use external volumes

This will work as well but please check the documentation (Ctrl-f "Volume configuration reference") for details. Specifically, be aware that docker-compose up -d will no longer try to create the unexisting volumes and fire an error and stop if they don't exist.

Simply change you volume section at the end of your file to:

volumes:
  db-data-user:
    external:
      name: db-data-user
  db-data-system:
    external:
      name: db-data-system


How to deploy Dev Azure SQL code to Prod Azure SQL using dacpac without knowing the target server credentials Here is my scenario -

I have access to source Azure SQL Database where i did few changes to schema (eg adding a table). Now I want to deploy this SQL code to prod Azure SQL Database using Azure DevOps dacpac task.

Build pipeline -  I have used MS Build task to generate .dacpac file

Release pipeline - I know Azure SQL server name and Database name but do not know Server admin credentials. How to deploy dacpac to the Prod target server? What are all the tasks required in Release pipeline to achieve this and it should be repeatable process.

Any help would be really appreciated

You either have to have the login information in order to connect to the database, or, you have to configure your database server to allow access to Azure Services. This means that the SQL Database and server and the Azure DevOps pipelines have to be part of the same account (not necessarily the same login, but Azure AD). I'd lean towards requiring the connection information rather than open up everything broadly between all your Azure services.


What exactly is the cloud term 'a Firehose'? I found a Firehose definition from Overview of the Loggregator System Cloud Foundry docs.

  The Firehose is a WebSocket endpoint that streams all the event data
  coming from a Cloud Foundry deployment. The data stream includes logs,
  HTTP events and container metrics from all applications, and metrics
  from all Cloud Foundry system components. Logs from system components
  such as Cloud Controller are not included in the firehose and are
  typically accessed through rsyslog configuration.

  Because the data coming from the Firehose may contain sensitive
  information, such as customer information in the application logs,
  only users with the correct permissions can access the Firehose.

Where does this term have it's roots and why it's called that way? Is the concept the same for other Cloud offerings and platforms?

It's funny when I translate this term to my native language.

The fire hose is a reference to an old Weird Al Yankovick movie, UHF (1989). In this movie, George Newman (portrayed by Yankovick) starts a local TV channel in the UHF band and is responsible for programming on the channel. After befriending the Station's Janitor, Stanley Spadowski, (portrayed by Michael Richards) George gives the Janitor a slot to run a show of his own making. The show, entitled "Stanley Spadowski's clubhouse" becomes an overnight sensation due to Spadowski's zany antics.

During a gameshow segment on the show, (video) Spadowski announces that a little boy, Joel Miller has "won" a contest by finding a marble in a wading pool of oatmeal and "gets to drink from the firehose!" - and this seems to be the likely origin of the term.

An oft-repeated sentiment in IT is that trying to comprehend a vast amount of material in a short amount of time (such as is the case when starting a new job, taking a training course or seminar on a technological subject or learning about a new product or device) is like "attempting to drink from a firehose" - it is nearly impossible to consume the entirety of such a large volume of information that is directed at you with such a rapid flow - a direct reference to the above scene.

It appears that in an attempt at humor, Cloudfoundry has chosen to call their Websocket "The Firehose" because websockets are used to "stream all application event data" (which they apparently regard as voluminous) most likely in reference to this sentiment/saying and scene.


Kafka from client to server vs. "router" in the DC I want to use Kafka as a message broker on the server side of a distributed application.  We have clients that can be located around the world, and that send frequent messages to the server.  Right now those messages are just JSON over ZMQ.  I am considering changing the connection between client and server to Kafka (since I'm going to use Kafka in the server side anyway) but have the following confusion:

Is Kafka even meant to be used this way (distributed clients connecting to the server with it)?  I don't see that in the docs.
If I don't use Kafka, I'll have a single choke point at my server/DC entry.  What's the efficient alternative to get information from the software gateway (which is now ZMQ) into kafka, without having that software gateway be a bottleneck?

Kafka doesn't care how you use it; it does nothing except transporting your arbitrary messages for you. Yes, you can use it that way, and actually I do not see anything specific enough about it to make it necessary to be mentioned in the docs.

Obviously you need to take a  little care regarding security (SSL or SASL...) but aside from that, there should be no particular problem.


Azure DevOps extract unique visitors I am running a version of Azure DevOps and would like to track adoption.

Is it possible to get the number of unique visitors who have logged in / used Azure DevOps in the last 30 days, for instance?

I don't know of a built-in way but here are two hacks:

You can look at https://dev.azure.com/{organization}/_settings/audit , then set the date range to what you want and manually count.

An automated way of doing that could be to use the ADO REST API and send a Query to get the Audit log, then use a bit of code to tally up the unique users from the response.


How to render assets when building a static website with GitLab Pages and Jekyll I'm using Jekyll to build a static site and deploy it with GitLab Pages. This site is compounded of posts (contained in the site/_posts folder) and refer to other HTML and Markdown documents that I placed in an assets folder at the project's root, as recommended by Jekyll docs.
When I build the website in the pipeline it runs and deploys it successfully to GitLab Pages. However, if I am able to find my posts, the links towards the files allegedly contained in assets return a 404. When I browse the public artifacts file, I can't see said files in the assets folder, even though I put them there before building the site.
What am I missing here? It runs fine when serving locally but as soon as it is deployed on GitLab Pages I can't access the assets file.
project architecture:
site/
|__> _posts/
|____> entry.md
assets/
|__> v{{ version }}/
|____> yellow-ducky.html
|____> black-ducky.md

.gitlab-ci.yml excerpt:
pages:
  stage: deploy
  tags: [docker-connect]
  script:
    # entry.md is the blog post I want to publish
    - mv entry.md site/_posts/`date +'%Y-%m-%d'`-entry.md
    # reports come from a previous job's artifacts and contain all files I want to place in assets
    - mkdir -p assets assets/v${TAG}
    - mv reports/* assets/v${TAG}/
    # build website
    - gem install bundler
    - cd site
    - bundle install
    - bundle exec jekyll build -d public
    # move built files to the folder where GitLab expects to find them
    - cd .. && mkdir -p public && mv site/public/* public/
  artifacts:
    paths:
      - public

_config.yml excerpt:
title: RubberDucky
email: rubber@duck.io
description: >- # this means to ignore newlines until "baseurl:"
  This is the place where to find all info you need about rubber ducks.
baseurl: "/toys/rubberduck" # the subpath of your site, e.g. /blog
url: "https://gitlab.troubleshoot.io" # the base hostname & protocol for your site, e.g. http://example.com
# Build settings
theme: minima
plugins:
  - jekyll-feed

blog post excerpt:

    {% for duck in ducks %}

{{ duck.name }}

View | Download

    {% endfor %}

It appears the issue is in the URL used to access the documents in the blog post excerpt. First of all, the URL should begin with a / since the assets folder is at the root of the site project. However (and most importantly), I also had to add the baseurl to the file URL as shown below:
blog post excerpt:

    {% for duck in ducks %}

{{ duck.name }}

View | Download

    {% endfor %}

If I weren't building my post entries from a template the link would actually be /{{ site.baseurl }}/assets/v{{ version }}/{{ duck.url }}, but Jinja does not understand the site.baseurl object when rendering the template. To bypass this, I read the value of the predefined GitLab CI_PROJECT_PATH variable and pass it to the template under the name baseurl when rendering.


To design a blue green deploy how to publish websocket traffic from live to hot swap server A blue green deploy involves pumping live prod data flows (blue) into a hot swap non prod environment (green) in preparation to deploy to live the green environment, such that the green has full data sync with the previous prod blue environment.

I am wondering what people are using to live copy ongoing websocket traffic from blue to green do I write my own or is there a publish/subscribe websocket library or possibly a different approach to blue green?

My app has nodejs REST servers which also manage websocket traffic from mobile devices ... a mongodb server, etc ... each in containers on GCE/AWS

I realize I could just keep the mongodb in sync from blue to green however that would not exercise the green nodejs servers with live traffic which is a nice regression sanity check I am looking for

Possibly if I just live forward HTTP traffic the underlying websocket which runs atop HTTP will just take care of itself and not demand specific blue green setup

Websockets don't really have any handling for this per se. You could build your own system but generally the idea for HA is that if anything happens you'll set the client up to automatically reconnect. There will be some non-zero downtime during a blue-green swap but if your app is well built it should handle that already.

If you wanted something more smooth, you could build a message in to your protocol to connect to a new server and then drop the original connection.


API Gateway - How Deploy API works? After creating API Gateway with two API names and integrating with lambda function,

AWS documentation recommends to deploy this API, as shown below:

1) What does deploy API mean? How creating API gateway different from deploying API?

2) Does deploy API option internally create Cloud formation template? that creates a stack and deploy

1.) Create API (create-rest-api from the cli) simply creates the API Gateway resource, but it does not create the method, stage, authorizer, etc. for your API Gateway.  Deploy API (create-deployment from the cli) will deploy either to an already created stage, or create a new stage for deployment.

2.) No, it will not deploy a CloudFormation stack in your account.


Initialize database with data I'm setting up Build Automation for a MS-SQL project.  One of the requirements is to initialize the database with data.  I have them as SQL scripts.  I'm trying to figure out what the best way to inizialize a database after it is deployed using VSTS build and release automation.

Depends on where the source data is when considering how you want to initialise the database.

If the data pre-exists in another SQL-Azure Database you could create a BACPAC and export the file to a blob store in Azure. Then using PowerShell in your pipeline import the data into your created database. Bare in mind this will be time costly dependent on the amount of data and the size of the database.
If the data is only small, an approach we have used before to build and populate tables is a DACPAC. Here we had the create scripts for our tables and then stored procedures with static reference data stored in insert statements which were run after the DAC built the tables & indexes etc. Here is an example in how to create a DACPAC project in Visual Studio and populate from an exisiting database. Alternatively if you're building from scratch you can add the required database objects to the solution and create them using SQL Statements.
The third option would be streaming data from a source using an Azure Function and blob storage however without knowing more about your set up I can't advise more.

Hope that helps


How to properly scale Jenkins? In my project we have one AWS server running Jenkins Master + 1 Jenkins slave (2 executors)... and we need more
In order to augment our build power we have three options:

Scale up: Make AWS instance bigger and add more executors.
Scale up: Make AWS instance bigger and add another jenkins slave process.
Scale out: Create another AWS instance with a jenkins slave and connect it to master

We want to do 2. as we are in a big organization and our current Jenkins Master has already access to every place he needs. Option 3. "New server" is complicated as it needs more bureaucratic approvals that will take weeks.

So my questions are:

Is there any technical issues in option 2?. Maybe the executors of each jenkins slave are not aware of the other slave executors?
In general, what is the best approach to scale Jenkins? Scaling up or scaling out?

There are no fundamental technical issues with running multiple jenkins slaves on the same machine. In fact Running Multiple Slaves on the Same Machine lists several good reasons for doing it:

  While the correct use of executors largely obviates the need for
  multiple slave instances on the same machine, there are some unique
  use cases to consider:

  You want more configurability between the configured nodes. Say you have one node set to be used as much as possible, and the other node
  to be used only when needed.
  You may have multiple Jenkins master installations building different things, and so this configuration would allow you to have
  slaves for more than one master on the same box. That's right, with
  Jenkins you really can serve two masters.
  You may wish to leverage the easiness of starting/stopping/replacing virtual machines, perhaps in conjunction with Jenkins plugins such as
  the Libvirt Slaves Plugin.
  You wish to maximize your hardware investment and utilization, at the same time minimizing operating cost (e.g. utility expenses for
  running idling slaves).

In general scaling out is preferred, primarily because ability to scale up is typically limited by the types/sizes of the available physical resources.

In particular for augmenting build power I'd recommend an analysis of your actual build  to determine how it uses the machine resources, which/where its bottlenecks are and what scalability limitations it raises to reveal if scaling up even helps.

For example I encountered cases in which the build time for 2 parallel builds on the same machine was longer than the combined build times of the same 2 builds executed sequentially (non-overlapping) on the same machine. In such case I wouldn't even consider scaling up as it would actually decrease the overall building capacity.


What are sources for configuration management best practices? I'm struggling with design for configuration management for a complex on-premise enterprise application with lots of moving parts. While it's fun and joy to go through this, there are lots of questions I'm really not sure of, and I need some source to refer to as best practices or just revise my design solution.

It might be I'm really bad at searching the internet, but whatever materials I've found, it's all about very basic examples, like "let's configure yet another web application behind nginx". Which feels like writing microservice application based on "hello world" example.

Example of issues I'm struggling with:

Configuration usability vs configuration complexity
Self-written solution vs open-source adaptation vs combination of those two
Pre-compiled configuration package vs dynamic template on actual deployment
Ways for a quick configuration roll-back

And others.

I do understand, that there won't be a silver bullet which answers all the questions I have, but at least it'll give me a hint if I'm moving in a right direction.

So what sources I should look into?

Well, that's pretty broad, mostly this can boils down to two options:

Policy based management
Promise Theory

In the first one you're describing the actions you want to be done to achieve the desired state, assuming previous state of the system or testing against it specifically, in the second you're describing what you expect as resulting state of the system and your tool should take care of doing what is necessary to achieve that.

A policy based management could be made of simple scripts, doing what you want, Promise Theory tools are inherently more complex (Ansible, Chef, Puppet, Salt) but they're mostly idempotent, this means they will only change something if the actual state doesn't match the desired state.

The only question which can be given a really definitive answer out of thin air without knowing the specifics of your application/service is to the second question:

Don't reinvent the wheel, there's serious softwares for configuration management nowadays that rewriting your own would be a road of pain for less functionalities than already existing in available softwares. There's of course a learning curve whatever path you choose, but the effort to learn a maintained piece is more worthy than the effort to write your own system and maintain it on top of the application configuration you'll have to manage with it.


Jenkins run build in docker on server (withDockerServer not working) I am trying to run my build inside a docker; but I don't want the docker to run on the jenkins master but on another server that has docker running.

What does work:

inside jenkins configure a docker Cloud to connect to docker server (using tcp://my-server:4243)
configure a docker agent template for certain docker images and assign labels to those docker containers (these need to be containers with sshd running; java installed etc such that they can be connected as jenkins slaves)
in my build I specify that I want the build to run on that label

What does not work:

pipeline {
        agent any
        stages {
                stage('test') {
                        steps { script {
                                withDockerServer([uri: 'tcp://my-server:4243']) {
                                        withDockerContainer([image: 'alpine']) {
                                                sh "whoami; pwd; ls /usr/local/bin"
                                        }
                                }
                        } }
                }
        }
}

in this case I get errors:

Caused: java.io.IOException: Cannot run program "docker": error=2, No such file or directory

since docker is not installed on my jenkins master; this means he is trying to execute the docker image on my jenkins master and not on my-server... any tips?

I am using:

jenkins 2.150.3
docker plugin 1.1.6
docker pipeline 1.17

EDIT
even with scripted pipeline this does not work:

node('master') {
        withDockerServer([uri: 'tcp://my-server:4243']) {
                withDockerContainer([image: 'alpine']) {
                        sh "whoami; pwd; ls /usr/local/bin"
                }
        }
}

he is still ignoring withDockerServer (it is shown in the pipeline.log, but the docker command is still run on jenkins master)

apparently docker needs to be installed on jenkins master in order for jenkins to be able to launch the docker on the server.

After installation of docker (even without the docker daemon running on jenkins master); jenkins was able to launch the docker on my-server.


AWS subnets for lambda synchronous invocations Am I right in thinking giving a lambda using an API gateway 3 subnets each with a range of 16 would result in 2 subnets being used at once and 1 IP used for the API gateway that is in the same subnets resulting in a max of 31 invocations before the error code 429 is returned due to Too Many Requests.
This is my thinking as there should be no other resources using an ip address that will be in these subnets but I can see testing it as being complicated and taking a very significant amount of time with my current knowledge.

I don't entirely understand your use-case, but it sounds like you are trying to limit the number of Lambda invocations by restricting its IP range.
There are two better ways to rate limit your application. Option one would be to use API Gateway request throttling:
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-request-throttling.html
Option two, Lambdas have a concurrency limit that can be set to achieve a similar behaviour to your approach:
https://aws.amazon.com/about-aws/whats-new/2017/11/set-concurrency-limits-on-individual-aws-lambda-functions
Hope that helps with your design!


How effective is ELK than other Monitoring tools? "ELK" is the acronym for three open source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine. Logstash is a server‑side data processing pipeline that ingests data from multiple sources simultaneously, transforms it, and then sends it to a "stash" like Elasticsearch. Kibana lets users visualize data with charts and graphs in Elasticsearch.

 How effective is ELK than other Monitoring tools?

ELK is pretty ineffective as a monitoring tool in a default configuration as it doesn't do monitoring - only reporting. ELK can only provide monitoring and alerting abilities if you configure additional add-ons such as ElastAlert.

Then, you are impacted by your ability to log errors via syslog. If messages come in via, say, SNMP traps, you will be unable to alert on these if you do not configure the appropriate plugin.

Like most monitoring systems, your effectiveness is determined mostly by 1) your input method and 2) if you have told the system to alert off of all of the correct messages.


How to disable some branches in Multibranch Jenkins Builds? I have a multibranch job set to run any branch with a Jenkinsfile.

I have some options I can think of if I want to remove a branch from the list of jobs running for the multi-branch pipeline.

I can delete the branch
I can delete the Jenkinsfile in that branch

The second solution is good, except I need to commit and push that to the git repo for my branch, and if that branch is merged into another branch it blows away the Jenkinsfile.

What is the best way to disable only some branches of a multibranch pipeline?

Jenkins can filter branches in a multibranch pipeline by name using a wildcard or regular expression.


Ansible - Handler not found on first attempt Command run: sudo ansible-playbook site.yml --connection=local

Distributor ID: Ubuntu
Description:    Ubuntu 18.04 LTS
Release:        18.04
Codename:       bionic

ansible 2.5.5

I'm encountering this error only on my first run through after I change the following variable in the following file:

./group_vars/all

ntpserver: pool.ntp.org

TASK [common : Configure NTP file] **************************************************************************************************************
ERROR! The requested handler 'restart ntp' was not found in either the main handlers list nor in the listening handlers list

On my second run through, I'm having no issues:

TASK [common : Configure NTP file] **************************************************************************************************************
ok: [127.0.0.1] => {"changed": false, "checksum": "cc9e1a2d8777588fa4675e52f7ca7e352b23188a", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 139, "state": "file", "uid": 0}

The folder and file structure I'm using is as follows:

https://github.com/ansible/ansible-examples/tree/master/lamp_simple_rhel7

./roles/common/handlers

- name: Restart NTP
  service: name=ntpd state=restarted`

./roles/common/tasks

---
- name: Install NTP
  apt: name=ntp state=present
  tags: ntp

- name: Configure NTP file
  template: src=ntp.conf.j2 dest=/etc/ntp.conf
  tags: ntp
  notify: restart ntp

- name: Install common packages
  apt: name={{ item }} state=present
  with_items:
    - fail2ban
    - ranger

- name: Start the NTP service
  service: name=ntp state=started enabled=yes
  tags: ntp

- name: Upgrade all packages to the latest version
  become: true
  apt:
    upgrade: yes
    update_cache: yes
    cache_valid_time: 86400 # one-day

Name of the handler "Restart NTP" does not fit the handler notified "restart ntp". This is the reason of the error. On the second run the configuration file /etc/ntp.conf did not change ("changed": false). Therefor the handler was not notified, was not looked for and no error was reported.


Ansible cloudformation vs cloudformation I would like to know the difference between
- AWS cloudformation
- Make AWS cloudformation template through Ansible (https://docs.ansible.com/ansible/latest/modules/cloudformation_module.html)

I dont know why would i use Ansible if i already have AWs cloudformation

You wouldn't need to use Ansible for AWS cloudformation if you are fine with just using AWS cloudformation.

Ansible integrated AWS cloudformation so that you can convert your local Ansible files into an AWS cloudformation easily.

I'd suggest using Ansible since it provides a great way to test your stacks locally and then you would make the conversion to AWS cloudformation when ready. This is usage dependent however.


I'm trying to run `vagrant provision` on PowerShell Window for Ansible playbook Here is what it replies:

Windows is not officially supported for the Ansible Control Machine.
Please check https://docs.ansible.com/intro_installation.html#control-machine-requirements

What should I do? Thank you

You have 2 viable options to run ansible on windows 10:

Install a linux vm with your favorite virtualization tool and run ansible inside that vm.
Run ansible from WSL

With the first option, you will need to install vagrant inside the vm + all virtualization tools needed.

With the second option, you will then need to install vagrant in WSL as well and configure it to access your windows environment.


How to convert a docker-compose file to kubernetes file(s)? Is there any easy way to convert a docker-compose.yml file to Kubernetes file(s)?

The original and most popular solution I've seen is kompose which does exactly as you are asking, converts the compose yaml file into kubernetes yaml files.

If you want to continue working with a compose.yml file for compatibility with other systems, then docker recently open sourced their compose on kubernetes solution that integrates directly into kubernetes.


Is it possible to deploy RDS in multiple region in AWS with read and write replica I have an application that I need to make it accessible across 4 regions, I saw blogs and other posts in the internet that speaks about the read replica of AWS RDS MySQL Server, is it possible to enable write operations in the regions so that the data can be in sync and be writable across all the regions.

You can look at Aurora Global Database for this. Based on this news announcement, you can also configure the setup, so you can write to your local db cluster and this is forwarded to your primary region.
Of course there would be some lag in write requests, essentially of the length of time it takes to write to the primary region and replicate the data changes back to the replica in your local region. Nonetheless this is likely easier to handle (and with the same methods in a web application), as if you wrote to the primary region yourself and checked if the data had arrived back in local replica.


is a running container created by each RUN command during multistage build? I'm new to Docker, sorry if my question sounds dumb.
below is a dockerfile from a textbook:
FROM diamol/base AS build-stage
RUN echo 'Building...' > /build.txt              #line 2

FROM diamol/base AS test-stage
COPY --from=build-stage /build.txt /build.txt
RUN echo 'Testing...' >> /build.txt              #line 6

FROM diamol/base
COPY --from=test-stage /build.txt /build.txt
CMD cat /build.txt

and the author says:
The RUN instruction executes a command inside a container during the build, and any output from that command is saved in the image layer.
My question is:
Since the author mentioned RUN instruction executes a command inside a container during the build, does it mean that in the first RUN command in line 2, a contianer is created based on the diamol/base image, and this container modify the content of the files then commit the change back to diamol/base image, then is container is terminated? and same thing happen in line 6,container created to run and then terminate, image gets updated, is my understanding correct?

Yes your understanding is correct. The RUN <commands> command in the Dockerfile is somewhat equivalent to
container = $(docker run base-image) #1
docker exec base-image commands #2
docker commit $container #3

Since images are static, we'll need a running instance/process to execute our command. this is why a container is created from the base image. #1
After the execution of the command in #2, the changes are committed back #3, which means a new layer is created on top of the previous layers. This layer will contain the changes introduced by the container in (#2).


Parallelizing dependent CI subjobs Imagine you have resources for a job which has part 1 and part 2.

Part 2 requires part 1, but both take considerable time.

And, part 2 requires part 1 only at the end.

Would you start them both in parallel and let part 2 somehow wait for part 1 getting ready? (chances are it 50/50 it will be ready by then)

The critical question is - can you teach your job #2 to patiently wait for job #1 to complete when it gets to that dependency point and appropriately proceed or bail depending on job #1 result?

If you can't, then the answer is no - you'll have more trouble dealing with #2 failures due to incomplete job #1, too much noise in the process.

If you can (and if you have enough resources) - yes, by all means - your process will be faster. You might not even need special support from the CI system for such dependencies, it could be completely oblivious, it would simply see the 2 jobs as parallel.

A middle-ground possibility, if your CI systems supports it, would be to stagger the jobs: start job #2 with a delay sufficient to bring the chances of job #1 completing before job #2 needs it in a much more comfortable range (say 99%). The delay would be plain eyeballed or empirically determined, either fixed or, in a more advanced approach, automatically derived from historic measurements/stats (if available).


Should worker processes of web applications be placed in separate containers? My setup is based on a Flask app that has a PostgreSQL and Redis database. In addition, a worker process is started to handle asynchronous tasks such as sending emails. I use Gunicorn to create multiple app processes. Passing of jobs from the app processes to the worker process is done via Redis.

The question is, should the worker process be started in a separate container despite being based on the same code? Currently I use a start script with the relevant bit:

gunicorn --bind=0.0.0.0:8000 --workers=3 manage:app &
python -u manage.py run_worker

Also if they are to be separated, how can I ensure that they reuse the common file system container? Additionally, when scaling either Gunicorn with the main app or the worker processes, should additional instances of the container be created or the number of processes within a container be increased?

Related: What are the advantages of dockerizing nginx and php in different containers?

I can think of only one case where overhead of running workers in separate containers is justified: if your setup uses docker swarm for clustered deployment. This way you will get all of the HA benefits.

Otherwise I don't see a reason to complicate such tasks, especially if they must use strictly the same codebase (which leads me to believe that they share one domain of concerns, so to say). It might seem elegant, but I just don't see enough benefits

p.s. any split of such kind will force you to manage interconnection and complicate setup for local development


Why container can use ifconfig to get the host information? Today I pull an image from the Internet, and I found that the docker networking mode can be detected in the image's startup script.
But docker0  only exists in the host  not the container. Why?
Dockerfile

FROM ubuntu:18.04

RUN apt-get update \
    && apt-get install -y  net-tools   \
    && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

ADD test/ /etc/test/

RUN chmod +x  /etc/test/*.sh

CMD ["/bin/bash", "/etc/test/start.sh"]

start.sh

#!/bin/sh
set -e

# check for presence of network interface docker0
check_network=$(ifconfig | grep docker0 || true)

# if network interface docker0 is present then we are running in host mode and thus must exit
if [[ ! -z "${check_network}" ]]; then
    echo "[crit] Network type detected as 'Host', this will cause major issues, please stop the container and switch back to 'Bridge' mode" | ts '%Y-%m-%d %H:%M:%.S' && exit 1
fi

I run it with  docker run --privileged  --net=host will get

[crit] Network type detected as 'Host', this will cause major issues, please stop the container and switch back to 'Bridge' mode

You are not accessing the hosts network interface information. Rather you are receiving data of the bridge adapter created for your Docker Containers Network.

Please note the results from my web server:

Container networking info:

# docker exec -it ghost sh
/var/lib/ghost # ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:AC:12:00:02
          inet addr:172.18.0.2  Bcast:172.18.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:20 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1536 (1.5 KiB)  TX bytes:0 (0.0 B)

Host network info:

root@li293-xxx:/data/mydomain.us# ifconfig
br-08d99d9f172e: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 172.18.0.1  netmask 255.255.0.0  broadcast 172.18.255.255
    inet6 fe80::42:57ff:fe74:c49b  prefixlen 64  scopeid 0x20<link>
    ether 02:42:57:74:c4:9b  txqueuelen 0  (Ethernet)
    RX packets 150  bytes 16535 (16.5 KB)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 234  bytes 21425 (21.4 KB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
    inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
    ether 02:42:82:3d:17:63  txqueuelen 0  (Ethernet)
    RX packets 0  bytes 0 (0.0 B)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 0  bytes 0 (0.0 B)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 66.xxx.xxx.205  netmask 255.255.255.0  broadcast 66.xxx.xxx.255

Please review the networking documentation at https://docs.docker.com/v17.09/engine/userguide/networking/


Should development tools be installed on the host or in an environment standardized at project level? On a development host, do you consider it still worth installing the development tools (compiler, interpreter, package manager, testing tools, ...) on the host? Or are the developers supposed to interact with those tools through an environment1 standardized at project level to ensure a consistent development stack across the team and with the production environment?

1local virtual environment, chroot, container, remote login on a compile farm, ...

Note: somewhat related to a previous
question

At many companies (eg banks) the provenance of the code that gets out to live needs to be known and traceable. This is typically done by having a controlled build farm that builds from source. Release builds upload the binary artefacts to a location where they cannot be overwritten. The artefacts are then deployed by any number of mechanisms that are traceable to an individual and an approved change request. This model is generally just referred to as a ”pipeline”.

Under such a model a ”dev host” or ”dev environment” can have anything and everything that developers want on it. It can use newer versions of software than are available in production. It's entirely down to the context whether that is going to cause problems with code failing testing in a live-like environment. It's typical to put release build code through at least one test environment before pushing it to live. That environment is kept as similar to the live environment as possible.


What is the best tool for the release management for Kubernetes in a small startup? In our small startup (6 developers) and we use Google Kubernetes Engine + CircleCi (which deploys docker images to K8S with simple bash scripts). We have ~10 services, but we are growing. We use a simplified git-flow model. W plan to use in the future: terraform for infrastructure, helm charts for managing deployments.
We want to have a possibility:

(now) to track our releases in every environment
(now) manually rollback / automatically rollback after seeing some alerts in grafana. Rollback must apply Django migrations first and then use the image from the previous deployment.
(now) release a set of services in one go
(future) use some more advanced development techniques like canary deployment
(future) have some metrics regarding deployments (eg. build time)

There are plenty of tools which could be used for this job and it's hard to choose

Spinnaker -> it is said that it needs 3 engineers to maintain it, so probably it's too complex.
Jenkins -> it's more CI/CD tool.
Harness/Argo/Google Deployment/Team City/Code Fresh -> maybe one of those?
Write a simple inhouse solution in Django -> the most flexible one, but also the most error-prone.

What do you recommend?

GitLab is a solid good option for most of this (with the appropriate plan level - I don't think the CE one has everything).  It's a single-pane-of-glass for your code, security, CI/CD, metrics, etc.  You can create and deploy to kubernetes clusters from it.

You can have multiple environments and it'll store the secrets for you to customize them.
It will monitor your kubernetes apps and show deployment marks so you can see if they get better/worse performance wise before and after deployment.
You can do multi project pipelines and actions in parallel to chain things together and get things done fast.
It supports review apps and canary deployments.
It has rollback buttons and tracks every release you do.  It also has a good API, so I assume you could hook up prometheus + alert manager and make the API trigger a rollback in response to an alert.

Spinnaker has https://www.armory.io/ if you're willing to pay a bit (not sure how much).  But that should make it easy to use / effective.

I'm not amazingly fond of Jenkins, but it can do everything.  Its hard to maintain over time / with bigger projects and teams though.  But it can be customized to do anything.  You may want to look into https://jenkins.io/projects/blueocean/ (the blue ocean plugin) for CD if you're leaning this way - it looks fairly impressive and I've heard good things.


How to find future (planned) versions of Docker? I need to know a rough release date for the next Docker version(s) or the version scheme they are using for their releases (like "a new version every 3 months" or similar).
So I can plan updates in advance to reserve time for them.
Where might I find information about upcoming Docker releases?

I'll assume you're talking about Docker Engine for Linux.
Docker doesn't plan releases like that currently. What they do is:
Release major and minor versions on GitHub: https://github.com/moby/moby/releases
Put release notes in their docs: https://docs.docker.com/engine/release-notes
The version number is date based on the month they begin the release process, so the current version "20.10" started its release process in October, 2020 and was eventually released in December, 2020. The last major release before that was 19.03.
Finally, you can see what they are working on (features) and what's next on their Roadmap https://github.com/docker/roadmap/projects/1


Is OpenStack the same type of software as Azure, AWS, etc.? I'm not sure how to ask it.  Is something like OpenStack the same thing as AWS and Azure, except it is Open Source?  If I wanted to create an onsite cloud for a Production environment, can OpenStack (or other) be used?

Generally yes - all of them are large ecosystems of related software products working together to implement "clouds".

There won't be an exact 1:1 mapping for each and every one of their individual products, features and APIs, so don't expect that. But there is a quite significant overlap in capabilities and functionality in general. Just like with the public cloud providers: they're similar to each-other, but not exactly 1:1 compatible.

You (and your customers) will have to learn the specifics of using the desired cloud products in the ecosystem, just like with the other cloud infra providers.

But you'll also need to learn how to install, maintain and operate the infrastructure in general as well as the individual products you're gonna offer, which is something users of the public cloud providers don't need to worry about.


Where to start? Scaling PHP applications on AWS Currently all of our PHP (Laravel) applications are running on one EC2 instance. To deploy code, we SSH onto the live box and pull from CodeCommit.

We now need to scale and deploy properly.
I'm not sure where to start.

Most examples seem to use Elastic Beanstalk but I have been advised that this could be too restrictive.

I guess we want to end up with a load balancer and application servers which are created on demand.

Can anyone point me in the right direction? (I am a bit overwhelmed by all the AWS documentation.)

Ok, this is the plan:

I have signed up for A Cloud Guru and I'm going to work through some of the videos on there.

I'm going to sign up for a AWS Free Tier account and start tinkering.

I'm considering the AWS Certified Solutions Architect - Associate  course, this looks like a great way to develop myself.

Here's a rough approach to scaling any stateless app on AWS:

Run the app in an Auto Scaling Group (ASG). An ASG makes it easy to manage multiple servers, will automatically replace failed servers, and allows you to automatically scale the number of servers up or down in response to load.
To run an ASG, you need to create a Launch Configuration. This is a "template" that specifies the configuration for each server in the ASG: i.e., what type of instance to run (e.g. t2.micro), what AMI to run on the instance, the security group settings, etc.
The AMI you run on each instance in the ASG should have your app already installed (i.e., your PHP app). You can define your AMI as code using a tool like Packer.
The app in your AMI should be configured to run on boot using a process supervisor (e.g., using systemd or supervisord; this will also ensure the app is restarted automatically if it crashes.
Every time you change something in your app, run a build process to create a new AMI, update the Launch Configuration to use this new AMI, and roll that AMI out across your ASG. For example, you could do a zero-downtime deployment (somewhat similar to a blue-green deployment) by doubling the size of the ASG, waiting until all the new servers (from the updated Launch Configuration) launch and pass health checks, and then reducing the ASG back to its previous size (the ASG will terminate oldest instances by default).
With an ASG, you have more than one server, but you want to give your users a single endpoint to use. The typical solution to this is to deploy a load balancer in front of the ASG. If you're building a typical HTTP/HTTPs app (e.g., just about all PHP apps), you'll want to use the Application Load Balancer (ALB); if you're building an app that listens directly for TCP connections, you'll want the Load Balancer Classic or Network Load Balancer. Whichever load balancer you pick, AWS will create a domain name for it that you can use, and any requests you send to that domain name will be routed in round-robin fashion to different servers in your ASG.
You should a) configure the load balancer to perform health checks on your app so that it only routes to healthy servers and b) configure your ASG to register all of its servers in the load balancer automatically and to use the load balancer for health checks.
You can create Auto Scaling Policies for the ALB that change the number of servers in your ASG depending on various CloudWatch metrics. For example, you could configure the ASG to add servers when average CPU usage is over 80% and to remove servers when CPU usage drops below 60%.

For more info, check out A Comprehensive Guide to Building a Scalable Web App on Amazon Web Services.


How to run multiple windows in cmd Wondering if possible to run multiple windows in command script. I have a nose.js file that automates an online process and wish to run it once in cmd so it has multiple windows doing the same thing for efficiency purposes. Please help

node.js runs via a command line (no windows involved). If you want to run more than one copy of the same program you can simply start it more that once.
Take this simple one:
https://nodejs.org/en/docs/guides/getting-started-guide/
Since it runs on a dedicated port (3000) if you start it more that once it will fail.
Here is something I found on google that details all the ways to do this:
https://itnext.io/4-solutions-to-run-multiple-node-js-or-npm-commands-simultaneously-9edaa6215a93


vagrant box tests in gitlab-ci I have a vagrant box, provisioned with Ansible that needs testing on github pull requests. Is it possible to setup CI test suite for the mirror in gitlab-ci without extra expenses on droplets? Or maybe this is something that could be possible with upcoming automation on github? Big fan of travis but looks like they not going to support virtualbox deployment in their containers anytime soon.

You can do via docker, since it's a test.
Here is an example repository, both docker and vagrant.
https://github.com/jonashackt/gitlab-ci-stack
https://blog.codecentric.de/en/2018/05/gitlab-ci-pipeline/


How do I create a Docker container which just waits to be shelled into? I'm testing a Docker container for development. And at the moment I just want to provision it with some libraries. After which I want to connect to it and test that I have all the libraries I need.

At the moment, my Dockerfile is just

FROM ubuntu:18.04

RUN apt update
RUN apt install -y git curl cmake libx11-dev libglu1-mesa-dev libxrandr-dev libxinerama-dev libxcursor-dev libxi-dev zlib1g-dev libasound2-dev libgtk2.0-dev libjack-jackd2-dev jq

But once it finishes building it terminates.

What CMD should I put at the end to keep it alive so I can shell in from another terminal?

Disclaimer: This is very much an anti-pattern since it looks like you are trying to create something similar to a VM, or at the very least a pet that you modify in place rather than making all your changes via code and redeploying a new container for each change.

That disclaimer aside, you just need a command that will hang indefinitely. The most common one I've seen in a tail -f /dev/null, e.g.:

FROM ubuntu:18.04

RUN apt update \
 && apt install -y \
      cmake \
      curl \
      git \
      jq \
      libasound2-dev \
      libglu1-mesa-dev \
      libgtk2.0-dev \
      libjack-jackd2-dev \
      libx11-dev \
      libxcursor-dev \
      libxinerama-dev \
      libxi-dev \
      libxrandr-dev \
      zlib1g-dev

CMD [ "tail", "-f", "/dev/null" ]

I've also rearranged the Dockerfile to merge the update and install commands, this is a best practice for Dockerfile's since it avoids having a stale apt update command used from the cache when you change the apt install command months later. Also, putting each package on a separate line is better for version control to see what changed if you add a single new package.

When using this, you would docker run -d --name your_container your_image to run it in the background, and then docker exec -it your_container /bin/bash to open a shell.


How to authenticate, with azure devops services, using SAML via on premise standalone java application? We have a network where azure DevOps services can be used by logging in via SAML. There is an independent java application in that same network.

Can anyone please point out a way that this java application can get authenticated to Azure DevOps services via SAML, without any manual intervention?

PS: I've explored a lot of blogs but I couldn't find what I need specifically. Closest I got, was to register my java application as a AD tenant, but in our use case this can't be done.

Is there any other alternative? Please suggest.

There is a fairly sizable range of authentication options, based upon what you have said you are looking for a non-interactive login which rules out some options, and leaves the following:

Device Profile - probably not ideal for you given you would need to find a way to this in Java as the examples are few and far between and mostly written in C#.
TFS Application - again not ideal given you would need to either find someone who has written a 3rd party client library for Java or write one yourself.
Azure DevOps Extension - while you could write an Azure DevOps extension, and use that to access Azure DevOps it's a lot of work and potentially would expose security issues.

Given that we have probably ruled out non-interactive login to Azure DevOps you could also explore one of the two following options:

Interactive Logins - is there an option in your environment from forcing the user to authenticate with Azure DevOps using OAuth then capture the token for non-interactive use?  This is quite a common pattern with GitHub, GitLab and Bitbucket.
Personal Access Tokens - mostly this would be abusing a feature of Azure DevOps as these are designed for users not applications, although it could work in your constrained environment.
Non-interactive Login using your Identity Provider - you haven't detailed who your Identity Provider is, but it is possible it supports authentication using Client Certificates, Shared Secrets, etc. which would give you a SAML token which could be exchanged for an Azure DevOps Access token.

Hope those help.


Jenkins libraries configuration in Jenkinsfile Right now we have many projects with similar Jenkinsfile's. Instead of copypasting we're developing a shared library with some utilities. This library is loaded dynamically in each Jenkinsfile, and has some parameters that repeat among calls. For example, an utility has a custom Slack message posting function, and the channel, bot token, and some fixed text are always the same.

Instead of providing those parameters to every call to this Slack function we have thought of making a config file and loading it globally at the beginning with all these "constant" parameters.

We tried putting a config.json and using the readJSON step like this:

def load(path) {
    env.CONFIGURATION =  readJSON(file: path)
}

So the CONFIGURATION is an environment variable and so it's global. The problem is that it seems that environment variables can only be strings, so the JSON is broken (we cannot do env.CONFIGURATION.slack.channel for example)

We tried the following also in Jenkinsfile:

def CONFIG = readJSON(file: path)

But CONFIG then is not visible inside the library scripts.

So now, the questions:

Is our approach correct? Putting all common config in one file and
loading it for the library scripts to see.
If it's not correct, how would you deal with our problem? Simply
passing the config as parameter on every call and that's all?

Thanks in advance for your help.

You should look at Global Shared Libraries, which will allow you to reuse logic across different Jenkinsfiles.

Rather than depending on the environment, it's probably safer to explicitly pass parameters and returns values from the functions, which will make their behavior more explicit. Another side benefit is that you will be able to unit test them using


No Checkbox to" Restart Jenkins when installation is complete".Unable to restart Jenkins post plugin download I have set up Jenkins version (ver. 2.155).

post adding plugin, i need to restart Jenkins but "Restart Jenkins" tab or link not working. I don't find my [] checkbox to check/tic click on arrow is not working (Refer image). Any help will be appreciated

To restart Jenkins manually, you can use either of the following commands (by entering their URL in a browser):

(jenkins_url)/safeRestart - Allows all running jobs to complete. New jobs will remain in the queue to run after the restart is complete.

(jenkins_url)/restart - Forces a restart without waiting for builds to complete.


Unable to launch multiple EC2 instances in different availability zones using terraform Whenever I try to launch the instances, I am getting the following error message. Please help with this.

Error: Error applying plan:

3 error(s) occurred:

* aws_instance.example[0]: 1 error(s) occurred:

* aws_instance.example.0: Error launching source instance: InvalidParameterValue: Invalid availability zone: [{us-east-1b]
        status code: 400, request id: 2637e297-236d-4e3b-b5a4-b5afe3df5e67
* aws_instance.example[2]: 1 error(s) occurred:

* aws_instance.example.2: Error launching source instance: InvalidParameterValue: Invalid availability zone: [ us-east-1d}]
        status code: 400, request id: 59244720-ab1c-48c7-a9b8-3eb6a23bd5ae
* aws_instance.example[1]: 1 error(s) occurred:

* aws_instance.example.1: Error launching source instance: InvalidParameterValue: Invalid availability zone: [ us-east-1c]
        status code: 400, request id: 1e335e56-f129-4ea0-a5c6-09b0501b6e18

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

It looks like you're using the wrong brackets? I think you need to use a list:

variable "azs" {
  description = "Run the EC2 Instances in these Availability Zones"
  type = "list"
  default = ["us-east-1a", "us-east-1b", "us-east-1c"]
}

Source


What minimal IAM permissions are needed by AWS Chatbot so that it can show logs? The AWS service called Chatbot allows CloudWatch Alarms to be relayed into a Slack channel. The message includes two buttons: Show logs and Show error logs.

However, when I press either of these buttons, I receive an error message like this:

❗ I can't get the logs for you because the role arn:aws:iam::123412341234:role/my-chatbot in account 123412341234 is missing necessary permissions related to commands.
If you want to get logs, go to the AWS Chatbot console and choose or create a role that includes Read-only command permission for this channel.

However, the recommended way of accomplishing this (the way automatically set up by AWS Console) grants Chatbot the ReadOnlyAccess policy which is way too permissive for my tastes. I don't want Chatbot to be able to read any object from any of our S3 buckets!
What is the minimal IAM policy that enables Chatbot to read CloudWatch Logs for Lambda functions?

AWS Chatbot uses CloudWatch Logs Insights, so the IAM policy for Chatbot must include these two statements:
{
    "Sid": "QueryLogs",
    "Effect": "Allow",
    "Action": [
        "logs:StopQuery",
        "logs:StartQuery",
        "logs:DescribeQueries"
    ],
    "Resource": "arn:aws:logs:::log-group:/aws/lambda/*"
},
{
    "Sid": "GetQueryResults",
    "Effect": "Allow",
    "Action": "logs:GetQueryResults",
    "Resource": "*"
}

With these statements in place, pressing the logs button will yield an initial message from Chatbot like this:

I ran this CloudWatch Logs Insights query to fetch logs from 2020-11-18T09:02 UTC to 2020-11-18T09:07 UTC.

... as well as several follow-up messages containing a plaintext dump of the logs.


AWS Batch use existing ECS cluster I would like AWS Batch to use one of my existing ECS clusters.

The problem is that when I create an unmanaged "compute environment" from the AWS Batch service, it doesn't allow me to specify an existing ECS cluster and it will automatically deploy a new ECS cluster (It doesn't let you choose). The reason why I want to associate my existing ECS cluster with the unmanaged "compute enviromnent" is because I don't want to pay for two ECS cluster when I already have one...

Any suggestions/alternatives?

I don't think what you want to do is possible.

I suggest you to use a managed compute environment so you don't care about cluster resources. Set the min and max CPU for the compute environment (min 0 is recommended so you don't waste many when no jobs are running). Then only set the resources needed by your jobs and batch will add compute capacity to satisfy the demand.

Your costs will be the same because batch will create an ASG(autoscaling group) and ECS cluster but it only charges you for EC2 instances.


Why a chef is the boss of the kitchen but a fullstack dev is not often the leader of a dev team? In one another comment here on DevOps SE we read:

  "Why a chef is the boss of the kitchen but a fullstack dev is not
  often the leader of a dev team, that's a question for another day"

So given the assumptions contained here are correct, why is so? Are we yet not there in terms of overall improvement and progress or is this economically nonsense (or, assumptions not correct)?

IMHO the comment is a bit misleading, in the sense that a chef is the boss of the kitchen mixes up and/or attempts to compare the unique leader/chef role (in French, chef literally means chief/boss/leader) with the expertise required to fill that role. By contrast, the full-stack developer is just an expertise, it is not a unique role.

If I'm not mistaken to become a chef one has to have experience in every other position in the kitchen. Similarly, a full-stack developer's expertise covers frontend, backend, etc - every other more specialized expertise required for the team's product. Which is probably the starting point for the comparison in the comment.

As I mentioned in my answer to that post, scale matters.

In a large team, the requirement for the leadership position isn't necessarily the technical expertise. Which is why often in software development the team leadership's role is a management position, not a technical one. Clear labour division example, if you want.

It's not impossible to even have multiple persons qualified to fill the chef role in the same kitchen, but only one can actually fill that position at any time, a chef in this context really refers to the role, not to the expertise. Also - there may be many types of chefs in a kitchen, see, for example Chef Jobs on Cruise Liners :)

But in a small resource-starved team where the team leader has to be selected from between the only full-stack developer and one or more less-experienced ones - probably the full-stack developer will be the leader, just like the only person qualified to be a chef in a small kitchen.


Ansible expected paths error I have a webserver playbook containing the following:

- hosts: webserver
  become: true
  roles:
    - apache2
    - { role: demo_app, db_user: demo, db_pass: demo, db_name: demo }

My folder structure is

.
├── ansible
│   ├── ansible
│   ├── ansible.cfg
│   ├── ansible.pub
│   ├── dev
│   ├── docker-compose.yml
│   ├── Docker.zip
│   ├── env
│   │   ├── ansible
│   │   ├── ansible.pub
│   │   ├── Dockerfile
│   │   ├── init-fake.conf
│   │   ├── ssh_config
│   │   └── ssh_host_config
│   ├── playbooks
│   │   ├── database.yml
│   │   ├── hostname.yml
│   │   └── loadbalancer.yml
│   ├── startup.sh
│   └── temp
│       ├── ansible
│       ├── ansible.cfg
│       ├── ansible.pub
│       ├── dev
│       ├── docker-compose.yml
│       └── Docker.zip
├── ansible.cfg
├── ansible.pub
├── control.yml
├── database.yml
├── demo
│   └── demo.conf
├── dev
├── docker-compose.yml
├── Docker.zip
├── env
│   ├── ansible
│   ├── ansible.pub
│   ├── Dockerfile
│   ├── init-fake.conf
│   ├── ssh_config
│   └── ssh_host_config
├── loadbalancer.yml
├── main.yml
├── playbooks
│   ├── hostname.yml
│   ├── stack_restart.yml
│   └── stack_status.yml
├── roles
│   ├── apache2
│   │   ├── defaults
│   │   │   └── main.yml
│   │   ├── files
│   │   ├── handlers
│   │   │   └── main.yml
│   │   ├── meta
│   │   │   └── main.yml
│   │   ├── README.md
│   │   ├── tasks
│   │   │   └── main.yml
│   │   ├── templates
│   │   ├── tests
│   │   │   ├── inventory
│   │   │   └── test.yml
│   │   └── vars
│   │       └── main.yml
│   ├── control
│   │   ├── defaults
│   │   │   └── main.yml
│   │   ├── files
│   │   ├── handlers
│   │   │   └── main.yml
│   │   ├── meta
│   │   │   └── main.yml
│   │   ├── README.md
│   │   ├── tasks
│   │   │   └── main.yml
│   │   ├── templates
│   │   ├── tests
│   │   │   ├── inventory
│   │   │   └── test.yml
│   │   └── vars
│   │       └── main.yml
│   ├── demo_app
│   │   ├── defaults
│   │   │   └── main.yml
│   │   ├── files
│   │   │   └── app
│   │   │       ├── demo.py
│   │   │       └── requirements.txt
│   │   ├── handlers
│   │   │   └── main.yml
│   │   ├── meta
│   │   │   └── main.yml
│   │   ├── README.md
│   │   ├── tasks
│   │   │   └── main.yml
│   │   ├── templates
│   │   │   └── demo.wsgi.j2
│   │   ├── tests
│   │   │   ├── inventory
│   │   │   └── test.yml
│   │   └── vars
│   │       └── main.yml
│   ├── mysql
│   │   ├── defaults
│   │   ├── files
│   │   ├── handlers
│   │   │   └── main.yml
│   │   ├── meta
│   │   │   └── main.yml
│   │   ├── README.md
│   │   ├── tasks
│   │   ├── templates
│   │   ├── tests
│   │   │   ├── inventory
│   │   │   └── test.yml
│   │   └── vars
│   │       └── main.yml
│   └── nginx
│       ├── defaults
│       ├── files
│       ├── handlers
│       │   └── main.yml
│       ├── meta
│       │   └── main.yml
│       ├── README.md
│       ├── tasks
│       ├── templates
│       │   └── nginx.conf.j2
│       ├── tests
│       │   ├── inventory
│       │   └── test.yml
│       └── vars
│           └── main.yml
├── site.yml
├── startup.sh
├── templates
├── webserver.retry
└── webserver.yml

Ansible version is 2.2.1. I'm getting the following error when trying to run ansible-playbook webserver.yml

TASK [demo_app : copy demo app source] *****************************************
fatal: [app01]: FAILED! => {"changed": false, "failed": true, "msg": "Unable to find 'demo/app/' in expected paths."}
fatal: [app02]: FAILED! => {"changed": false, "failed": true, "msg": "Unable to find 'demo/app/' in expected paths."}
    to retry, use: --limit @/work/webserver.retry

PLAY RECAP *********************************************************************
app01                      : ok=6    changed=0    unreachable=0    failed=1
app02                      : ok=6    changed=0    unreachable=0    failed=1

The relevant task is:

- name: copy demo app source
  copy: src=demo/app/ dest=/var/www/demo mode=0755
  notify: restart apache2

I've tried altering the above copy src to /demo_app and also src=demo_app/files/app but this didnt help.

The task which fails is:

- name: copy demo app source
  copy: src=demo/app/ dest=/var/www/demo mode=0755
  notify: restart apache2

If you have a look at your folder structure and more specifically the demo_app role, you have:

│   ├── demo_app
│   │   ├── defaults
│   │   │   └── main.yml
│   │   ├── files
│   │   │   └── app
│   │   │       ├── demo.py
│   │   │       └── requirements.txt
│   │   ├── handlers
│   │   │   └── main.yml
│   │   ├── meta
│   │   │   └── main.yml
│   │   ├── README.md
│   │   ├── tasks
│   │   │   └── main.yml
│   │   ├── templates
│   │   │   └── demo.wsgi.j2
│   │   ├── tests
│   │   │   ├── inventory
│   │   │   └── test.yml
│   │   └── vars
│   │       └── main.yml

The copy task will look for the src in demo_app/files/. So, either change your task to:

- name: copy demo app source
  copy: src=app dest=/var/www/demo mode=0755
  notify: restart apache2

Or create a demo directory in demo_app/files/ and place the app directory in it (like demo_app/files/demo/app/).

Setting an absolute path to the directory in the task will also work, but don't think it's an elegant solution.


How to get the device name of a specified mount point with ansible facts? Using Ansible facts I can get the list of "ansible_mounts":

        "ansible_mounts": [
            {
                "block_available": 423698,
                "block_size": 4096,
                "block_total": 507604,
                "block_used": 83906,
                "device": "/dev/sdb1",
                "fstype": "ext4",
                "inode_available": 130999,
                "inode_total": 131072,
                "inode_used": 73,
                "mount": "/boot",
                "options": "rw",
                "size_available": 1735467008,
                "size_total": 2079145984,
                "uuid": "b5cfa485-21d2-4e66-b3a3-80f250acf9d9"
            },
            {
                "block_available": 42446481,
                "block_size": 4096,
                "block_total": 131007993,
                "block_used": 88561512,
                "device": "/dev/sdd1",
                "fstype": "xfs",
                "inode_available": 524243436,
                "inode_total": 524287936,
                "inode_used": 44500,
                "mount": "/u01",
                "options": "rw",
                "size_available": 173860786176,
                "size_total": 536608739328,
                "uuid": "f34a9499-ca4e-4db6-801a-16efb72a629d"
            },
            {
                "block_available": 3010686,
                "block_size": 4096,
                "block_total": 4095466,
                "block_used": 1084780,
                "device": "/dev/sdc1",
                "fstype": "ext4",
                "inode_available": 984674,
                "inode_total": 1048576,
                "inode_used": 63902,
                "mount": "/",
                "options": "rw",
                "size_available": 12331769856,
                "size_total": 16775028736,
                "uuid": "829e9504-965b-4660-acc2-12d10d022a3c"
            }
        ],

How I can get the value of device where mount value is "/" ?

Use json_query. For example
    - set_fact:
        my_device: "{{ ansible_mounts|json_query(query) }}"
      vars:
        query: "[?mount=='/'].device "
    - debug:
        var: my_device

gives
    my_device:
      - /dev/sdc1

To get the first element of the list only append the filter first
    - set_fact:
        my_device: "{{ ansible_mounts|json_query(query)|first }}"


Can't install MongoDB docker on raspberry If I want to execute any docker file from https://hub.docker.com/r/library/mongo/tags/ on my raspberry Pi 3B with raspbian stretch installed I always get the same error namely something like :

Pulling mongodb (mongo:)...
latest: Pulling from library/mongo
ERROR: no matching manifest for unknown in the manifest list entries

How can I resolve this error and what causes it? If you need any more details please let me know I'm pretty new to docker so I don't really know what's relevant.

This means that the container that you are trying to pull is not available for the architecture, in this case it would be linux/arm64. Maybe there is an image available now. I would search for 'docker mongodb arm64' to see. I see support for mongodb under arm64v8 architecture, which would be in Raspberry Pi 4. Maybe you want to upgrade, as your chip is arm64v7.

This issue explains error message in a bit more detail.

Answers under this issue are going to help you run MongoDB under ARM64.


Is there a reason not to use Bamboo to run repetitive tasks instead of cron? Is there a reason it would be a bad decision to use Bamboo ( or Jenkins ) to execute a scheduled task ( say every 15 minutes ) rather than using a cron job?

Using task schedulers like Bamboo or Airflow have a lot of advantages over cron, particularly in their ability to recognize errors and process dependent tasks accordingly.  By far the most common reason not to use them is that you simply haven't set up the infrastructure yet: some form of cron is installed on every Linux machine, and it's easy to add a new job in, whereas these more advanced schedulers require setting up fleets of machines, installing and configuring various pieces of software, and learning how to write new tasks.

Assuming that you already have this in place, though, there are a few usecases that cron handles better:

Running a job on a specific machine.  You can do this with other schedulers, but they're generally designed for tasks that can be run on any one of a set of machines; when you want to do something like run a log rotation on every server in your fleet, that's much easier to set up with cron.
Running time-sensitive tasks at exact times.  Distributed schedulers will schedule a job to run, but it may not actually run for some period of time, depending on what else is in the queue.  Cron doesn't have a queue, and so a task that needs to run every minute will run every minute (or at least as close to that as the OS scheduler will allow).


kubernetes nodeport service load balancing Let's assume I have three node named n1, n2, n3. created four pods named p1, p2, p3, p4, k8s scheduler placed pods in nodes as follows
n1 -> p1
n2 -> p2
n3 -> p3, p4
Now, to access these pods I have created a nodeport service. Nodeport service will open same numbered ports in each nodes. Nodes will have iptables configured to forward traffics from service to pods.
Let's also assume that I will have a NLB configured top of it. how traffic will be distributed? is it like below?
p1 -> 33.3330%
p2 -> 33.3330%
p3 -> 16.6665%
p4 -> 16.6665%

service is a cluster-wide object. In each node, kube-proxy will create iptable load balancing entry with all available pod's IP based on selector for a service. So NLB will distributed traffic evenly among all nodes. Then each will loadbalance the traffic to all available pods using round robin algorithm.
We will have those traffic distributed among all pods like below.
p1 -> 33.3330%
p2 -> 33.3330%
p3 -> 16.6665%
p4 -> 16.6665%
This will deviate a little bit.


Azure Pipelines: Exclude folders using Azure App Service Deploy I have an Azure DevOps Pipeline which includes an Azure App Service Deploy task (AzureRmWebAppDeployment) for deploying an ASP.NET Core 3.1 project:

- task: AzureRmWebAppDeployment@4
  inputs:
    ConnectionType: 'AzureRM'
    azureSubscription: 'Azure Subscription(01234567-89ab-cdef-0123-456789abcdef)'
    appType: 'webApp'
    WebAppName: 'MyStagingSite'
    packageForLinux: '$(Build.ArtifactStagingDirectory)/**/*.zip'
    enableCustomDeployment: true
    DeploymentType: 'webDeploy'
    enableXmlTransform: false
    enableXmlVariableSubstitution: true
    RemoveAdditionalFilesFlag: false

The Azure App Service destination, however, contains files in several pre-established folders which are managed independent of the continuous delivery process. We would like to use the Remove additional files at destination flag (RemoveAdditionalFilesFlag) while leaving those folders intact.

  Disclaimer: I don't consider this a best practice and, in the future, we will be moving these files off to a separate storage location. Until then, I'd like to find a solution that will work resolve this.

In Visual Studio 2019, we achieve this by excluding those files from our publishing process using the MsDeploySkipRules in our csproj file:

<Project Sdk="Microsoft.NET.Sdk.Web">
  …
  <ItemGroup>
    <MsDeploySkipRules Include="CustomSkipFolder">
      <ObjectName>dirPath</ObjectName>
      <AbsolutePath>wwwroot\\Uploads</AbsolutePath>
    </MsDeploySkipRules>
</Project>

This approach works well for Visual Studio. Those rules do not appear to be honored by the AzureRmWebAppDeployment task, however, even when using the "Web Deploy" deployment method (DeploymentType).

Is there a way to honor the MsDeploySkipRules when using the AzureRmWebAppDeployment task? If not, if there a way to provide a list of folders which should be skipped or ignored as part of the deployment process? Or, alternatively, if there another task that will permit one of these options?

As @yang-shen-msft notes on Stack Overflow, there doesn't appear to be a way to honor the MSDeploySkipRules defined in the csproj file. Instead, files and folders can be skipped by defining the Additional Arguments (AdditionalArguments) parameter of the Azure App Service Deploy (AzureRmWebAppDeployment) task, as discussed in another Stack Overflow answer.
Since there doesn't appear to be any official documentation for the -skip rules, and the MSDeploy.exe documentation that Azure Pipelines references is out-of-date, the following provides additional details.
First, it's useful to recognize that when you deploy a project via Visual Studio, it's simply taking the MSDeploySkipRules configured in the csproj file and adding them to it's internal call of msdeploy.exe as -skip rules. So, given the following rule defined in the csproj:
<Project Sdk="Microsoft.NET.Sdk.Web">
  …
  <ItemGroup>
    <MsDeploySkipRules Include="CustomSkipFolder">
      <ObjectName>dirPath</ObjectName>
      <AbsolutePath>wwwroot\\Uploads</AbsolutePath>
    </MsDeploySkipRules>
  </ItemGroup>
</Project>

The -skip rule is interpreted as:
msdeploy.exe -skip:objectName=dirPath,absolutePath=wwwroot\\Uploads

Translating this to the Azure App Service Deploy (AzureRmWebAppDeployment) task, the resulting yml might look like:
- task: AzureRmWebAppDeployment@4
  inputs:
    ConnectionType: 'AzureRM'
    azureSubscription: 'Azure Subscription'
    appType: 'webApp'
    WebAppName: 'MyStagingSite'
    packageForLinux: '$(Build.ArtifactStagingDirectory)/**/*.zip'
    enableCustomDeployment: true
    DeploymentType: 'webDeploy'
    enableXmlTransform: false
    enableXmlVariableSubstitution: true
    RemoveAdditionalFilesFlag: true
    AdditionalArguments: '-skip:objectName=dirPath,absolutePath=wwwroot\\Uploads'

Note: Multiple -skip rules can be defined on the same msdeploy.exe call.

Unfortunately, as mentioned above, there doesn't appear to be any official, first-party documentation for the -skip rules on msdeploy.exe. The 2014 documentation acknowledges them, and provides two examples, but doesn't expand on the options. That said, way back in 2012, @richard-szalay wrote a useful article, "Demystifying MSDeploy skip rules", which provides useful direction here for anyone requiring additional control over their -skip rules.


Service check with api in Icinga2 I am trying to do add service of client using icinga2 api. I have followed https://github.com/saurabh-hirani/icinga2-api-examples. Here is what I have done so far.

First I created zone and endpoint in master as follows:

Endpoint

Object Endpoint "client.com" {
}

Zone

Object Zone "client.com" {
    endpoints = [ "client.com" ]
    parent = "master.com"
}

Restarted icinga2

Then in client server

#!/bin/bash -e

. icinga2_vars

echo "CREATING HOSTGROUP"

curl -s -u $ICINGA2_API_USER:$ICINGA2_API_PASSWORD \
    -H 'Accept: application/json' -X PUT \
    -k "https://$ICINGA2_HOST:$ICINGA2_API_PORT/v1/objects/hostgroups/client_hostgroup" \
    -d '{ "attrs": {
            "display_name": "client host group"
} }' | python -m json.tool

echo "CREATING HOST"

curl -s -u $ICINGA2_API_USER:$ICINGA2_API_PASSWORD  \
    -H 'Accept: application/json' -X PUT \
    -k "https://$ICINGA2_HOST:$ICINGA2_API_PORT/v1/objects/hosts/client.com" \
    -d '{ "templates": [ "generic-host" ],
        "attrs": {
        "address": "127.0.0.1",
        "check_command": "hostalive",
        "groups": ["client_hostgroup"],
        "vars.notification['mail']": {
            "groups": ["client_hostgroup_users"]
            }
} }' | python -m json.tool

echo "CREATING SERVICE"

curl -s -u $ICINGA2_API_USER:$ICINGA2_API_PASSWORD  \
    -H 'Accept: application/json' -X PUT \
    -k "https://$ICINGA2_HOST:$ICINGA2_API_PORT/v1/objects/services/client.com!load" \
    -d '{
        "templates": [ "generic-service" ],
        "attrs": {
            "display_name": "check load",
            "check_command" : "load",
            "command_endpoint": "client.com",
            "host_name": "client.com"
        }
}' | python -m json.tool

It is working but checking service of master itself. It should check service of client server.

You should add the correct zone attribute for the host and the service, so they will propagate to the client node.

Add "zone": "client.com" to the api requests.


Looking for a good book explaining how things work in the industrial system test environment and production environment Any advice on a good book explaining how things work in the industrial system test environment and production environment?
It's my first time in this topic, so I would need a general overview.

Key words: builds, web logic levels, machines, clusters, logs, cache, balancers, UDC

(If you thing I posted this question in the wrong place, please, could you point me the right one? Thanks)

Dude, will depends of what you want to use or follow. Actually cloud + DevOps it's a buzzword in use.
So you can start reading about Cloud Architecture, Modern Infrastructure.
Refer about one book, maybe these: https://www.springer.com/gp/book/9783030037116


Terraform | Upgrade to 0.12 error Reference to "count" in non-counted context I'm trying to upgrade terraform from 0.11 to 0.12 but receiving error:

 Error: Reference to "count" in non-counted context

  on main.tf line 33, in data "aws_subnet" "public":
  33:   id = data.aws_subnet_ids.public.ids[count.index]

The "count" object can be used only in "resource" and "data" blocks, and only
when the "count" argument is set.

Here is my part of code:

data "aws_subnet_ids" "public" {
  vpc_id = data.aws_vpc.vpc.id

  tags = {
    Tier = "public"
  }
}

data "aws_subnet" "public" {
  id = data.aws_subnet_ids.public.ids[count.index]
}

Maybe someone know how to fix this error?
Thanks in advance.

It looks like your underlying goal here is to retrieve the full set of attributes for each of your subnets that are marked as Tier = public. Here's a different way to do that using features from Terraform 0.12.6:

data "aws_subnet_ids" "public" {
  vpc_id = data.aws_vpc.vpc.id

  tags = {
    Tier = "public"
  }
}

data "aws_subnet" "public" {
  for_each = data.aws_subnet_ids.public.ids

  id = each.value
}

With the above, you should find that data.aws_subnet.public is a map from subnet id to the attributes of that particular subnet.

The reason for the error in your case is that you used count.index but didn't set the count argument, and so there is no count index to return.

The count-based equivalent of the above, which is more similar to your original example, would be this:

data "aws_subnet_ids" "public" {
  vpc_id = data.aws_vpc.vpc.id

  tags = {
    Tier = "public"
  }
}

data "aws_subnet" "public" {
  count = length(data.aws_subnet_ids.public.ids)

  id = sort(data.aws_subnet_ids.public.ids)[count.index]
}

The result of this is very similar to the for_each-based example I gave above, but with one significant difference: in this case data.aws_subnet.public will be a list of subnet objects, ordered by the lexical ordering of their subnet ids. That will usually make the result harder to use elsewhere in the configuration, so I'd suggest using the for_each approach unless there's a specific reason why you need a list.

(If you use a map and then later find that you need a list in a specific context, you can always use values(data.aws_subnet.public) to take the values from the map, in the same lexical order.)


Rerun of Bamboo jobs on commit Imagine the following setup: a build plan has a set of jobs (J), each builds a module (M). Latest successful build for all of them is also there. (BX)
E.g. (imagine there are ten or so; modules are interchangeable i.e. will be never linked to each other)

J1->M1->BM1-X
J2->M2->BM2-Y
J3->M3->BM3-Z

Now imagine there are changes in just one of repos, say M1. We know that Bamboo runs jobs in parallel.

Questions:

will Bamboo completely rerun builds also J2 and J3, even if it finds out there are no changes in repos M2 and M3?
If yes (I think so by a test setup but not sure), what is the purpose of doing so? (at least in terms of Bamboo logic)

I think you have fundamentally misunderstood the relationship between plans, stages and jobs in bamboo.

A good rule of thumb is to have a single repository of code, which is linked to a single build plan, which produces a single independent module.

You may initially start out creating a single stage with a single job inside each single plan where you just run a single maven command and the unit tests built into your maven build may be perfectly sufficient to exercise your modules completely in this extremely simple use case you probably don't need a tool like bamboo or Jenkins to deal with your builds.

However, very quickly your conceptualisation of a "build" gets very complicated. As you mentioned in your question you might quickly amass multiple modules that are interrelated, or you might need to support your code on multiple platforms or in multiple browsers, and if you get all the way to the CD end of the work flow you might even need to deploy binaries into environments as the result of a code change.

This is where you would want to use the extra orchestration capabilities of your CI tool.

For example, if you are developing a module that you expect to load in a browser then your might have a build pipeline that looks like:

Check in Code -> Build/Compile/Unit Test -> Deploy to a CI environment -> Stand up a half dozen browsers and verify that they all still work correctly.

In the above work flow you would have a single stage to handle your compilation and unit testing which would spit out a compiled or built artifact. Then you have a stage that does whatever needs doing to make your module useful, ie it might create your final shippable product by combining your module with other modules built from other plans, or it might package some software into an RPM, a DEB and so on, depends what you're actually building. If you need to ship to multiple platforms then this is your first stage where parallelism becomes meaningful.

Alternatively, if you are building for a single platform but for multiple consumers you might follow up this build of your component with a bunch of headless browsers kicking off test cases to validate that your changes didn't break anything. These can all happen in parallel on different agents running in different OS's, depends how you have bamboo configured.

Then finally, assuming you are shopping something more complex than a single module, you will want to do a bit of integration testing with all the other components that you need to interact with. This step will usually be separated into a build plan of its own, triggered by the successful completion of the plans which are spitting out individual modules. Here you get a chance to run all the same tests but with your latest version running along side everyone else's module so you can be sure that there are no breakages caused by dependencies.

So at any one time you only have a single repo triggering a single plan. If you have multiple repositories feeding into the same plan and they are completely unrelated but use the same build pipeline then I'd recommend taking advantage of bamboos plan branches so that you can reuse the same build process but with multiple different code bases (if that fits your use case)


How should I store my environment variables? This is a very broad question about methods and advice regarding environment variables / structure. But ultimately I'm looking for answers for the very specific question of 'How should I store my environment variables?'

Firstly some clarifications:

An environment to me can be from 3 to 10 servers and is a way of containing a specific customer's infrastructure.
Inside each environment there are a few variables which are mostly automatically generated from a few key inputs (name, size etc).

As it stands right now we're storing all of our environment variables in a structure like so:

<playbook>.yml                   # Various playbooks for deployment
roles/windows                    # Ansible role for Ubuntu
roles/ubuntu                     # Ansible role for Ubuntu
config/hosts/<name>.yml          # Ansible inventory
config/hosts/vars/<name>.json    # Environment specific variables

Right now the config is initialised as a submodule in the git repository above. As the variables file changes rather frequently this has caused issues with the data changing, once, twice or even three times between commits making changes increasingly difficult to trace.

As I personally see it going forward we should be looking to store all our customer variables in a centralised / scalable way and then hook into it with a dynamic inventory with ansible.

I understand there are a few technologies which seem to do a part of what could be required such as Consul but they seem to work best in an environment which serves one big application rather than lots of smaller slightly differing ones.

I essentially see us having to write an inventory script and then just shove all of our data into some not-for-purpose built database and then keep going on as if nothing changed. I see this conceivably as a way to potentially trim down on a lot of the data we currently store and perhaps look into different ways of storing the data rather than just scaling up what serves it again.

I'm hoping somebody has some kind of experience in implementing infrastructure as code when having to deal with lots of smaller environments as opposed to one, two or three huge ones.

Any suggestions?

I've had two runs at doing environment variables in a scalable way and neither has ended up perfect because,  as I've discovered, is a very tricky thing to get right. I'll give a summary of both of my experiences below:

Common Factors

Environment variables are stored in a separate repository from the original source code (they are submoduled together but are still based on separate repos)
There is a separate "build" process for the artifact and it's variables.
There is not a separate release process for the environment variables. If you want to change environment variables you need to go through the same change review boards and usual

Using Consul KV Pairs

The environment variables are loaded from an artifact repository (never the original git repo) and loaded into a namespaced KV pair tree, for example

/env/dev1/my/application/v1.1.1

Where the preceding dev1 is the name of the environment, the my/application is the application namespace, and the v1.1.1 is the version of the environment variables to use.

To developers all these things are invisible. At runtime the platform checks that the environment exists in the current consul cluster (if it doesn't there's a problem and it's errors out), it then checks the subtree for the applications namespace (this way there can be no cross contamination where one app references another apps vars) then the version number of the configuration is taken from label connected to the deployable artifact. Updating this label is the key thing here because it means if we lost both production data centers we could stand up the environment again by simple reading the meta data from our deployable artifacts and loading all the environment variables into the KV store.

Problems With This Approach
Developers always, and I mean every single time, found a way to slip configuration changes into the environment that had significant impacts to how the application would run. Because it always ended up being easier to get configuration changes approved than code changes.

Storing a "Deployment" Artifact with variables embedded

This tightly couples the exact version of the artifact to the version of configuration. If you changed configuration then you had to rebuild this deployment artifact.

The deployment artifact itself was essentially a yaml file which contained the URL to the releasable binary and all the configuration attached to it.

The platform contains componentry to read variables and then out them into the process tree of the application when it starts up.

This has so far been a lot more successful because there is an artifact we can trace the history of and which we can hold up to any review board and say "this is the only artifact we care about, we don't need t look at any other changes, only changes to this thing" (ie. Version of application to deploy,  environment variables included etc.

This makes it just a little bit harder for developers to try and build logic into their application that will change its behavior based on variables so they can slip in changes without going through appropriate testing cycles.

Bonus Points

Consider application secrets. Our solution to this has so far been to provide a public RSA key which development teams use to encrypt an extended Java key store (almost every language has a library somewhere that can read Java keys stores) this is then treated like a third type of artifact and is pulled onto the server, decrypted with our platform private key, and provided to the application at run time.

Admittedly secrets management is its own can of worms. But it's probably worth considering.


Mobile CI: Versioning and Tracking Builds in JIRA We're in process of defining a workflow for mobile builds across multiple developers.  We've only had one until recently, so we didn't have to worry too much about tracking simultaneous feature builds for QA (we use gitflow).

1) Versioning

Builds for the apple and android stores require versions.  But, features built for QA won't have an assigned version yet, since we don't know what release it will be in.  We are thinking of just using the current version in production, and using build counts to differentiate builds.  Is there a best practice for tracking builds in Testflight / Google Play only targeted for internal test?

2) Tracking Builds in JIRA

With multiple builds in the store for testing, we need to tag our JIRA tickets with the associated build in the store.  We're going to do this manually, but I'm not sure the best way to do that is.  And, if we wanted to build in some automation, what would that look like?

The answer will vary depend on the CI/CD tooling you are using.

If you are planning on using Atlassian's CI/CD tool Bamboo, you could use the Bamboo variable ${bamboo.buildNumber} which ties the build number of Bamboo to your version number.  You can have a task that will adjust the version based on this variable.  Bamboo
 integrates with Jira so that you can see the builds and deployments directly within Jira.  There is also Git so you could see the commits that went into the deployment.  This effectively would automate the second requirement you listed.

There are plugins available for Jira that will integrate with other CI/CD toolsets (e.g. Jenkins, CircleCI), and the idea would be the same: let the CI/CD server automate the versioning for you.


Looping through found files in Ansible I am trying to set the sendmail_path to use Mailhog in all php.ini files on a server with multiple versions of PHP installed for PHP FPM.

Why is the following snippet is not working?

- hosts: localhost
  tasks:
    - name: Find ini files
      find:
        paths: /home/myuser/projects/infra/php
        file_type: file
        recurse: Yes
        patterns: "*php.ini"
      register: files_matched

    - name: Debug files_matched
      debug:
        var: files_matched.files

    - name: Debug files_matched loop
      debug:
        var: item.path
      loop: "{{ files_matched.files|flatten(levels=1) }}"
      loop_control:
        label: "{{ item.path }}"

    - name: Ensure "sendmail_path=/opt/mailhog/mhsendmail is in section "[mail function]"
      ini_file:
        path: item.path
        section: "mail function"
        option: sendmail_path
        value: /opt/mailhog/mhsendmail
        mode: 0644
        backup: no
      loop: "{{ files_matched.files|flatten(levels=1) }}"
      loop_control:
        label: "{{ item.path }}"

The error I am getting is essentially No such file or directory: '' on the item.path variable.

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: OSError: [Errno 2] No such file or directory: ''
failed: [localhost] (item=/home/myuser/projects/infra/php/7.0/fpm/php.ini) => {"changed": false, "item": {"atime": 1551116700.7761922, "ctime": 1551115509.596934, "dev": 66309, "gid": 1000, "gr_name": "myuser", "inode": 5640238, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mode": "0644", "mtime": 1544171307.0, "nlink": 1, "path": "/home/myuser/projects/infra/php/7.0/fpm/php.ini", "pw_name": "myuser", "rgrp": true, "roth": true, "rusr": true, "size": 70999, "uid": 1000, "wgrp": false, "woth": false, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}, "module_stderr": "Traceback (most recent call last):\n  File \"/home/myuser/.ansible/tmp/ansible-tmp-1551192384.86-125024514722793/AnsiballZ_ini_file.py\", line 113, in <module>\n    _ansiballz_main()\n  File \"/home/myuser/.ansible/tmp/ansible-tmp-1551192384.86-125024514722793/AnsiballZ_ini_file.py\", line 105, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/myuser/.ansible/tmp/ansible-tmp-1551192384.86-125024514722793/AnsiballZ_ini_file.py\", line 48, in invoke_module\n    imp.load_module('__main__', mod, module, MOD_DESC)\n  File \"/tmp/ansible_ini_file_payload_yVPEa4/__main__.py\", line 342, in <module>\n  File \"/tmp/ansible_ini_file_payload_yVPEa4/__main__.py\", line 322, in main\n  File \"/tmp/ansible_ini_file_payload_yVPEa4/__main__.py\", line 151, in do_ini\n  File \"/usr/lib/python2.7/os.py\", line 157, in makedirs\n    mkdir(name, mode)\nOSError: [Errno 2] No such file or directory: ''\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

As you can see from the error and debug output the file path should not be empty.

OK. The answer was simple. I needed to template the item.path. I guess this is a beginner mistake.
- hosts: localhost
  tasks:
    - name: Find ini files
      find:
        paths: /home/myuser/projects/infra/php
        file_type: file
        recurse: Yes
        patterns: "*php.ini"
      register: files_matched

    - name: Debug files_matched
      debug:
        var: files_matched.files

    - name: Debug files_matched loop
      debug:
        var: item.path
      loop: "{{ files_matched.files|flatten(levels=1) }}"
      loop_control:
        label: "{{ item.path }}"

    - name: Ensure "sendmail_path=/opt/mailhog/mhsendmail is in section "[mail function]"
      ini_file:
        path: "{{ item.path }}"
        section: "mail function"
        option: sendmail_path
        value: /opt/mailhog/mhsendmail
        mode: 0644
        backup: no
      loop: "{{ files_matched.files|flatten(levels=1) }}"
      loop_control:
        label: "{{ item.path }}"