Enabling elasticsearch-readonlyrest-plugin

1. Install the plugin

export ES_VERSION=2.3.0
bin/plugin install https://github.com/sscarduzio/elasticsearch-readonlyrest-plugin/releases/download/v1.10.0_es-v$ES_VERSION/elasticsearch-readonlyrest-v1.10.0_es-v$ES_VERSION.zip

2. Configuration

Append either of these snippets to conf/elasticsearch.yml

# remember to set the right CORS origin (or disable it, if you're brave). See https://github.com/elastic/kibana/issues/6719
http.cors.enabled: true
http.cors.allow-origin: /https?:\/\/localhost(:[0-9]+)?/

readonlyrest:
    enable: true

    response_if_req_forbidden: Forbidden by ReadonlyREST ES plugin

    access_control_rules:

    - name: "Logstash can write and create its own indices"
      # auth_key is good for testing, but replace it with `auth_key_sha1`!
      auth_key: logstash:logstash
      type: allow
      actions: ["indices:data/read/*","indices:data/write/*","indices:admin/template/*","indices:admin/create"]
      indices: ["logstash-*", "<no-index>"]

    - name: Kibana Server (we trust this server side component, full access granted via HTTP authentication)
      # auth_key is good for testing, but replace it with `auth_key_sha1`!
      auth_key: admin:passwd3
      type: allow

    - name: Developer (reads only logstash indices, but can create new charts/dashboards)
      # auth_key is good for testing, but replace it with `auth_key_sha1`!
      auth_key: dev:dev
      type: allow
      kibana_access: ro+
      indices: ["<no-index>", ".kibana*", "logstash*", "default"]

Now activate authentication in Kibana server: let the Kibana daemon connect to ElasticSearch in privileged mode.

edit the kibana configuration file: kibana.yml and add the following:


elasticsearch.username: "admin"
elasticsearch.password: "passwd3"


This is secure because the users connecting from their browsers will be asked to login separately anyways

4. restart elastic search

ENABLING KIBANA AUTHENTICATION USING NGINX:

1. Install nginx:
1. rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
2. yum -y install nginx httpd-tools
2. Set one username & password
1. htpasswd -c /etc/nginx/conf.d/kibana.htpasswd admin
3. Configure Nginx
1. vi /etc/nginx/conf.d/kibana.conf
2. server {
listen *:8080;
server_name 192.168.1.5;
access_log /var/log/nginx/kibana-access.log;
error_log /var/log/nginx/kibana-error.log;
   location / {
auth_basic "Restricted Access";
auth_basic_user_file /etc/nginx/conf.d/kibana.htpasswd;
proxy_pass http://192.168.1.5:5601;
#proxy_connect_timeout 150;
#proxy_send_timeout 100;
#proxy_read_timeout 100;
}
}
4. Restart Nginx
1. sudo service nginx restart
5. Go to the URL : http://192.168.1.5:8080, we should get a authentication screen on successful setup

  NOTE: In case if it doesn't work:

1. Disable the selinux by running the command
1. sudo setsebool -P httpd_can_network_connect 1
   

   
   
2. sudo service nginx restart

Getting started with Ansible

Ansible Demo

Installing Ansible on CentOS 7 Machine 1 (Eg IP: 192.168.56.101)

yum install -y ansible

Creating inventory file

$ mv /etc/ansible/hosts /etc/ansible/hosts.orig

$ vi /etc/ansible/hosts 

[web]

192.168.56.102 ansible_ssh_pass=<Password> ansible_ssh_user=<UserName>

Save this file with "!wq" command.

Creating Playbook file 

$vi SamplePlaybook.yml

---

- hosts: web

  tasks:

   - name: install apache

     yum: name=httpd state=latest

   - name: copy my indexfile

     template: src=/root/index.html dest=/var/www/index.html

     notify:

      - restart apache

   - name: ensure apache is running

     service: name=httpd state=started enabled=yes

  handlers:

  - name: restart apache

    service: name=httpd state=restarted

Save this file with "!wq" command.

Now execute this file using ansible playbook command:

$ ansible-playbook SamplePlaybook.yml

Output

----------------------------------------------------------------------------------

PLAY [web] *********************************************************************

TASK [setup] *******************************************************************

ok: [192.168.56.102]

TASK [install apache] **********************************************************

ok: [192.168.56.102]

TASK [copy my indexfile] *******************************************************

changed: [192.168.56.102]

TASK [ensure apache is running] ************************************************

changed: [192.168.56.102]

RUNNING HANDLER [restart apache] ***********************************************

changed: [192.168.56.102]

PLAY RECAP *********************************************************************

192.168.56.102             : ok=5    changed=3    unreachable=0    failed=0

Fluentd secure forwarding the logs

Following are the steps on FluentD server side:

·       Install fluentd secure forward plugin:

sudo /usr/sbin/td-agent-gem install fluent-plugin-secure-forward

·       Append following configuration in /etc/td-agent/td-agent.conf

<source>

  type secure_forward

  shared_key FLUENTD_SECRET

  self_hostname efk

  secure true

  ca_cert_path         /opt/ fluendconf /certificate/ca_cert.pem

  ca_private_key_path /opt/ fluendconf /certificate/ca_key.pem

  ca_private_key_passphrase passphrase_for_private_CA_secret_key

</source>

·       Generate CA certificate:

    cd /opt/td-agent/embedded/lib/ruby/gems/2.1.0/bin/     sudo ./secure-forward-ca-generate /opt/fluendconf/certificate/       passphrase_for_private_CA_secret_key

·       Restart the server

sudo service td-agent restart

Following are the steps on FluentD Agent side:

·       Install fluentd secure forward plugin:

sudo /usr/sbin/td-agent-gem install fluent-plugin-secure-forward

Append following configuration in /etc/td-agent/td-agent.conf

        <match **>

          type secure_forward

          shared_key FLUENTD_SECRET

          self_hostname ${hostname}

          secure true

          ca_cert_path /opt/ fluendconf /certificate/ca_cert.pem

         <server>

           host centralize_logging_server

         </server>

       </match>

Copy server’s ca_cert.pem file

               copy servers ca_cert.pem file to /opt/fluendconf/certificate/

Restart the server

       sudo service td-agent restart

Writing your own Dockerfile

mkdir My_Sample_Docker_Image

vi My_Sample_Docker_Image\Dockerfile

FROM ubuntu
RUN apt-get update
RUN apt-get install -y nginx
ENTRYPOINT [“/usr/sbin/nginx”,”-g”,”daemon off;”]
EXPOSE 80

Docker Cheatsheet

Docker Cheatsheet

Build Commands:

docker build -t myapp:1.0 .

This command will build your docker image from the Dockerfile in the current directory.

docker images

This command will list all images which are locally stored with Docker engine.

docker rmi myapp:1.0

This command will remove/Delete the stored image from Docker engine.

Ship Commands:

docker pull myapp:latest

This command will pull image from registry.

docker push myrepo/myapp:1.1

Run Commands:

docker run --rm remove container automatically after exit

                   -t connect the container to terminal 

                   -name test name the container

                   -p 5000:80 expose port no 5000 externally and map to port 80

docker stop test

Stop the running container with name test.

docker rm test

remove the stopped container with name test

docker exec -it test bash

Create a new bash process inside the container and connect it to the terminal.

docker logs --tail 100 test

Print last 100 line of container logs.