2015年11月4日 星期三

[PHP] 把玩 PHP AWS SDK - 列出 EC2 指定規格列表、 ELB 底下的機器列表 @ Ubuntu 14.04

之前用一些 tool-based 的工具做了一些事情,例如呼叫完指令再動態抽 JSON 資料:
  • $ aws elb describe-load-balancers --load-balancer-name "XXX"
  • $ aws ec2 describe-instances --instance-ids  "XXX"
來試試 AWS SDK 來做事吧,先挑 PHP !

為了高移植性,我採用下載 aws.phar 的模式:
接著則是簡單的登入資料填寫一下:

<?php
require 'aws.phar';

$config = array(
'version' => 'latest',
'credentials' => array(
'key' => 'key',
'secret' => 'secret',
),
'region' => 'us-west-2',
);


列出 EC2 - m1.small instance:

<?php
$ec2Client = Aws\Ec2\Ec2Client::factory($config);

// http://docs.aws.amazon.com/aws-sdk-php/v3/api/class-Aws.Ec2.Ec2Client.html
$result = $ec2Client->DescribeInstances(array(
        'Filters' => array(
                 array('Name' => 'instance-type', 'Values' => array('m1.small')),
        )
));

print_r($result);


想要列出 EC2 - 指定 Load Balancer 下的機器:

<?php
$client = Aws\ElasticLoadBalancing\ElasticLoadBalancingClient::factory($config);
$result = $client->DescribeLoadBalancers( array(
        'LoadBalancerNames' => array('Load Balancer Name')
));
print_r($result['LoadBalancerDescriptions'][0]['Instances']);
print_r($result);


其中,雖然 $result 都是 Aws\Result Object ,且裡頭都是 [data:Aws\Result:private] => Array ,但可以透過 array access 的方式去存取囉。

接下來就可以試試連續動作,從 ELB 得知機器後,再得到機器的 public ip:

<?php

require 'aws.phar';

$ELB_NAME = 'XXXX';
$client = Aws\ElasticLoadBalancing\ElasticLoadBalancingClient::factory($config);
$result = $client->DescribeLoadBalancers( array('LoadBalancerNames' => array($ELB_NAME)));
if (isset($result['LoadBalancerDescriptions']) && is_array($result['LoadBalancerDescriptions']) && count($result['LoadBalancerDescriptions']) && isset($result['LoadBalancerDescriptions'][0]['Instances'])) {
$instances = $result['LoadBalancerDescriptions'][0]['Instances'];
//print_r($instances);
$ec2_id = array();
foreach($instances as $ec2)
if (isset($ec2['InstanceId']))
array_push($ec2_id, $ec2['InstanceId']);
if (count($ec2_id) > 0) {
$client = Aws\Ec2\Ec2Client::factory($setup);
$result = $client->DescribeInstances(array(
'Filters' => array(
array('Name' => 'instance-id', 'Values' => $ec2_id),
)
));
}
$public_ip = array();
if (isset($result['Reservations'])) {
foreach($result['Reservations'] as $target) {
//print_r($target['Instances'][0]['PublicIpAddress']);
array_push($public_ip, $target['Instances'][0]['PublicIpAddress']);
}
}
//print_r($result['Reservations']);
print_r($public_ip);
}

2015年11月3日 星期二

[Linux] 透過 rpmbuild 建立 RPM @ CentOS 6.6

RPM 的製作過程真叫人又愛又恨 XD 好多細節啊。例如執行 rpmbuild 的角色,通常會在家目錄出現 rpmbuild 目錄結構等,整體上建立 rpm 有四個步驟:
  1. 建立相關的目錄結構  rpmbuild/BUILD rpmbuild/BUILDROOT rpmbuild/RPMS rpmbuild/SOURCES rpmbuild/SPECS  rpmbuild/SRPMS
  2. 將資料擺進 rpmbuild/SOURCES 位置
  3. 撰寫 rpmbuild/SPECS/example.spec
  4. 切換進 rpmbuild/SPECS 執行 rpmbuild -bb example.spec
若上述一切正常,在 rpmbuild/RPMS/ 裡的目錄長出 rpm 檔案。

這邊要筆記的是 example.spec 這個檔案格式:
  • % 開頭的是 spec 的保留指令(Macro),例如 %define 是定義變數,而 %description, %prep, %setup, %build, %install, %files, %changelog, %clean 則是對應的狀態
    • %prep 下方的位置代表預備動作
    • %setup 代表進行 Source 解壓縮
    • %build 例如執行 make 指令
    • %install 開始執行建立 RPM 的動作,可把所需檔案進行搬移
    • %files 成列出真正要打包的檔案列表,若沒列的話,RPM 會產生不了
    • %clean 打包完 rpm 後的動作,例如清掉資料等
  • 其他資訊
除了這些以外,spec 還定義了 package info:
  • Summary: 
  • Name:
  • Version: 1.0
  • Release: 1
  • Vendor: vendor
  • Packager: packager
  • License: BSD
  • Group: Applications/File
  • SOURCE: your_source_be_extraced_at_setup_macro.tar.gz
  • BuildArch: noarch
  • BuildRoot: %BuildRoot
範例:

%define Name test-repo
%define Source test-repo.tar.gz
%define Version 1.0
%define Release 1
%define RPM_ARCH x64_86
%define OWNER root
%define Vendor no
%define Packager no
%define License MIT
%define Group Applications/File

%define INSTALL_DIR /opt/%NAME

%define _topdir /tmp/rpm-build
%define RPM_BUILD_ROOT _topdir
%define BuildRoot /tmp/rpm-build/BUILDROOT/%Name-%Version-%Release.%RPM_ARCH

Summary: %Name
Name: %Name
Version: %Version
Release: %Release
Vendor: %Vendor
Packager: %Packager
License: %License
Group: %Group
SOURCE: %Source

%description

%prep

%setup -q

%build

%install
install -d %BuildRoot/%INSTALL_DIR
cp -r $PACKAGE_DIR/*.* %BuildRoot/%INSTALL_DIR

%files
%INSTALL_DIR
%defattr(-,%OWNER,%OWNER,-)

%changelog


最後,我打包成一個 bash script 來筆記 :p 有需要可以試試看,但要留意 bash script 有刪檔的動作,若擔心可以先把刪檔的動作註解起來:

$ git clone https://github.com/changyy/rpm-builder
$ cd rpm-builder
$ bash rpm_build_for_files_archive.sh
Usage> bash rpm_build_for_files_archive.sh SOURCE_DIR RPM_OUTPUT_DIR RPM_PACKAGE_NAME RPM_PACKAGE_VERSION RPM_PACKAGE_VERSION RPM_PACKAGE_INSTALL_DIR

$ bash rpm_build_for_files_archive.sh /etc/yum.repos.d/ /tmp test-yum 1.0 1 /opt


產出 /tmp/test-yum-repos-1.0-1.noarch.rpm ,會安裝在 /opt/test-yum 目錄

最後提一下 rpm 相關指令,方便 debug 用途:
  • 安裝
    • $ rpm -ivh your.rpm
  • 解除安裝 
    • $ rpm -e your_package_name
  • 查詢描述資料
    • $ rpm -qi your_package_name
  • 列出 rpm 所安裝的檔案清單
    • $ rpm -q --filesbypkg your_package_name
  • 查詢檔案是屬於哪個 rpm 
    • $ rpm -qf /opt/path/filename
  • 列出所有已安裝的 rpm 清單,可搭配 grep 挑選關鍵字出來
    • $ rpm -qa

2015年11月2日 星期一

[Linux] 透過 yum 安裝 Nginx 官方版 @ CentOS 6.6、nginx-1.8.0

最近在整理 yum server 才驚覺 nginx 沒有在 CentOS package 裡頭,同理 tmux 也是 Orz

$ vim /etc/yum.repos.d/nginx.repo
name=nginx repo
baseurl=http://nginx.org/packages/centos/6/$basearch/
gpgcheck=0
enabled=1

$ yum update
$ yum search nginx
Loaded plugins: security
============================================================= N/S Matched: nginx =============================================================
nginx-debug.x86_64 : debug version of nginx
nginx-debuginfo.x86_64 : Debug information for package nginx
nginx-nr-agent.noarch : New Relic agent for NGINX and NGINX Plus
nginx.x86_64 : High performance web server

$ yum install nginx
$ yum info nginx
Installed Packages
Name        : nginx
Arch        : x86_64
Version     : 1.8.0
Release     : 1.el6.ngx
Size        : 872 k
Repo        : installed
From repo   : nginx
Summary     : High performance web server
URL         : http://nginx.org/
License     : 2-clause BSD-like license
Description : nginx [engine x] is an HTTP and reverse proxy server, as well as
            : a mail proxy server.


對照 nginx.org 官方版本公告,其中 nginx-1.8.0 是 2015/04/21 釋放出的穩定版。

2015年10月30日 星期五

[Linux] Python Fabric - 在 SSH 架構上,進行服務部署 @ Ubuntu 14.04

強者同事又獻技 fab 指令,從官網簡介:

Fabric is a Python (2.5-2.7) library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks.

原來如此!我還活在播接時代再把玩 bash script XDD 其實之前也有試過一些同時發 ssh remote command 的工具,但悲劇收場後,就沒再用了,現在就在來吸收這套吧!

$ sudo apt-get install fabric
$ fab
Fatal error: Couldn't find any fabfiles!

Remember that -f can be used to specify fabfile path, and use -h for help.

Aborting.


是的, fab 指令會尋找當前目錄下的 fabfile.py:

$ vim fabfile.py
def hello(what):
print 'hello ' + str(what)


$ fab hello:world
hello world

Done.


非常簡潔明瞭,反正 fab 後面接的字串就是個 fabfile.py 裡定義的函數,冒號後面就是給進去的參數。接著要寫一些遠端部署指令就是寫個 function 定義一些想傳的參數,接著就愉快地寫 python 就好!可以省去用一堆 ssh target_host "remote_command" 的方式了!

$ vim fabfile.py
from fabric.api import run, task, execute, runs_once
from fabric.colors import green, magenta
import fabric.api as api

@task(default=True)
def heartbeat():
        _info("hearbeat")
run("date")

def _info(message):
        print green(message + " on %s" % api.env.host_string)

def _warn(message):
        print magenta(message + " on %s" % api.env.host_string)
$ fab
hearbeat on None
No hosts found. Please specify (single) host string for connection: localhost
[localhost] run: date
[localhost] Login password for 'user':
[localhost] out: Fri Oct 30 12:51:51 UTC 2015
[localhost] out:


Done.
Disconnecting from localhost... done.

$ fab -H localhost
[localhost] Executing task 'heartbeat'
hearbeat on localhost
[localhost] run: date
[localhost] Login password for 'user':
[localhost] out: Fri Oct 30 12:53:09 UTC 2015
[localhost] out:


Done.
Disconnecting from localhost... done.


接下來,就可以想想一連串的行為了!

[Linux] 架設私有 Yum Server 服務 @ CentOS 6.6

經強者同事的經驗傳授,對於線上服務機器應追蹤限定軟體的使用版本,可降低環境複雜度,以便偵錯。因此,學一下架設 YUM Server 來提供服務吧!

原理就是挑一台機器留個 30GB 空間,把官方的 Packages 都下載回來,接著更新其他台機器的 repo 位置即可。此例以 CentOS 6.6 x86_64 為例。

$ yum install yum-arch createrepo nginx

其中 nginx 只是為了 web server 而已,這是 CentOS yum server 架構,透過 filesytem + sqlite + http server。

$ yum install nginx
$ service nginx start
$ mkdir -p /var/www
$ ln -s /usr/share/nginx/html/ /var/www/html


接著要將資料 Yum package 都下載下來,先確認來源是不是官方版,也可以將 /etc/yum.repos.d/ 清到只剩 CentOS-Base.repo 檔案即可:

$ vim /etc/yum.repos.d/CentOS-Base.repo
[base]
name=CentOS-$releasever - Base
baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

#released updates
[updates]
name=CentOS-$releasever - Updates
baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras
baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus
baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

$ mkdir -p /var/www/html/centos/6/sync
$ cd /var/www/html/centos/6/sync
$ time reposync --repoid=updates --repoid=extras --repoid=base --repoid=centosplus
...
$ createrepo /var/www/html/centos/6/sync/base/
$ createrepo /var/www/html/centos/6/sync/updates/
$ createrepo /var/www/html/centos/6/sync/extras/
$ createrepo /var/www/html/centos/6/sync/centosplus/
Spawning worker 0 with ### pkgs
Workers Finished
Gathering worker results

Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete

$ mkdir -p /var/www/html/centos/6/updates /var/www/html/centos/6/extras/ /var/www/html/centos/6/os/ /var/www/html/centos/6/centosplus /var/www/html/centos/6/contrib
$ cd /var/www/html/centos/6/updates && ln -s ../sync/updates/ x86_64 && cd -
$ cd /var/www/html/centos/6/extras && ln -s ../sync/extras/ x86_64 && cd -
$ cd /var/www/html/centos/6/os && ln -s ../sync/base/ x86_64 && cd -
$ cd /var/www/html/centos/6/centosplus && ln -s ../sync/centosplus/ x86_64 && cd -


如此一來,搭配 web server 就可以存取這些位置:

http://public_ip/centos/6/updates
http://public_ip/centos/6/extras
http://public_ip/centos/6/os
http://public_ip/centos/6/centosplus


接著,其他 yum client 的 /etc/yum.repos.d/CentOS-Base.repo 位置更新為:

[base]
name=CentOS-$releasever - Base
baseurl=http://public_ip/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

#released updates
[updates]
name=CentOS-$releasever - Updates
baseurl=http://public_ip/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras
baseurl=http://public_ip/centos/$releasever/extras/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus
baseurl=http://public_ip/centos/$releasever/centosplus/$basearch/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6


接著在做個 yum update 後,其 packages 即可被控管使用,而製作私有的 package 時,大概就多添加個區段,並把 gpgcheck 設定成 0:

$ cat /etc/yum.repos.d/self.repo
[self]
name=SelfYumServer-$releasever
baseurl=http://public_ip/centos/$releasever/self/$basearch/
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6


此外,對於私有 Yum server 來說,每次變動 /var/www/html/centos/6/self/x64_86/Packages 後,都必須執行 createrepo /var/www/html/centos/6/self/x64_86/ 來更新狀態,如此一來別人就能找到。

最後,若 client 出現無法找到最新版的情況,請試試在 client server 清掉 cache,屆時重應該就會找到:

$ yum clean metadata dbcache 或 $ yum clean all