[PHP] 下載 Maxmind GeoIP Legacy Databases 和 ngx_http_geoip_module 相關處理 @ macOS 11.2, PHP 7.3.24

由於 Maxmind 在推 GeoIP2 ，不開放 .dat 的 GeoIP DB 了，以前下載位置：

• http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
• http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz

現在官方都說要改用 GeoLite2 DB ，其格式：

• GeoIP2 Binary (.mmdb)
• GeoIP2 CSV 

目前先偷懶不處理 nginx ，因為 nginx 採用 GeoIP Legacy Databases。

有找到一個網站 https://www.miyuru.lk/geoiplegacy，從中下載：

• https://dl.miyuru.lk/geoip/maxmind/city/maxmind4.dat.gz

連續動作：

% php -v

WARNING: PHP is not recommended

PHP is included in macOS for compatibility with legacy software.

Future versions of macOS will not include PHP.

PHP 7.3.24-(to be removed in future macOS) (cli) (built: Dec 21 2020 21:33:25) ( NTS )

Copyright (c) 1997-2018 The PHP Group

Zend Engine v3.3.24, Copyright (c) 1998-2018 Zend Technologies

% php composer.phar require geoip/geoip:~1.16

% wget https://dl.miyuru.lk/geoip/maxmind/city/maxmind4.dat.gz

% gunzip -d maxmind4.dat.gz

% cat test-ip-via-geoip.php

<?php

require 'vendor/autoload.php';

$gi = geoip_open("maxmind4.dat",GEOIP_STANDARD);

$country = geoip_country_code_by_addr($gi, $argv[1]);

echo "lookup [".$argv[1]."], result: [$country]\n";

% php test-ip-via-geoip.php 8.8.8.8

lookup [8.8.8.8], result: [US]

% nslookup tw.yahoo.com

Server: 8.8.8.8

Address: 8.8.8.8#53

Non-authoritative answer:

tw.yahoo.com canonical name = atsv2-fp-shed.wg1.b.yahoo.com.

Name: atsv2-fp-shed.wg1.b.yahoo.com

Address: 180.222.102.201

Name: atsv2-fp-shed.wg1.b.yahoo.com

Address: 180.222.102.202

% php test-ip-via-geoip.php 180.222.102.202

lookup [180.222.102.202], result: [IN]

% curl ipinfo.io/180.222.102.202

{

  "ip": "180.222.102.202",

  "hostname": "media-router-fp74.prod.media.vip.tp2.yahoo.com",

  "city": "Taoyuan City",

  "region": "Taiwan",

  "country": "TW",

  "loc": "24.9937,121.2970",

  "org": "AS24506 YAHOO! TAIWAN",

  "timezone": "Asia/Taipei",

  "readme": "https://ipinfo.io/missingauth"

}

% nslookup facebook.com

Server: 8.8.8.8

Address: 8.8.8.8#53

Non-authoritative answer:

Name: facebook.com

Address: 31.13.87.36

% curl ipinfo.io/31.13.87.36

{

  "ip": "31.13.87.36",

  "hostname": "edge-star-mini-shv-01-tpe1.facebook.com",

  "city": "Hong Kong",

  "region": "Central and Western",

  "country": "HK",

  "loc": "22.2783,114.1747",

  "org": "AS32934 Facebook, Inc.",

  "timezone": "Asia/Hong_Kong",

  "readme": "https://ipinfo.io/missingauth"

}

% nslookup www.gov.tw

Server: 8.8.8.8

Address: 8.8.8.8#53

Non-authoritative answer:

Name: www.gov.tw

Address: 223.200.155.55

% php test-ip-via-geoip.php 223.200.155.55

lookup [223.200.155.55], result: [TW]

% curl ipinfo.io/223.200.155.55

{

  "ip": "223.200.155.55",

  "hostname": "223-200-155-55.hinet-ip.hinet.net",

  "city": "Hualien City",

  "region": "Taiwan",

  "country": "TW",

  "loc": "23.9769,121.6044",

  "org": "AS4782 Data Communication Business Group",

  "timezone": "Asia/Taipei",

  "readme": "https://ipinfo.io/missingauth"

}

Ansible 筆記 - 設定 Nginx 啟用 ngx_http_geoip_module.so 片段程式 @ Ubuntu 14.0

其實該寫成 Ansible Role 的，但一時之間有點懶，先把練習的片段紀錄一下。

安裝 nginx-module-geoip：

$ cat geoip-nginx.yml
---
    - name: install geoip packages
      apt: name={{ item }} update_cache=yes state=latest
      with_items:
        - nginx-module-geoip
      when: install_package is defined and install_package

    - name: check maxmind db - http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
      command: bash -c 'test -e /data/GeoIP.dat || curl http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz | gunzip - > /data/GeoIP.dat'

    - name: check maxmind db - http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
      command: bash -c 'test -e /data/GeoLiteCity.dat || curl http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz | gunzip - > /data/GeoLiteCity.dat'

設定 nginx.conf：

$ cat server-deploy.yml
...
  tasks:
    ...
    - include: geoip-nginx.yml
    - name: setup nginx config file - add modules/ngx_http_geoip_module.so
      lineinfile:
        dest: /etc/nginx/nginx.conf
        insertbefore: '^user '
        line: 'load_module "modules/ngx_http_geoip_module.so";'

    - name: setup nginx config file - setup geoip resource path
      lineinfile:
        dest: /etc/nginx/nginx.conf
        insertafter: '^http {'
        line: '    geoip_country /data/GeoIP.dat; geoip_proxy 192.168.0.0/16; geoip_proxy 10.0.0.0/24; geoip_proxy_recursive on;'
        #line: '    geoip_city    /data/GeoLiteCity.dat;'

    - name: setup nginx file to enable geoip - update log format
      lineinfile:
        dest: /etc/nginx/nginx.conf
        regexp: ".*?http_user_agent.*?http_x_forwarded_for"
        line: "                      '\"$http_user_agent\" \"$http_x_forwarded_for\"' $geoip_country_code \"$geoip_country_name\" ;"

    - name: setup nginx file to enable geoip - add GEOIP_COUNTRY_CODE for FastCGI
      lineinfile:
        dest: /etc/nginx/fastcgi_params
        insertbefore: "^fastcgi_param QUERY_STRING"
        line: "fastcgi_param     GEOIP_COUNTRY_CODE $geoip_country_code;"

如此一來，產生的 nginx.conf 如下：

$ cat /etc/nginx/nginx.conf && sudo nginx -t

load_module "modules/ngx_http_geoip_module.so";
user www-data;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    geoip_country /data/GeoIP.dat; geoip_proxy 192.168.0.0/16; geoip_proxy 10.0.0.0/24; geoip_proxy_recursive on;
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"' $geoip_country_code "$geoip_country_name" ;

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

查看 /var/log/nginx/access.log 就可以看到類似的範例：

.... CN,CHN,"China"
.... US,USA,"United States"

更多變數定義，請參考：http://nginx.org/en/docs/http/ngx_http_geoip_module.html

[PHP] 簡易的 Web Service Unit Test: HTML validator、JSON Format @ Ubuntu 14.04

把 web service 移到 AWS 上後，想到時時刻刻都人工測試感到很 Orz 於是想了一些簡易 Unit Test 方式：

• 檢驗網頁是否可讀取正常 (strlen)
• 檢驗網頁 HTML 格式 (tidy 或 xmllint)
• 檢驗網頁上的 link 個數 (preg_match_all)
• 檢驗 JSON 格式 (json_decode && json_last_error)

剩下的就...再看看 XD

其他筆記：changyy / web-service-unit-test / phpunit / web-service-unit-test.php