yt-dlp 開發筆記 @ Python3, macOS 13.2.1

yt-dlp 是知名的網路影音網站研究的專案，內含不少各大網站的爬蟲技術。若某個服務失效想要貢獻的話，可這樣操作，邊改 code 變測試：

% git clone https://github.com/yt-dlp/yt-dlp.git

% cd yt-dlp

% python3 yt_dlp/__main__.py 'URL' --skip-download -j

以 2023/08/31 即將關站的 Xuite 為例，從 github.com/yt-dlp/yt-dlp/tree/master/yt_dlp/extractor/xuite.py 找一則 unit test 來跑：

% python3 yt_dlp/__main__.py 'http://vlog.xuite.net/embed/cE1xbENoLTI3NDQ3MzM2LmZsdg==?ar=0&as=0' -f mp4 --skip-download -j | jq '' | tail -n 25

  "format_id": "720",

  "height": 720,

  "protocol": "http",

  "resolution": "720p",

  "dynamic_range": "SDR",

  "aspect_ratio": null,

  "http_headers": {

    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.50 Safari/537.36",

    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",

    "Accept-Language": "en-us,en;q=0.5",

    "Sec-Fetch-Mode": "navigate"

  },

  "video_ext": "mp4",

  "audio_ext": "none",

  "format": "720 - 720p",

  "_filename": "男女平權只是口號？專家解釋約會時男生是否該幫女生付錢 (中字) [27447336].mp4",

  "filename": "男女平權只是口號？專家解釋約會時男生是否該幫女生付錢 (中字) [27447336].mp4",

  "_type": "video",

  "_version": {

    "version": "2023.03.04",

    "current_git_head": "b423b6a48",

    "release_git_head": "392389b7df7b818f794b231f14dc396d4875fbad",

    "repository": "yt-dlp/yt-dlp"

  }

}

至於，若要貢獻給 yt-dlp 社群，可以仿其架構，挑一個 extractor 來模仿，過程要繼承 InfoExtractor 來做事，例如實作 _VALID_URL 和 _real_extract(self, url) 等等，可以挑幾個實作沒那麼複雜的網站研究一下流程：

• github.com/yt-dlp/yt-dlp/tree/master/yt_dlp/extractor/xuite.py
• github.com/yt-dlp/yt-dlp/tree/master/yt_dlp/extractor/udn.py
• github.com/yt-dlp/yt-dlp/tree/master/yt_dlp/extractor/imgur.py
• github.com/yt-dlp/yt-dlp/tree/master/yt_dlp/extractor/sohu.py

[Linux] cross compile tmux for Synology DS216play via DSM 6.1 Tool Chains - STMicroelectronics Monaco Linux 3.10.102 @ Ubuntu 16.04 64bit

連假最後一天跑來把玩一下 cross compiler ，想說未來應該會越來越常用 NAS 做事，覺得裡頭沒有個 tmux/screen 很煩，然而，等了很久還是沒人為 DS216play 編譯 tmux ，一直很搞不懂，直到我翻了這張表出來：

https://www.synology.com/en-global/knowledgebase/DSM/tutorial/General/What_kind_of_CPU_does_my_NAS_have

System Model CPU Model Cores Threads FPU Package Arch RAM
DS216play STM STiH412 Dual Core 2 Yes Monaco DDR3 1 GB

好的，看起來有點冷門的 STM STiH412/Monaco 囧，不過找一些討論串才知道，由於要讓 DS216play 可以做一些多媒體影音的事，所以，就別抱怨了 XD

下載 Tool chains: https://sourceforge.net/projects/dsgpl/files/DSM%206.1%20Tool%20Chains/STMicroelectronics%20Monaco%20Linux%203.10.102/

$ wget https://sourceforge.net/projects/dsgpl/files/DSM%206.1%20Tool%20Chains/STMicroelectronics%20Monaco%20Linux%203.10.102/monaco-gcc493_glibc220_hard-GPL.txz/download -O monaco-gcc493_glibc220_hard-GPL.txz
$ tar -xvf monaco-gcc493_glibc220_hard-GPL.txz
$ tree -L 1 arm-unknown-linux-gnueabi/
arm-unknown-linux-gnueabi/
├── arm-unknown-linux-gnueabi
├── bin
├── build.log.bz2
├── include
├── lib
├── libexec
└── share

前置環境：

$ sudo apt install make python gcc
$ chmod 755 $HOME/arm-unknown-linux-gnueabi $HOME/arm-unknown-linux-gnueabi/include $HOME/arm-unknown-linux-gnueabi/lib $HOME/arm-unknown-linux-gnueabi/bin $HOME/arm-unknown-linux-gnueabi/share

編譯 ncurses:

$ wget ftp://invisible-island.net/ncurses/ncurses.tar.gz
$ tar -xvf ncurses.tar.gz
$ cd ncurses-5.9
$ PATH=$PATH:$HOME/arm-unknown-linux-gnueabi/bin CC=$HOME/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc CFLAGS="-I$HOME/arm-unknown-linux-gnueabi/include"  ./configure --host arm-unknown-linux-gnueabi --prefix=$HOME/arm-unknown-linux-gnueabi
$ PATH=$PATH:$HOME/arm-unknown-linux-gnueabi/bin make
$ PATH=$PATH:$HOME/arm-unknown-linux-gnueabi/bin make install

編譯 libevent:

$ wget https://github.com/libevent/libevent/releases/download/release-2.1.8-stable/libevent-2.1.8-stable.tar.gz
$ tar -xvf libevent-2.1.8-stable.tar.gz
$ cd libevent-2.1.8-stable
$ PATH=$PATH:$HOME/arm-unknown-linux-gnueabi/bin CC=$HOME/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc CFLAGS="-I$HOME/arm-unknown-linux-gnueabi/include"  ./configure --host arm-unknown-linux-gnueabi --prefix=$HOME/arm-unknown-linux-gnueabi
$ PATH=$PATH:$HOME/arm-unknown-linux-gnueabi/bin CC=$HOME/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc CFLAGS="-I$HOME/arm-unknown-linux-gnueabi/include"  make
$ PATH=$PATH:$HOME/arm-unknown-linux-gnueabi/bin make install

編譯 tmux:

$ wget https://github.com/tmux/tmux/releases/download/2.5/tmux-2.5.tar.gz
$ tar -xvf tmux-2.5.tar.gz
$ cd tmux-2.5
$ PATH=$PATH:$HOME/arm-unknown-linux-gnueabi/bin CC=$HOME/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc CFLAGS="-I$HOME/arm-unknown-linux-gnueabi/include -I$HOME/arm-unknown-linux-gnueabi/include/ncurses" LDFLAGS="-L$HOME/arm-unknown-linux-gnueabi/lib" ./configure --host arm-unknown-linux-gnueabi --prefix=$HOME/arm-unknown-linux-gnueabi
$ PATH=$PATH:$HOME/arm-unknown-linux-gnueabi/bin make
$ file tmux
tmux: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, not stripped

or:

$ PATH=$PATH:$HOME/arm-unknown-linux-gnueabi/bin CC=$HOME/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc CFLAGS="-I$HOME/arm-unknown-linux-gnueabi/include -I$HOME/arm-unknown-linux-gnueabi/include/ncurses" LDFLAGS="-L$HOME/arm-unknown-linux-gnueabi/lib" ./configure --host arm-unknown-linux-gnueabi --prefix=$HOME/arm-unknown-linux-gnueabi --enable-static
$ PATH=$PATH:$HOME/arm-unknown-linux-gnueabi/bin make
...
...
cmd-string.o: In function `cmd_string_split':
cmd-string.c:(.text+0x820): warning: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
cmd-string.c:(.text+0x760): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
$HOME/arm-unknown-linux-gnueabi/lib/libevent.a(evutil.o): In function `test_for_getaddrinfo_hacks':
evutil.c:(.text+0x1be8): warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
$HOME/arm-unknown-linux-gnueabi/lib/libevent.a(evutil.o): In function `evutil_unparse_protoname':
evutil.c:(.text+0x1554): warning: Using 'getprotobynumber' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
$HOME/arm-unknown-linux-gnueabi/lib/libevent.a(evutil.o): In function `evutil_parse_servname':
evutil.c:(.text+0x14a0): warning: Using 'getservbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
$ file tmux
tmux: ELF 32-bit LSB executable, ARM, EABI5 version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.32, not stripped

收工