```
const { spawn } = require('node:child_process');
const ls = spawn('ls', ['-lh', '/usr']);
ls.stdout.on('data', (data) => {
console.log(`stdout: ${data}`);
});
ls.stderr.on('data', (data) => {
console.error(`stderr: ${data}`);
});
ls.on('close', (code) => {
console.log(`child process exited with code ${code}`);
});
```
然而,有沒有可能在 node.js 內,直接做 python 直譯的過程等方式?當個樂子,找了一下,還真的有,這樣搞的優勢是降低環境部署的變因,當然,效率上不見得是好辦法,但可以讓不同語言的開發者進行融合(誤),目前看到兩種整合方式:
其中 Brython 屬於設計在 Web Browser 下運行(需要 DOM 資源),而 Pyodide 則不需要。分別筆記一下用法。
首先是要執行的 python code 內有 python 的 re 跟 json 模組的使用:
```
% cat script.py
import re
import json
def runTest(inputData):
output = {}
flags = 0
pattern = r'''(?x)
(?:
\.get\("n"\)\)&&\(b=|
(?:
b=String\.fromCharCode\(110\)|
(?P<str_idx>[a-zA-Z0-9_$.]+)&&\(b="nn"\[\+(?P=str_idx)\]
),c=a\.get\(b\)\)&&\(c=|
\b(?P<var>[a-zA-Z0-9_$]+)=
)(?P<nfunc>[a-zA-Z0-9_$]+)(?:\[(?P<idx>\d+)\])?\([a-zA-Z]\)
(?(var),[a-zA-Z0-9_$]+\.set\("n"\,(?P=var)\),(?P=nfunc)\.length)'''
try:
result = re.search(pattern, inputData, flags)
if result:
output["status"] = True
output["data"] = result.groupdict()
except Exception as e:
output["error"] = str(e)
return json.dumps(output, indent=4)
runTest(data)
```
Pyodide 用法:
```
% nvm use v20
Now using node v20.10.0 (npm v10.2.3)
% npm install pyodide
% cat package.json
{
"dependencies": {
"pyodide": "^0.26.2"
}
}
% cat run.js
const fs = require('fs').promises;
const { loadPyodide } = require("pyodide");
async function main() {
const fileContent = await fs.readFile('mydata.bin', 'utf8');
let pyodide = await loadPyodide();
pyodide.globals.set("data", pyodide.toPy(fileContent));
const pythonCode = await fs.readFile('script.py', 'utf8');
let result = pyodide.runPython(pythonCode);
console.log(result);
}
main();
% echo "Hello World" > mydata.bin
% node run.js
{}
```
上述使用過程算直觀,但偷懶把要傳遞的資料設定在全域變數,在用 node.js 環境接住 python 運算的結果,看來這個效果是很 OK ,有正常運行得到期待的結果。
接著研究 Brython 用法,他設計上需要 Browser 環境做事:
% head -n 9 brython.js
// brython.js brython.info
// version [3, 11, 0, 'final', 0]
// implementation [3, 11, 3, 'dev', 0]
// version compiled from commented, indented source files at
// github.com/brython-dev/brython
var __BRYTHON__=__BRYTHON__ ||{}
try{
eval("async function* f(){}")}catch(err){console.warn("Your browser is not fully supported. If you are using "+
"Microsoft Edge, please upgrade to the latest version")}
在 node.js 需要 jsdom 模擬一些環境,而下載 brython.js 和 brython_stdlib.js 則參考官網文件,透過 pip install brython 工具出來使用,所以這邊的流程會多了 python 工具的安裝,且現況用 python 3.12 會顯示有些問題,就先定在 3.11 版。此外 brython.js 運行環境,也是可以用最新版 node.js v22 ,但是會看到 [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. 訊息,所以先退到 node.js v20 避免額外的訊息
連續動作:
```
% python3.11 -m venv venv
% source venv/bin/activate
(venv) % pip install brython
Collecting brython
Using cached brython-3.11.3-py3-none-any.whl.metadata (1.0 kB)
Using cached brython-3.11.3-py3-none-any.whl (1.6 MB)
Installing collected packages: brython
Successfully installed brython
(venv) % brython-cli install
Installing Brython 3.11.3
done
(venv) % ls
README.txt brython_stdlib.js index.html venv
brython.js demo.html unicode.txt
```
接著回到 node.js 主場:
```
% nvm use v20
Now using node v20.10.0 (npm v10.2.3)
% cat package.json
{
"dependencies": {
"jsdom": "^24.1.1"
}
}
% cat run-via-dom.js
const { JSDOM } = require('jsdom');
const fs = require('fs');
const path = require('path');
const dom = new JSDOM(`<!DOCTYPE html>
<html>
<head></head>
<body>
<script></script>
</body>
</html>`, {
runScripts: "dangerously",
resources: "usable"
});
const brythonJsPath = path.join(__dirname, 'brython.js');
const brythonStdlibJsPath = path.join(__dirname, 'brython_stdlib.js');
const brythonJs = fs.readFileSync(brythonJsPath, 'utf8');
const brythonStdlibJs = fs.readFileSync(brythonStdlibJsPath, 'utf8');
try {
dom.window.eval(brythonJs);
dom.window.eval(brythonStdlibJs);
} catch (error) {
console.error('Error executing brython.js:', error);
}
const scriptPath = path.join(__dirname, 'script.py');
const pythonScript = fs.readFileSync(scriptPath, 'utf8');
const dataPath = path.join(__dirname, 'mydata.bin');
const binaryData = fs.readFileSync(dataPath, 'utf8');
const base64Data = Buffer.from(binaryData).toString('base64');
const scriptElement = dom.window.document.createElement('script');
scriptElement.type = 'text/python';
scriptElement.textContent = `
import base64
data = base64.b64decode("""${base64Data}""")
${pythonScript}
from browser import document
document.output = runTest(data)
`
dom.window.document.body.appendChild(scriptElement);
try {
dom.window.brython({debug: 1, pythonpath: ['.']})
console.log(dom.window.document.output);
} catch (error) {
console.error('Error executing dom.window.brython:', error);
}
console.log('Python script execution completed.');
% echo "Hello World" > mydata.bin
% node run-via-dom.js
{"status": false, "data": {}, "error": "not the same type for string and pattern"}
Python script execution completed.
```
很可惜的,剛好要實驗複雜的 python regular expression,在 brython.js + node.js v20 + jsdom 環境上失敗了,甚至小改 index.html 搭配 python3 -m http.server 用 Chrome browser 執行(給予他完整的 Chrome 瀏覽器環境)還是有一樣的錯誤訊息,這邊就暫時推論失敗了,而上述的範例已經包括從 node.js 傳資料到 python code ,以及運行完如何把回傳資料傳到 node.js 使用,眼尖的人,應該會發現在 brython 用法內,使用了 `document.output = runTest(data)` ,其實是多呼叫了一次 runTest(data),因為原先 `${pythonScript}` 也有做,但沒在細追怎樣接運算結果,剛好不合預期就放棄研究。
最後,就是 WebAssembly 領域(一開始寫這篇筆記就是要研究 WebAssembly ,不小心走偏),把某一種 python code 轉成 wasm 格式,接著用 wasmer 運行,或是在其他語言(如 node.js)運行 wasm code。
先透過 MacPorts 安裝 wasmer:
% port search wasmer
wasmer @4.3.5 (lang, devel)
The leading WebAssembly Runtime supporting WASI and Emscripten
% sudo port install wasmer
接著試著用 py2wasm 把 script-main.py 轉成 script-main.wasm,其中 py2wasm 官網有提到目前僅支援 python3.11:
% python3.11 -m venv venv
% source venv/bin/activate
(venv) % pip install py2wasm
(venv) % py2wasm script-main.py -o script-main.wasm
程式碼:
```
% cat script-main.py
import re
import json
def runTest(inputData):
output = { "status": False, "data": {}, "error": None}
flags = 0
pattern = r'''(?x)
(?:
\.get\("n"\)\)&&\(b=|
(?:
b=String\.fromCharCode\(110\)|
(?P<str_idx>[a-zA-Z0-9_$.]+)&&\(b="nn"\[\+(?P=str_idx)\]
),c=a\.get\(b\)\)&&\(c=|
\b(?P<var>[a-zA-Z0-9_$]+)=
)(?P<nfunc>[a-zA-Z0-9_$]+)(?:\[(?P<idx>\d+)\])?\([a-zA-Z]\)
(?(var),[a-zA-Z0-9_$]+\.set\("n"\,(?P=var)\),(?P=nfunc)\.length)'''
try:
result = re.search(pattern, inputData, flags)
if result:
output["status"] = True
output["data"] = result.groupdict()
except Exception as e:
output["error"] = str(e)
return json.dumps(output, indent=4)
if __name__ == "__main__":
import sys
if len(sys.argv) < 2:
print("Usage: python script-main.py <inputData>")
else:
print(runTest(sys.argv[1]))
% python3 script-main.py
Usage: python script-main.py <inputData>
% python3 script-main.py "Hello World"
{
"status": false,
"data": {},
"error": null
}
```
wasmer 實測:
```
% wasmer run script-main.wasm
Usage: python script-main.py <inputData>
% wasmer run script-main.wasm "Hello World"
{
"status": false,
"data": {},
"error": null
}
```
接著讓 Node.JS 來運行,這邊就來煩 ChatGPT 並小改一下,有了一個比較堪用的版本:
```
% cat run.js
const fs = require('fs');
const { WASI } = require('wasi');
const path = require('path');
const { TextDecoder } = require('util');
const runWasm = async (inputData) => {
const wasmPath = path.resolve('./script-main.wasm');
const wasmBinary = fs.readFileSync(wasmPath);
// Setup a WASI instance
const wasi = new WASI({
args: inputData ? ['script-main.wasm', inputData] : ['script-main.wasm'],
env: {},
version: 'preview1'
});
// Create a memory buffer for the stdout
const memory = new WebAssembly.Memory({ initial: 1 });
// Compile and instantiate the WebAssembly module
const { instance } = await WebAssembly.instantiate(wasmBinary, {
wasi_snapshot_preview1: wasi.wasiImport,
env: { memory }
});
// Start the WASI instance
wasi.start(instance);
// Read and decode the stdout data
const stdout = new Uint8Array(memory.buffer);
const decoder = new TextDecoder('utf8');
const output = decoder.decode(stdout);
console.log(output.trim());
};
// Get the inputData from the command line arguments
runWasm(process.argv[2] || null).catch(console.error);
% nvm use v20
Now using node v20.10.0 (npm v10.2.3)
% node run.js
(node:22046) ExperimentalWarning: WASI is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
Usage: python script-main.py <inputData>
% node run.js "Hello World"
(node:22050) ExperimentalWarning: WASI is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
{
"status": false,
"data": {},
"error": null
}
```
回過頭來,故事起源是想善用一些 open source 甚至不同程式語言的整合架構,因此稍微研究一些跨語言的整合,很可惜的,最佳的路線應當還是各自跑在各自的 runtime 環境,以上就當趣味筆記一下。