RPC Technology and Its Framework Sekiro’s Application in Web Scraping Reverse Engineering

一、What is RPC

RPC, short for Remote Procedure Call, is a conceptual technology rather than a specific specification or protocol. The emergence of RPC is closely tied to the development of distributed systems, primarily addressing the issue of inter-service communication within such systems. It makes remote calls as convenient as local calls, allowing the caller to be unaware of the remote invocation logic. Compared to the HTTP protocol, RPC uses binary bytecode transmission, making it more efficient and secure.

The complete RPC architecture is illustrated in the following diagram:

RPC Architecture

二、JSRPC

In reverse engineering, RPC can be thought of as a simple communication between a local environment and a browser, akin to a server-client relationship, communicating via the WebSocket protocol. Exposing encryption functions in the browser and directly calling the corresponding encryption functions locally can yield encrypted results, saving a significant amount of reverse engineering debugging time.

三、An Example

Let’s demonstrate the specific use of RPC in reverse engineering with the example of login on a web page of a certain platform.

  • Homepage (base64): $aHR0cHM6Ly9wYXNzcG9ydC5tZWl0dWFuLmNvbS9hY2NvdW50L3VuaXRpdmVsb2dpbg==$
  • Parameter: $h5Fingerprint$

First, capture the login interface request, and you will find an excessively long parameter, h5Fingerprint.

A direct search will lead you to the encryption function:

In this function, utility.getH5fingerprint() formatted with the parameter window.location.origin + url, the parameters are as follows:

url = "https://passport.obfuscated.com/account/unitivelogin"
params = {
    "risk_partner": "0",
    "risk_platform": "1",
    "risk_app": "-1",
    "uuid": "96309b5f00ba4143b920.1644805104.1.0.0",
    "token_id": "DNCmLoBpSbBD6leXFdqIxA",
    "service": "www",
    "continue": "https://www.obfuscated.com/account/settoken?continue=https%3A%2F%2Fwww.obfuscated.com%2F"
}

Next, using RPC technology, directly call the utility.getH5fingerprint() method in the browser. First, write local server-side code to input the string to be encrypted, receive and print the encrypted string.

import sys
import asyncio
import websockets

async def receive_massage(websocket):
    while True:
        send_text = input("请输入要加密的字符串: ")
        if send_text == "exit":
            print("Exit, goodbye!")
            await websocket.send(send_text)
            await websocket.close()
            sys.exit()
        else:
            await websocket.send(send_text)
            response_text = await websocket.recv()
            print("\n加密结果:", response_text)

start_server = websockets.serve(receive_massage, '127.0.0.1', 5678)  # 自定义端口
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()

Write browser client-side JS code that, upon receiving a message, directly calls utility.getH5fingerprint() to obtain the encrypted parameter and send it to the server:

var ws = new WebSocket("ws://127.0.0.1:5678");  // 自定义端口

ws.onmessage = function (evt) {
    console.log("Received Message: " + evt.data);
    if (evt.data == "exit") {
        ws.close();
    } else {
        ws.send(utility.getH5fingerprint(evt.data))
    }
};

Then we need to inject the client-side code into the web page. There are many methods to achieve this, such as using packet capture software like Fiddler to replace the response, using browser plugins like ReRes to replace JavaScript, using browser developer tools Overrides to rewrite functionality, and also injecting through plugins, Tampermonkey, and other hook injection methods. For those not familiar with these methods, you can refer to K’s previous articles, which provide introductions to these techniques.

Here, we will use the browser developer tools Overrides to rewrite functionality. We will add the WebSocket client-side code to the encrypted JavaScript file and save it by pressing Ctrl+S. Here, we have written it as an IIFE (Immediately Invoked Function Expression) to prevent global variable pollution, although it could also be done without using the IIFE approach.

img_3.png

Next, run the local server-side code and log in to the web page once, twice, three times - it’s important to emphasize this step three times! After that, you can input the string to be encrypted locally and obtain the result of utility.getH5fingerprint() encryption:

img_4.png

IV. Sekiro

From the previous examples, it’s evident that writing our own server is quite cumbersome and not easily scalable. Are there any ready-made solutions in this area? The answer is yes, and here are two projects to consider:

JsRPC-hliang is written in Go and is a project specifically designed for JavaScript reverse engineering. On the other hand, Sekiro is more powerful. It is a framework developed by Deng Weijia, commonly known as “Zha Zong,” based on long connections and code injection for exposing Android Private APIs. It can be used in scenarios such as app reverse engineering, app data acquisition, and Android group control. Additionally, Sekiro is currently the only stable JSRPC framework in the public domain. Both projects have similar methods for JavaScript reverse engineering. This article mainly focuses on the application of Sekiro in web JavaScript reverse engineering.

Referring to the Sekiro documentation, first, compile the project locally:

  • For Linux & Mac: run the script build_demo_server.sh, which will produce the release package: sekiro-service-demo/target/sekiro-release-demo.zip
  • For Windows: the package can be directly downloaded from: https://oss.virjar.com/sekiro/sekiro-demo

Then, run it locally (requires Java environment, configure it yourself):

  • For Linux & Mac: run bin/sekiro.sh
  • For Windows: run bin/sekiro.bat

Inject the code into the browser: inject the provided sekiro_web_client.js (download link: https://sekiro.virjar.com/sekiro-doc/assets/sekiro_web_client.js) into the browser environment. Then, communicate with the SekiroClient and the Sekiro server, enabling direct RPC calls to browser internal methods.

In the wss link, if using the free version, change “business” to “business-demo”. Let’s explain some of the terms involved:

  • group: business type (interface group), each business has a group, and under the group, multiple terminals (SekiroClient) can be registered. At the same time, multiple actions can be mounted under the group.
  • clientId: refers to the device. Multiple devices use multiple machines to provide API services, providing group control capabilities and load balancing capabilities.
  • SekiroClient: service provider client, mainly used for mobile phones/browsers, etc. The final Sekiro call will be forwarded to SekiroClient. Each client needs to have a unique clientId.
  • registerAction: interface. Under the same group, there can be multiple interfaces, each performing different functions.
  • resolve: method to return content to the client.
  • request: the request from the client. If there are multiple parameters in the request, you can extract the parameters as key-value pairs and then process them.

After all this explanation, it might still be a bit difficult to understand. Let’s jump straight into a practical example. Taking a web login on a certain platform as an example, we will combine sekiro_web_client.js with the SekiroClient communication code. Then, according to our needs, we will modify the communication code section as follows:

  • Change the ws link to: ws://127.0.0.1:5620/business-demo/register?group=rpc-test&clientId=, customizing the group as rpc-test.
  • Register an event registerAction as getH5fingerprint.
  • The result returned by resolve will be utility.getH5fingerprint(request["url"]), encrypting and returning the url parameter passed by the client.

The complete code is as follows:

(function () {
    'use strict';
    function SekiroClient(wsURL) {
        this.wsURL = wsURL;
        this.handlers = {};
        this.socket = {};
        // check
        if (!wsURL) {
            throw new Error('wsURL can not be empty!!')
        }
        this.webSocketFactory = this.resolveWebSocketFactory();
        this.connect()
    }

    SekiroClient.prototype.resolveWebSocketFactory = function () {
        if (typeof window === 'object') {
            var theWebSocket = window.WebSocket ? window.WebSocket : window.MozWebSocket;
            return function (wsURL) {

                function WindowWebSocketWrapper(wsURL) {
                    this.mSocket = new theWebSocket(wsURL);
                }

                WindowWebSocketWrapper.prototype.close = function () {
                    this.mSocket.close();
                };

                WindowWebSocketWrapper.prototype.onmessage = function (onMessageFunction) {
                    this.mSocket.onmessage = onMessageFunction;
                };

                WindowWebSocketWrapper.prototype.onopen = function (onOpenFunction) {
                    this.mSocket.onopen = onOpenFunction;
                };
                WindowWebSocketWrapper.prototype.onclose = function (onCloseFunction) {
                    this.mSocket.onclose = onCloseFunction;
                };

                WindowWebSocketWrapper.prototype.send = function (message) {
                    this.mSocket.send(message);
                };

                return new WindowWebSocketWrapper(wsURL);
            }
        }
        if (typeof weex === 'object') {
            // this is weex env : https://weex.apache.org/zh/docs/modules/websockets.html
            try {
                console.log("test webSocket for weex");
                var ws = weex.requireModule('webSocket');
                console.log("find webSocket for weex:" + ws);
                return function (wsURL) {
                    try {
                        ws.close();
                    } catch (e) {
                    }
                    ws.WebSocket(wsURL, '');
                    return ws;
                }
            } catch (e) {
                console.log(e);
                //ignore
            }
        }
        //TODO support ReactNative
        if (typeof WebSocket === 'object') {
            return function (wsURL) {
                return new theWebSocket(wsURL);
            }
        }
        // weex 和 PC环境的websocket API不完全一致,所以做了抽象兼容
        throw new Error("the js environment do not support websocket");
    };

    SekiroClient.prototype.connect = function () {
        console.log('sekiro: begin of connect to wsURL: ' + this.wsURL);
        var _this = this;
        // 不check close,让
        // if (this.socket && this.socket.readyState === 1) {
        //     this.socket.close();
        // }
        try {
            this.socket = this.webSocketFactory(this.wsURL);
        } catch (e) {
            console.log("sekiro: create connection failed,reconnect after 2s");
            setTimeout(function () {
                _this.connect()
            }, 2000)
        }

        this.socket.onmessage(function (event) {
            _this.handleSekiroRequest(event.data)
        });

        this.socket.onopen(function (event) {
            console.log('sekiro: open a sekiro client connection')
        });

        this.socket.onclose(function (event) {
            console.log('sekiro: disconnected ,reconnection after 2s');
            setTimeout(function () {
                _this.connect()
            }, 2000)
        });
    };

    SekiroClient.prototype.handleSekiroRequest = function (requestJson) {
        console.log("receive sekiro request: " + requestJson);
        var request = JSON.parse(requestJson);
        var seq = request['__sekiro_seq__'];

        if (!request['action']) {
            this.sendFailed(seq, 'need request param {action}');
            return
        }
        var action = request['action'];
        if (!this.handlers[action]) {
            this.sendFailed(seq, 'no action handler: ' + action + ' defined');
            return
        }

        var theHandler = this.handlers[action];
        var _this = this;
        try {
            theHandler(request, function (response) {
                try {
                    _this.sendSuccess(seq, response)
                } catch (e) {
                    _this.sendFailed(seq, "e:" + e);
                }
            }, function (errorMessage) {
                _this.sendFailed(seq, errorMessage)
            })
        } catch (e) {
            console.log("error: " + e);
            _this.sendFailed(seq, ":" + e);
        }
    };

    SekiroClient.prototype.sendSuccess = function (seq, response) {
        var responseJson;
        if (typeof response == 'string') {
            try {
                responseJson = JSON.parse(response);
            } catch (e) {
                responseJson = {};
                responseJson['data'] = response;
            }
        } else if (typeof response == 'object') {
            responseJson = response;
        } else {
            responseJson = {};
            responseJson['data'] = response;
        }


        if (Array.isArray(responseJson)) {
            responseJson = {
                data: responseJson,
                code: 0
            }
        }

        if (responseJson['code']) {
            responseJson['code'] = 0;
        } else if (responseJson['status']) {
            responseJson['status'] = 0;
        } else {
            responseJson['status'] = 0;
        }
        responseJson['__sekiro_seq__'] = seq;
        var responseText = JSON.stringify(responseJson);
        console.log("response :" + responseText);
        this.socket.send(responseText);
    };

    SekiroClient.prototype.sendFailed = function (seq, errorMessage) {
        if (typeof errorMessage != 'string') {
            errorMessage = JSON.stringify(errorMessage);
        }
        var responseJson = {};
        responseJson['message'] = errorMessage;
        responseJson['status'] = -1;
        responseJson['__sekiro_seq__'] = seq;
        var responseText = JSON.stringify(responseJson);
        console.log("sekiro: response :" + responseText);
        this.socket.send(responseText)
    };

    SekiroClient.prototype.registerAction = function (action, handler) {
        if (typeof action !== 'string') {
            throw new Error("an action must be string");
        }
        if (typeof handler !== 'function') {
            throw new Error("a handler must be function");
        }
        console.log("sekiro: register action: " + action);
        this.handlers[action] = handler;
        return this;
    };

    function guid() {
        function S4() {
            return (((1 + Math.random()) * 0x10000) | 0).toString(16).substring(1);
        }

        return (S4() + S4() + "-" + S4() + "-" + S4() + "-" + S4() + "-" + S4() + S4() + S4());
    }

    var client = new SekiroClient("ws://127.0.0.1:5620/business-demo/register?group=rpc-test&clientId=" + guid());

    client.registerAction("getH5fingerprint", function (request, resolve, reject) {
        resolve(utility.getH5fingerprint(request["url"]));
    })

})();

To inject the provided code into a web page’s JavaScript using the browser developer tools Overrides, follow these steps:

  1. Open the web page in your browser.
  2. Open the developer tools by pressing F12 or right-clicking on the page and selecting “Inspect” or “Inspect Element.”
  3. In the developer tools panel, navigate to the “Sources” tab.
  4. Find the JavaScript file where you want to inject the code, such as the main JavaScript file or any relevant file.
  5. In the “Sources” tab, locate the specific location where you want to inject the code.
  6. Copy and paste the provided code at the desired location.
  7. Save the changes by pressing Ctrl + S or right-clicking and selecting “Save” or “Save As.”

After injecting the code, manually log in to the web page as instructed. Then, you can use Sekiro’s API to call the utility.getH5fingerprint() encryption method.

Use the following API call to invoke the method:

http://127.0.0.1:5620/business-demo/invoke?group=rpc-test&action=getH5fingerprint&url=https://www.baidu.com/

Open this URL in your browser. The returned dictionary will contain the encrypted result. img_6.png

Source of this article https://blog.csdn.net/qq_36759224/article/details/123082574