109 KiB
ATEN iKVM Reverse Engineering Notes
Reverse engineering notes for the ATEN IPMI KVM viewer, extracted from JAR
iKVM__V1.69.21.0x0 and native library package liblinux_x86_64__V1.0.5.
Detailed protocol specifications are in separate files:
- KVM_PROTOCOL.md -- RFB/VNC-based KVM console protocol
- MOUNT_PROTOCOL.md -- Virtual media mount protocol
Table of Contents
- Package Contents
- Binary Overview
- Build Environment
- libiKVM64.so -- KVM Console Library
- libSharedLibrary64.so -- Virtual Media Library
- Java Application Layer
- Class Hierarchy and Vtables
- Security Analysis
- Key Data Structures and Globals
- Round 5 Verification Results
- Round 6 Final Sweep Results
- Areas for Further Investigation
Package Contents
original_binaries/client/
├── iKVM__V1.69.21.0x0/ Java classes + resources
│ ├── tw/com/aten/ikvm/ KVM viewer application
│ │ ├── KVMMain.class Entry point
│ │ ├── KVMParam.class Parameter parsing
│ │ ├── BoardInfo.class Board identification
│ │ ├── ui/ GUI components
│ │ │ ├── Viewer.class Main window
│ │ │ ├── RemoteVideo.class Video display + input (44 native methods)
│ │ │ ├── onScreenKeyboard.class
│ │ │ ├── MyKeyboardPanel.class
│ │ │ ├── OptionFrame.class Settings dialogs
│ │ │ ├── AST2050OptionFrame.class
│ │ │ ├── HermonOptionFrame.class
│ │ │ ├── SettingPanel.class
│ │ │ ├── HotKeyFrame.class
│ │ │ └── UserListFrame.class
│ │ ├── jni/ JNI bridge classes
│ │ │ ├── RMConnection.class Connection management (3 native methods)
│ │ │ └── RMDesktop.class Desktop composition wrapper
│ │ ├── util/ Utilities
│ │ │ ├── KeyMap.class VK→HID keycode translation
│ │ │ ├── Converter.class Data type conversion
│ │ │ ├── VideoRecorder.class Screen recording
│ │ │ └── ZipFile.class ZIP handling
│ │ └── bean/ Data beans
│ │ ├── KVMInfoRepository.class
│ │ ├── MouseMode.class, KeyboardMode.class
│ │ ├── ResolutionInfo.class, YUVMode.class
│ │ └── HotKey.class, KeyDef.class, etc.
│ ├── tw/com/aten/vstorage/ Virtual media storage
│ │ ├── VirtualStorage.class Main VM class (16 native methods)
│ │ └── bean/VSBtnEventInfo.class
│ ├── tw/com/aten/bean/ Shared data beans
│ │ ├── ConnInfo.class Connection info fields
│ │ ├── UserInfo.class Username/password
│ │ ├── PlatformInfo.class Platform identification
│ │ ├── VirtualUsbInfo.class VM port/config
│ │ └── InfoRepository.class Shared state
│ ├── ch/randelshofer/media/ Third-party AVI library
│ │ └── avi/AVIOutputStream.class (for screen recording)
│ └── res/ UI resources (keyboard layout PNGs)
│
└── liblinux_x86_64__V1.0.5/ Native shared libraries
├── libiKVM64.so KVM protocol engine (204,592 bytes)
└── libSharedLibrary64.so Virtual media engine (261,688 bytes)
Binary Overview
libiKVM64.so
| Property | Value |
|---|---|
| Format | ELF 64-bit LSB shared object, x86-64 |
| Size | 204,592 bytes (200 KB) |
| .text size | 0x134D8 (78,040 bytes) |
| .rodata size | 0x595B (22,875 bytes) |
| .data size | 0x3450 (13,392 bytes) |
| .bss size | 0x46F2610 (~74 MB) |
| Functions | ~576 (Ghidra), 443 exported text symbols |
| JNI exports | 47 (3 RMConnection + 44 RemoteVideo) |
| MD5 | a5cd4f2be1f674a0251c14576412c568 |
| SHA256 | 46897ab5bdaacf45f42acdaa0e95afdbeb83c06147c0cb6d78c9d4cb8eeb6df5 |
libSharedLibrary64.so
| Property | Value |
|---|---|
| Format | ELF 64-bit LSB shared object, x86-64 |
| Size | 261,688 bytes (256 KB) |
| .text size | 0x21BC8 (138,184 bytes) |
| .rodata size | 0xB7B (2,939 bytes) |
| .data size | 0x17E0 (6,112 bytes) |
| .bss size | 0x21370 (136,048 bytes) |
| Functions | ~722 (Ghidra), 341 exported text symbols |
| JNI exports | 16 (VirtualStorage) |
| MD5 | 8a02baead64cf2947c372b1218fa1751 |
| SHA256 | 32bc5f23f479e4d6fa5033d31ee90a5e8cdc46104c511dfbc588c5c93950025a |
Notable .bss size difference
libiKVM64.so has a ~74 MB .bss section, primarily due to:
- Video decoder buffers (ast_Buffer at BSS, various YUV/Huffman tables)
- Framebuffer memory (1920x1200x4 = ~9 MB + decoder working buffers)
- Multiple Huffman decode tables (HTAC, HTDC at ~384 KB combined)
- Color conversion lookup tables (Cr_tab, Cb_tab, Cr_Cb_green_tab)
Build Environment
| Property | Value |
|---|---|
| Compiler | GCC 3.4.6 20060404 (Red Hat 3.4.6-10/11) |
| Target | x86-64 Linux, System V ABI |
| C++ ABI | CXXABI_1.3, itanium-style name mangling |
| Linking | Dynamic, stripped (no debug symbols) |
| Symbols | C++ mangled names preserved in .dynsym |
| File dates | February 3, 2015 |
Both binaries were compiled with a very old GCC (3.4.6 from RHEL/CentOS 4 era).
C++ exceptions are used (__cxa_throw, __cxa_begin_catch, personality routines).
The code uses the older C++ ABI conventions.
Shared Library Dependencies
Both libraries link to:
libstdc++.so.6-- C++ standard librarylibm.so.6-- Math librarylibgcc_s.so.1-- GCC runtimelibc.so.6-- C standard library
libiKVM64.so additionally uses X11 functions loaded at runtime:
XOpenDisplay,XCloseDisplayXkbGetIndicatorState(keyboard LED state for CapsLock/NumLock/ScrollLock)XKeysymToKeycode,XTestFakeKeyEvent
libiKVM64.so -- KVM Console Library
JNI Interface
RMConnection (3 methods at tw.com.aten.ikvm.jni.RMConnection):
| Export | Address | Purpose |
|---|---|---|
Java_..._RMConnection_init |
0x00119950 | Initialize JNI field IDs for ConnInfo/UserInfo |
Java_..._RMConnection_keepActive |
0x00119a80 | Establish TCP connection via TcpSocket |
Java_..._RMConnection_checkValidUser |
0x00119c60 | Authenticate (triggers RFB handshake) |
RemoteVideo (44 methods at tw.com.aten.ikvm.ui.RemoteVideo):
| Export | Address | Category |
|---|---|---|
init |
0x0011fbd0 | Initialize video system, JNI callbacks, X11 LED state |
destory [sic] |
0x0011ffd0 | Cleanup/destruction |
runImage |
0x001201e0 | Receive + decode one frame |
updateImage |
0x00120050 | Copy framebuffer to Java ByteBuffer |
getDecodeImage |
0x001207b0 | Get decoded image data |
keyboardAction |
0x00121000 | Send keyboard event |
mouseAction |
0x001212d0 | Send mouse event |
catchLoop |
0x001216c0 | No-op (empty function body) |
doCatch |
0x00121570 | Set filterFlag global (no message processing) |
refresh |
0x00121510 | JNI buffer management (MonitorEnter/Exit, no network I/O) |
syncMouse |
0x00121500 | Send MouseSync |
sendKeepAliveAck |
0x00121a00 | Respond to server keepalive |
sendPrivilegeCtrl |
0x00121970 | Send privilege control |
hotPlug |
0x00121a30 | Mouse hot-plug reset |
setPowerOn/Off/Reset |
0x00121a50-b0 | Power control |
setScreenUILang |
0x00121b00 | Set OSD language |
setQosParameter |
0x00121c00 | Set video QoS |
changeLEDstate |
0x00120cd0 | Local X11 key sim (no network I/O) |
getLEDstate/setLEDstate |
0x00120de0/e60 | LED state get/set |
screenRecording* |
0x001216f0+ | Screen capture to AVI |
get/setMouseMode |
0x00121470/a0 | Mouse mode control |
getFPS/resetFPS |
0x00121400/f0 | FPS counter |
getScreenLevel/Mode |
0x00121410/40 | Screen info query |
getPlatVer |
0x001216e0 | Get platform version |
changeScreenInfo |
0x00120c00 | ScreenGetRect+ScreenSetInfo (type 0x32 if changed) |
updateInfo |
0x00120be0 | Calls MouseUpdateInfo (sends type 0x37) |
setReplayRecord |
0x00120bc0 | Set replay recording flag |
getPowerStatus |
0x00121ad0 | Misnamed: calls MouseGetPT() (returns mouse config, not power) |
getScreenUILang |
0x00121b50 | Query OSD language |
stopCatch |
0x001216d0 | Stop message receive loop |
JNI Callbacks (C → Java, registered in init):
| Callback | Signature | Purpose |
|---|---|---|
changeResolution |
(II)V |
Notify Java of resolution change |
errorHandler |
(I)V |
Report errors |
genKeyEvent |
(II)V |
Generate synthetic key events |
privilegeCtrl |
(II[B)V |
Privilege control notification |
setViewerConfig |
(I[B)V |
Pass viewer config to Java |
addClipBounds |
(IIII)V |
Report dirty rectangles for repaint |
getScreenUILangConf |
(II)V |
OSD language config |
getQuickCursorUpdate |
(I)V |
Quick cursor update notification |
setNormalStart |
(I)V |
Normal session start notification |
C++ Class Architecture
TcpSocket
│
StreamSocket
│
NtwStream ─── Buffered network I/O
│
RMProtocol (abstract base)
│
RFBProtocol ─── RFB 3.8 protocol handler
│
┌───────────────────────┼───────────────────────┐
│ │ │
RMDesktop RMConnection RMAuth
(composition) (connect/auth)
│
├── RMKeyboard → RFBKeyboard
├── RMMouse → RFBMouse
├── RMScreen → RFBScreen
├── RMPrivilege→ RFBPrivilege
└── RMCryto → RFBKMCryto
RMDecoder (abstract base)
│
┌───────────────┬───────┴───────┬──────────────┐
│ │ │ │
ASTVideoDecoder HermonVideo YarkonVideo Pilot3Video
│ Decoder Decoder Decoder
├── ast2100 (AST2050)
│ └── ASTDecode
└── ast_jpeg (AST2100+)
Additional: RMStorage, RMMisc, ErrMsg (exception type, 0x44 bytes: int + char[64])
Object sizes (from operator_new calls in constructors):
| Class | Allocation Size | Notes |
|---|---|---|
RMConnection |
0x10 | Contains vtable + RFBProtocol* |
RFBProtocol |
0x70 | Protocol state, fw_protocol_flag at 0x48 |
RMDesktop |
0x30 | Composition: keyboard+mouse+screen+privilege ptrs |
RFBKeyboard |
0xB8 | Includes 3 std::map lookup tables for VK->keysym |
RFBMouse |
0x28 | Includes RFBKMCryto* for mouse encryption |
RFBScreen |
0x2080 | Large: includes cursor data, frame refs |
RFBPrivilege |
0x120 | Includes 256-byte key material buffer + RFBKMCryto* |
RFBKMCryto |
0x2110 | AES state: key schedule, S-boxes, round keys |
ASTVideoDecoder |
0x58 | Wrapper for ast2100 or ast_jpeg |
HermonVideoDecoder |
0x58 | Hermon chipset decoder |
YarkonVideoDecoder |
0x60 | Yarkon chipset decoder |
Pilot3VideoDecoder |
0x458 | Pilot III chipset decoder |
ast_jpeg |
0x588 | JPEG-variant decoder (AST2100+) |
ast2100 |
0x1029E8 | VQ + Huffman decoder (AST2050), ~1 MB |
ErrMsg |
0x44 | Exception: int error_code + char[64] message |
Video Decoder Architecture
Each decoder class inherits from RMDecoder and implements:
Decode(UpdateBlocks_t&)-- Main decode entry point (virtual)GetDecodedFrame()-- Return pointer to decoded pixels (virtual)MixedCursor(CursorInfo)-- Composite hardware cursor (ASTVideoDecoder)
decoder_fun is a shared utility class used by the JPEG-like decoders:
IDCT_transform-- Inverse DCT (two implementations at 0x0010dff8, 0x001144f0)process_Huffman_data_unit-- Huffman decodeinit_JPG_decoding,init_jpg_table-- JPEG table initget_DCT,get_codebuf,get_newbits,get_newbuf-- Bitstream helpersget_CrtoR,get_limittable-- Color space helperscalculate_tabs-- Quantization table calculationWORD_hi_lo-- Byte swap utility
AST2050 (encoding 0x57) via ast2100 class:
- Contains
VQ_Decompress-- Vector Quantization decompression - Contains
RC4_crypt(at 0x0011b8f0) -- apparently uses RC4 in VQ decode InitParameter,init_QT,getKbits-- Initialization and bitstream
AST2100+ (encoding 0x58) via ast_jpeg class:
- Pure JPEG variant decoder
YUVToRGB(at 0x0010e748) -- Color space conversionPsudoStreamSwap16/32-- Byte order helpersget_decoderID,get_buffer,get_ImageData-- Accessor methodssetScreenInfo-- Resolution configurationupdatereadbuf-- Buffer management
Hermon (encoding 0x59) via HermonVideoDecoder:
ConvertVierwerPixelFormat[sic] -- Pixel format conversion- Constructor takes
(Decodeinfo, byte*, byte*)parameters
Yarkon (encoding 0x60) via YarkonVideoDecoder:
ConvertVierwerPixelFormat-- Pixel format conversion (same typo)
Pilot3 (encoding 0x61) via Pilot3VideoDecoder:
SetRect-- Rectangle operations
NtwStream -- Network I/O Layer
Buffered network stream with batched writes. Reads are unbuffered (each
StreamRead call goes directly to recv() via TcpSocket). The 1520-byte
internal buffer is exclusively for write batching.
NtwStream object layout (allocated size = 0x628 bytes):
Offset Size Type Field Notes
0x00 0x28 pthread_mutex_t write_mutex Protects write buffer
0x28 0x5F0 uchar[] write_buffer 1520-byte write batch buffer
0x618 8 uchar* write_ptr Current position in write buffer
0x620 8 TcpSocket* socket Underlying TCP socket (0x60 bytes)
Constructor at 0x0010efb0: allocates TcpSocket (0x60 bytes), stores at offset 0x620. Initializes the pthread mutex at offset 0x00. Sets write_ptr (offset 0x618) to point to the write buffer start (this + 0x28).
| Method | Address | Purpose |
|---|---|---|
StreamRead(buf, len) |
0x0010f0a0 | Read exact bytes from socket (loop until all received) |
StreamRead8() |
0x0010f0f0 | Read 1 byte |
StreamRead16() |
0x0010f110 | Read 2 bytes (big-endian) |
StreamRead32() |
0x0010f140 | Read 4 bytes (big-endian) |
StreamReadSkip(n) |
0x0010f180 | Skip n bytes (allocates temp buffer, wasteful) |
StreamWrite(buf, len) |
0x0010f1f0 | Buffer write data (flushes + sends direct if overflow) |
StreamWrite8(val) |
0x0010f340 | Buffer 1 byte |
StreamWrite16(val) |
0x0010f300 | Buffer 2 bytes (big-endian) |
StreamWrite32(val) |
0x0010f2b0 | Buffer 4 bytes (big-endian) |
StreamWriteStart() |
0x0010f370 | Begin write batch (pthread_mutex_lock only) |
StreamWriteFlush() |
0x0010f380 | Flush buffer via socket write + pthread_mutex_unlock |
StreamWriteSkip(n) |
0x0010f3f0 | Write n zero bytes (allocates temp buffer, wasteful) |
StreamRecv() |
0x0010f4b0 | Single raw recv() call (no retry loop) |
Connect() |
0x0010f080 | Delegates to TcpSocket::Connect() via vtable |
PsudoStreamSwap16() |
-- | Byte-swap u16 (host to network order) |
PsudoStreamSwap32() |
-- | Byte-swap u32 |
Write batching protocol: StreamWriteStart() locks the mutex →
multiple StreamWrite*() calls accumulate into buffer via memcpy →
StreamWriteFlush() sends all buffered data in one socket->write() call
and unlocks the mutex. This ensures protocol messages are sent atomically.
TcpSocket -- TCP Socket Layer
TcpSocket object layout (allocated size = 0x60 bytes):
Offset Size Type Field Notes
0x00 0x08 void** vtable VFT pointer
0x08 0x34 char[52] hostname Copied via strcpy (overflow risk)
0x3C 0x04 int port TCP port number
0x40 0x04 int record_flag Debug: record recv'd data to file
0x44 0x04 int playback_flag Debug: read from file instead of socket
0x48 0x08 FILE* record_file File handle for recording
0x50 0x08 FILE* playback_file File handle for playback
0x58 0x04 int socket_fd Connected socket FD
0x5C 0x04 int is_server 1=listen/accept mode, 0=connect mode
TcpSocket vtable (at 0x0022f750):
| Offset | Address | Method |
|---|---|---|
| +0x00 | 0x00117b70 | ~TcpSocket() D1 destructor |
| +0x08 | 0x00117b20 | ~TcpSocket() D0 destructor |
| +0x10 | 0x00117a80 | Connect() |
| +0x18 | 0x00117310 | read(buf, len) -- uses select(30s) + recv() |
| +0x20 | 0x00117530 | write(buf, len) -- uses select(30s) + send() |
Socket options set in CreateScok (0x001179c0, note typo):
SO_SNDBUF = 0x40000(256 KB)SO_RCVBUF = 0x40000(256 KB)enableNagles()is a no-op (returns 1) -- TCP_NODELAY is NOT set- No SO_KEEPALIVE set
- Connection via
getaddrinfo()+socket()+connect(), SOCK_STREAM
Timeout: Both read() and write() use select() with 30-second
timeout (tv_sec = 0x1e). On timeout or error, throws C++ ErrMsg
exception. Error codes: -1 = socket closed/failed, -2 = timeout.
SIGPIPE handler: BrokenPipe_handle prints message and returns
(prevents JVM crash). Registered via InitSignal() at 0x001170e0.
Connection teardown: No graceful disconnect message. TcpSocket::EndSock
(0x00117aa0) calls shutdown(fd, SHUT_RDWR) then close(fd).
File playback mode: TcpSocket has built-in debug support for reading
from/writing to files instead of sockets, controlled by record_flag and
playback_flag. Used for session replay/recording.
Protocol Handler Dispatch
RFBProtocol::ProtocolHandler() at 0x001183d0 dispatches server messages by
type byte using a relative jump table at 0x00127790 (61 entries, 0x00-0x3C).
Each jump table entry is a 32-bit signed offset relative to the table base address 0x00127790. The default handler (54 entries) is at 0x001183f5 and simply returns the type byte to the caller. All non-default handler code is inline within the ProtocolHandler function body (not separate functions).
ProtocolHandler operating modes (controlled by this->0x30):
- Mode 0: Normal operation. Clears screen update state, reads type byte, dispatches via jump table.
- Mode 1: Replay/capture mode. Copies
this->0x38(source file pointer) to the screen decoder context, sets capture flag, reads type byte, dispatches. - Other: Reads type byte directly (no state setup), dispatches.
Complete jump table (7 non-default handlers, all inline at 0x001184xx):
| Type | Hex | Jump Target | Handler Called | Wire Bytes After Type |
|---|---|---|---|---|
| 0 | 0x00 | 0x0011849a | RFBScreen::ScreenDecode (0x0010E7E8) | 3+2+2+2+2+4+4+4+data |
| 4 | 0x04 | 0x001184de | RFBScreen::ScreenCursorPosProc (0x0010E7F8) | 4+4+4+4+4 [+4+pixels] |
| 22 | 0x16 | 0x00118488 | NtwStream::StreamRead8 (inline, reads 1 byte) | 1 byte |
| 53 | 0x35 | 0x0011844d | RFBKeyboard::ProcKeyboardInfo (0x0011A720) then | 2+3 = 5 bytes |
| RFBMouse::ProcMouseInfo (0x001198C0) (fallthrough) | ||||
| 55 | 0x37 | 0x00118457 | RFBMouse::ProcMouseInfo (0x001198C0) | 3 bytes |
| 57 | 0x39 | 0x00118479 | RFBPrivilege::ProcPrivilegeInfo (0x0011A120) | 4+4+256 = 264 bytes |
| 60 | 0x3C | 0x0011846a | RFBScreen::GetScreenUILang (0x00118CB0) | 4+4 = 8 bytes |
Type 0x35 and 0x37 relationship: Type 0x35 calls ProcKeyboardInfo (reads 2 bytes: mode, type) then falls through to the same code that type 0x37 jumps to (ProcMouseInfo, reads 3 bytes: mode, type, extra). So type 0x35 is a combined keyboard+mouse info message (5 bytes after type), while type 0x37 is mouse-only (3 bytes after type).
Type 0x00 capture mode: When the this->0x20 & 1 flag is set (capture
mode), the type 0x00 handler also writes the received data to a capture file
using fwrite().
Encryption (RFBKMCryto)
AES-128-CBC for mouse event encryption (keyboard is always cleartext):
| Method | Address | Purpose |
|---|---|---|
RFBKMCryto() |
0x001148f0/0x00114920 | Constructor (C2/C1 variants) |
~RFBKMCryto() |
0x00114950/0x00114970/0x00114990 | Destructors (D2/D1/D0) |
aes_gen_tables() |
0x001149d0 | Generate AES S-boxes and tables |
aes_set_key() |
0x001149e0 | Set AES key from key material |
aes_encrypt() |
0x0010eaf8 (DEFAULT) | AES block encrypt |
aes_decrypt() |
0x00115d50 | AES block decrypt |
SW_AES_ECB() |
0x00116be0 | Software AES-ECB: mode=0 encrypt, mode=1 decrypt |
SW_AES_CBC() |
0x0010e2e8 (DEFAULT) | Software AES-CBC (used for mouse events) |
RFB_AES128_EventCryto() |
0x00116e30 | High-level event encrypt: hardcoded key+IV |
SetCipherKey() |
0x001149b0 | No-op stub: returns 1 without doing anything |
EnCryto() |
0x00117050 | Encrypt interface: calls RFB_AES128_EventCryto |
DeCryto() |
0x001149c0 | Decrypt interface |
RSb |
0x001225e0 | AES inverse S-box table |
The implementation is a complete software AES with key expansion, S-box generation, and CBC mode -- no dependency on OpenSSL or system crypto.
Key insight: SetCipherKey() is a no-op (returns 1 immediately). The AES
key used for mouse event encryption is entirely hardcoded in
RFB_AES128_EventCryto() -- it is the NIST AES-128 test vector:
Key: 2B 7E 15 16 28 AE D2 A6 AB F7 15 88 09 CF 4F 3C
IV: 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
The function also contains hardcoded expanded round key constants on the stack. The server-provided 256-byte key material from PrivilegeInfo is NOT used for native-layer encryption -- it is forwarded to Java only.
SW_AES_ECB (at 0x00116be0): Parameters: (this, mode, keysize, input, num_blocks, key, output). Mode 0 = encrypt, mode 1 = decrypt. Key size is
keysize * 64 + 128 bits. Processes blocks sequentially (no chaining).
SW_AES_CBC: Wraps SW_AES_ECB with CBC chaining (XOR with IV/previous ciphertext for encryption; decrypt then XOR for decryption).
libSharedLibrary64.so -- Virtual Media Library
JNI Interface
VirtualStorage (16 methods at tw.com.aten.vstorage.VirtualStorage):
| Export | Address | Purpose |
|---|---|---|
JASWInit |
0x00114c18 | Initialize VM subsystem (platformVer, companyId, boardId) |
JASetUNamePwdIPPort |
0x00114e74 | Set credentials and target BMC |
VUSPlugIn |
0x00114aac | Mount virtual device |
VUSPlugOut |
0x00114b78 | Unmount virtual device |
StartDev1Thread |
0x00114876 | Start device 1 protocol thread |
StartDev2Thread |
0x00114892 | Start device 2 protocol thread |
StartDev3Thread |
0x001148ae | Start device 3 protocol thread |
StartCheckDevExistThread |
0x0011485e | Device presence check thread |
StartShowMsgThread |
0x001148ca | GUI message display thread |
StartChangeGUIObjectThread |
0x0011496c | GUI state update thread |
RefreshDevCBContents |
0x00114a48 | Enumerate available devices |
ChangeCBSelect |
0x00114bb6 | Change device selection |
SetEachVMLanPort |
0x00114bda | Set per-device LAN port |
JThterminate |
0x00114c00 | Terminate all threads |
ResetLANDisconnect |
0x00114f82 | Reset after network disconnect |
Also exports the global struct:
java_vsuiat 0x0023f560 (0x2A0 bytes) -- shared Java↔native state
Architecture
libSharedLibrary64.so is a standalone C library (no C++ class hierarchy like libiKVM64.so). It uses function pointer tables for platform abstraction and global structures for state management.
Key Function Groups
Protocol Engine:
| Function | Address | Purpose |
|---|---|---|
VMSM_USB_BulkOnly |
0x0011b922 | Main USB Mass Storage state machine |
State_Fn_RX_PDU_TAG |
0x0011b24c | PDU header parser, type/CBW dispatch |
State_Fn_EXECUTE_CBW |
0x0011ab24 | SCSI command execution state |
State_Fn_RX_CBW |
0x0011ac68 | CBW parsing state |
State_Fn_RX_BULK_OUT_DATA |
0x00117cc8 | Bulk OUT (write) data receive state |
State_Fn_RX_MOUNT_DEV_STATUS |
0x0011b694 | Mount status response handler |
SetState_ReadPDU |
0x00117b92 | Reset state to 0x01, expect 8-byte header |
Execute_VM_CMD |
0x00119c06 | SCSI command dispatch |
FillUSBPlugInPkt |
0x001167c0 | Build plug-in packet with USB descriptors |
FillSetEPCMDPkt |
0x00116234 | Build endpoint configuration |
FillCSW |
0x00119fe8 | Build Command Status Wrapper |
FillDataPDUTag |
0x00119d10 | Build data PDU header |
FillKeepAlivePkt |
0x0011b1c2 | Build keep-alive response |
FillErrorData |
0x00119f5a | Build error response (CSW status from ASC/ASCQ) |
FillDevID |
0x001161d4 | Build device identification |
FillHostDescriptorData |
0x0011cd4a | Build host descriptor data |
Core_VM_SendUnMountPkt |
0x0011bdf2 | Send unmount |
RefreshBtnMode_Fill |
0x0011c4d2 | GUI button state |
Mount Methods:
| Function | Address | Purpose |
|---|---|---|
Core_Mount_VM |
0x0001f8ea | Main mount orchestration |
MtMethod_Media |
0x0011df42 | Direct TCP mount (primary) |
MtMethod_WebISO |
0x0011e946 | HTTP ISO upload mount |
MtMethod_UploadIMA |
0x0011e14a | HTTP floppy upload mount |
UnMtMethod_WebISO |
0x0011db14 | HTTP ISO unmount |
DetermineTCPType |
0x0001f76a | Choose IPv4 vs IPv6 |
CheckVMInfoBetSWAndFW |
0x0001cc7e | Validate VM config with firmware |
Encryption:
| Function | Address | Purpose |
|---|---|---|
Encrypt_RC4 |
0x0011e0a6 | RC4 encrypt wrapper |
RC4_init |
0x00115c10 | RC4 Key Scheduling Algorithm |
RC4_byte |
0x00115cf2 | RC4 single byte |
RC4_encrypt |
0x00115d88 | RC4 Pseudo-Random Generation Algorithm |
ISO/CD-ROM SCSI Handlers:
| Function | Address | SCSI Command |
|---|---|---|
IsoCommand |
0x0012a9da | Main dispatch (switch on opcode) |
Inquiry |
0x00128478 | INQUIRY (0x12) |
Startstopunit |
0x00128aa0 | START STOP UNIT (0x1B) -- eject/load emulation |
MediumRemoval |
-- | PREVENT ALLOW MEDIUM REMOVAL (0x1E) |
ReadCapacity |
0x0012a082 | READ CAPACITY (0x25) |
Read10 |
0x00129f42 | READ(10) (0x28) |
Read12 |
0x0012a868 | READ(12) (0xA8) -- stub, returns 30 zero bytes |
ReadBuffer |
0x0012a7aa | READ BUFFER (0x3C) -- always returns error |
ReadToc |
0x00129ba4 | READ TOC (0x43) |
ReadCD |
-- | READ CD (0xBE) |
ModeSense |
0x00128f56 | MODE SENSE(10) (0x5A) |
GetConfig |
0x00127378 | GET CONFIGURATION (0x46) |
GetEventStatus |
0x00129962 | GET EVENT STATUS NOTIFICATION (0x4A) |
ReadFormatCapacities |
0x0012a1d4 | READ FORMAT CAPACITIES (0x23) |
ReadDiscInformation |
0x0002a378 | READ DISC INFORMATION (0x51) |
ReadBufferCapacity |
0x0002a708 | READ BUFFER CAPACITY (0x5C) |
ReadMediaSerialNumber |
0x0002a7f4 | READ MEDIA SERIAL NUMBER (0xAB) |
SetReadAhead |
0x00128378 | SET READ AHEAD (0xA7) -- validates LBA ordering |
ReportKey |
0x00029e16 | REPORT KEY (0xA4) |
SetStream |
0x00028edc | SET STREAMING (0xB6) |
SetCDSpeed |
0x00128e80 | SET CD SPEED (0xBB) -- returns 32 zero bytes |
MachanismStatus |
0x001288be | MECHANISM STATUS (0xBD) -- returns 8-byte status |
Test |
0x0002a8dc | TEST (0x01) |
TestUnitReady |
0x00128f10 | TEST UNIT READY (0x00) |
RequestSense |
0x001282d6 | REQUEST SENSE (0x03) |
Zero |
0x00029b72 | Zero-fill utility |
ParOK |
0x000271fe | Parameter validation |
EventDes |
0x00029b42 | Event descriptor builder |
Floppy/IMA Handlers:
| Function | Address | Purpose |
|---|---|---|
FileStorCMD_IMA_API |
0x00018380 | IMA file SCSI command API |
FileStorDevCMDExecuteIMA_Default |
0x00017fea | Default IMA command execution |
FileStorDevCMDExecuteISO_Default |
0x0001879a | Default ISO command execution |
FileStorDevCMDExecuteISO_FixInquiry |
0x00018d6a | ISO INQUIRY fixup |
FileStorCMD_ISO_API |
0x00019b52 | ISO file SCSI command API |
CalculateISOFileBlockNum |
0x00115e20 | Calculate ISO sector count |
Physical Device Handlers (pass-through to real hardware):
| Function | Address | Purpose |
|---|---|---|
Linux_PhyStorDevCMDExecute_Default |
0x00122902 | Default physical device passthrough |
Linux_PhyStorDevCMDExecute_IDECDROMFixInquiry |
0x00123218 | IDE CD-ROM INQUIRY fixup |
Linux_PhyStorDevCMDExecute_USBCDROMFixInquiry |
0x00122fd2 | USB CD-ROM INQUIRY fixup |
Linux_PhyStorDevCMDExecute_SimToFloppy |
0x001151ac | Simulate floppy from physical device |
Linux_PhyStorDevCMDExecute_SimToFlash |
0x00115746 | Simulate flash from physical |
Linux_PhyStorDevCMDExecute_SASHDFixInquiry |
0x00123650 | SAS/HDD INQUIRY fixup |
Linux_PhyStorDevCMDExecute_HD_SimToRemovableDisk |
0x00124a4e | HDD as removable disk |
Linux_PhyStorDevCMDExecute_HD_SimToRemovableDisk_logical |
0x00124edc | Logical volume |
Linux_PhyStorDevCMDExecute_SASHD_SimToRemovableDisk_logical |
0x00125124 | SAS logical volume |
Linux_PhyStorDevCMDExecute_IDECDROM_ReadFile |
0x00123a80 | IDE CD-ROM file read |
Physical Device Scanning:
| Function | Address | Purpose |
|---|---|---|
Linux_ScanPhyStor_IDE_Floppy |
0x00124570 | Scan IDE floppy devices |
Linux_PhyStorDevCheckDevType_IDEFloppy |
0x001221e4 | Check IDE floppy type |
Linux_PhyStorDevCheckDevType_USBFloppy |
0x00122114 | Check USB floppy type |
Linux_FileStorDevOpenISO |
0x0012485c | Open ISO file |
Linux_FileStorDevCheckISOFileName |
0x00124778 | Validate ISO filename |
Virtual Folder Mount (FAT filesystem emulation):
| Function | Address | Purpose |
|---|---|---|
TFATFileSystem_Open |
0x0002b4b4 | Open FAT filesystem |
TFATFileSystem_Format |
0x0002be82 | Format virtual FAT volume |
TFATFileSystem_DiskIO |
0x0002b760 | Disk I/O operations |
VirImgFATFileSysIO |
0x0002b85a | Virtual image FAT I/O |
TFATFileSystem_FAT* |
various | FAT table manipulation |
TFATFileSystem_Dir* |
various | Directory operations |
TFATFileSystemImage_* |
various | Image file operations |
Folder_RemoveImage |
0x0002af6a | Remove folder image |
Linux_TFATFileSystemImage_GetAllFiles |
0x00030a30 | Enumerate host files |
Linux_TFATFileSystem_DirGetFromWin32 |
0x0002fe24 | Convert host dir to FAT |
ConvertToDOSFormat |
0x00031cc2 | Convert filenames to 8.3 |
The folder mount feature creates a virtual FAT16 filesystem image from a host directory, emulating a removable USB disk.
Network Layer:
| Function | Address | Purpose |
|---|---|---|
Linux_VMTCPConnect_IPv4_Default |
0x00121b02 | IPv4 TCP connect |
Linux_VMTCPConnect_IPv6_Default |
0x00121d4c | IPv6 TCP connect |
Linux_VMTCPClose_IPv4_Default |
-- | IPv4 close |
Linux_VMTCPTx_IPv4_Default |
-- | IPv4 send |
Linux_VMTCPRx_IPv4_Default |
-- | IPv4 receive |
| (IPv6 variants) | -- | IPv6 equivalents |
Error Handling:
| Function | Address | Purpose |
|---|---|---|
ErrorHandle_Init |
0x0001cb2e | Initialize error handler |
ErrorHandle_Set |
0x0001cb5a | Set error state |
ErrorHandle_Get |
0x0001cb88 | Get error state |
ErrorHandle_Handle |
0x00015f4c | Process error |
ErrorHandle_DevSync |
0x000160ea | Device sync on error |
Utility Functions:
| Function | Address | Purpose |
|---|---|---|
ByteToDWord |
0x00015dec | Byte array to 32-bit word |
CMDSetTranslate |
0x0001d090 | Command set translation |
DetectFileStorDevCMDType |
0x0001d16c | Detect file-backed device command set |
DetectHostCMDType |
0x0001d1fa | Detect host device command set |
DetectPhyStorDevCMDType |
0x0001d0aa | Detect physical device command set |
GetDevCMDSet |
0x0001d208 | Get device command set |
FunIDToDevInfo |
0x0001448c | Map function ID to device info |
FunIDToHostDevType |
0x0001450e | Map function ID to host device type |
FunIDToOEMStr |
0x00014700 | Map function ID to OEM string |
GetRandomTimeStamp |
0x00017860 | Generate random timestamp (SID + HTTP boundaries) |
InitVMSW |
0x001178c8 | Clear st_VMMainInfo, init queues, generate SID |
VMInfoCalloc |
0x00120bb2 | Allocate per-device structures: calloc(devCount+1, 0x15920) |
UI_PrepareVMResource |
0x0011667c | Init config tables, alloc per-device memory, set media types |
StoreVMInfoFromGUI |
0x0011d28c | Validate + copy credentials (username<17, password<21) |
UI_Mount_VM |
0x0011f958 | Full mount orchestration (check busy, store info, connect) |
SetupVMInfoBetSWAndFW |
0x0011ced4 | Pre-mount setup: descriptors, HTTP port, persistent TCP |
UI_PreVerifyVMInfo |
0x0011ef0a | Credential pre-verification (bit 6 verify-only path) |
Core_GetDevStatusFromFW |
0x0011741e | Device status query via temp TCP (type=0x08) |
GetHttpPortFromFW |
0x0011cdd4 | HTTP port query via temp TCP (type=0x0A, leaks socket) |
UI_SetUSBPlugInPktEncrypt |
-- | Set RC4 enable flag (DAT_0023f803) |
UI_SetUSBPlugInPktSIDAuth |
-- | Set SID auth flag (DAT_0023f804) |
VM_Thread |
0x0011baa4 | Main per-device protocol loop (blocking I/O) |
VM_Thread_VarInit |
0x0011b9cc | Initialize thread state vars |
MtVM_Engine |
0x0011de84 | Mount dispatch engine (by device type) |
UnMtVM_Engine_Normal |
0x0011d48c | Unmount dispatch engine |
UnMountStatusInit |
0x0011b5d0 | BMC-initiated unmount handler |
VM_TCPSocket_Terminate |
0x00115f0c | Close socket and reset flags |
UI_GetDevStatusFromFW |
0x001170fe | Multi-slot device status query |
GUIAtbQueueAPI |
0x000202b0 | GUI attribute queue management |
UI_GetVMLibVersion |
-- | Return library version string |
Imports (Notable)
Beyond standard libc/libstdc++, libSharedLibrary64.so imports:
glob64-- for scanning device pathsioctl-- for physical device controlpopen/pclose-- for shell command executionsystem-- for system command execution (security concern)readdir64/opendir/closedir-- directory enumerationopen64/lseek64/fseeko64/fopen64-- large file supportinet_pton/inet_ntoa/htons-- network address conversionwcstombs-- wide character conversion (for internationalization)
The use of system() and popen() is notable for security analysis.
Java Application Layer
Entry Point
KVMMain.main() parses command-line arguments:
args[0]: IP address
args[1]: Username
args[2]: Password
args[3]: KVM port
args[4]: Hostname
args[5]: VM port (default 623)
args[6]: Company ID
args[7]: Board ID
args[8]: Language
Static Initialization
On load:
System.loadLibrary("iKVM64")-- KVM native librarySystem.loadLibrary("SharedLibrary64")-- Virtual media native library
Key Java Classes
| Class | Package | Role |
|---|---|---|
KVMMain |
tw.com.aten.ikvm |
Entry point, argument parsing |
KVMParam |
tw.com.aten.ikvm |
Configuration parameters |
BoardInfo |
tw.com.aten.ikvm |
Board identification / feature flags |
Viewer |
tw.com.aten.ikvm.ui |
Main window, thread management |
RemoteVideo |
tw.com.aten.ikvm.ui |
Video display, input handling, JNI bridge |
KeyMap |
tw.com.aten.ikvm.util |
Java VK → USB HID keycode translation |
RMConnection |
tw.com.aten.ikvm.jni |
Connection JNI wrapper |
RMDesktop |
tw.com.aten.ikvm.jni |
Desktop composition JNI wrapper |
VirtualStorage |
tw.com.aten.vstorage |
Virtual media GUI + JNI bridge |
ConnInfo |
tw.com.aten.bean |
Connection info data bean |
UserInfo |
tw.com.aten.bean |
User credentials data bean |
PlatformInfo |
tw.com.aten.bean |
Platform identification |
VirtualUsbInfo |
tw.com.aten.bean |
Virtual USB configuration |
InfoRepository |
tw.com.aten.bean |
Shared state repository |
Threading Model
RemoteVideo manages three threads:
- CatchThread --
catchLoop()→ nativeProtocolHandler()loop - DecodeThread --
runImage()→ receive + decode video frames - LazyWorker -- Periodic: refresh, updateImage, FPS calc
VirtualStorage manages per-device threads:
- Dev1Thread / Dev2Thread / Dev3Thread -- Protocol loops for each VM device
- EventThread -- Event processing
- ShowMsgThread -- GUI message display
- C_GUIObjectThreadThread -- GUI state updates
- C_DevExistThread -- Device presence monitoring
- GetDevHealthStatusThread -- Health status polling
Third-Party Code
ch.randelshofer.media.avi.AVIOutputStream-- AVI file writer for screen recording feature. Licensed separately (seelicense.html).
Class Hierarchy and Vtables
libiKVM64.so Vtable Addresses
All vtables are in .data section, accessible via _ZTV symbols:
| Class | Vtable Address | RTTI Address | Parent |
|---|---|---|---|
TcpSocket |
0x0022f740 | 0x0022f780 | -- |
StreamSocket |
0x0022f7c0 | 0x0022f7a0 | TcpSocket |
NtwStream |
(inline) | -- | StreamSocket |
RMProtocol |
0x00231020 | 0x00231080 | -- |
RFBProtocol |
0x0022f840 | 0x0022f8a0 | RMProtocol |
RMScreen |
0x0022dd00 | 0x0022dd70 | -- |
RFBScreen |
0x0022f8c0 | 0x0022f940 | RMScreen |
RMKeyboard |
0x0022f960 | 0x0022f9c0 | -- |
RFBKeyboard |
0x00230280 | 0x002302f0 | RMKeyboard |
RMMouse |
0x0022dd80 | 0x0022dde0 | -- |
RFBMouse |
0x0022f9e0 | 0x0022fa40 | RMMouse |
RMPrivilege |
0x00230f00 | 0x00230f40 | -- |
RFBPrivilege |
0x0022fcc0 | 0x0022fd00 | RMPrivilege |
RMCryto |
0x00230f60 | 0x00230fa0 | -- |
RFBKMCryto |
0x0022f6a0 | 0x0022f6e0 | RMCryto |
RMDecoder |
0x0022df80 | 0x0022dfb0 | -- |
ASTVideoDecoder |
0x0022de00 | 0x0022de30 | RMDecoder |
HermonVideoDecoder |
0x0022de60 | 0x0022de90 | RMDecoder |
YarkonVideoDecoder |
0x0022dec0 | 0x0022def0 | RMDecoder |
Pilot3VideoDecoder |
0x0022df20 | 0x0022df50 | RMDecoder |
ASTDecode |
0x00231100 | 0x00230ef0 | -- |
ast2100 |
0x00230ea0 | 0x00230ed0 | ASTDecode |
ast_jpeg |
0x00230fc0 | 0x00230ff0 | ASTDecode |
RMAuth |
0x0022fa60 | 0x0022fa80 | -- |
RMConnection |
0x0022dfc0 | 0x0022e010 | -- |
RMStorage |
0x0022f800 | 0x0022f830 | -- |
RMMisc |
0x0022f700 | 0x0022f720 | -- |
ErrMsg |
-- | 0x0022df70 | (exception type, 16 references) |
RFBProtocol Object Layout
The RFBProtocol object (allocated size = 0x70 bytes) holds the protocol state.
Constructors:
RMProtocol::RMProtocolat 0x0011f9a0: Sets vtable to 0x00231030, sets mode_flags (0x20) = 0, operating_mode (0x30) = 0.RFBProtocol::RFBProtocolat 0x00117cc0 (C1) / 0x00117c90 (C2): CallsRMProtocol::RMProtocol(), then sets rfb_major (0x40) = 3, rfb_minor (0x44) = 8, and vtable pointer to 0x0022f850.
The sub-object pointers (0x50-0x68) are NOT set by the RFBProtocol
constructor. Instead, each sub-object's constructor stores its this
pointer back into the RFBProtocol at the appropriate offset:
- RFBMouse ctor stores
thisatprotocol + 0x50 - RFBKeyboard ctor stores
thisatprotocol + 0x58 - RFBScreen ctor stores
thisatprotocol + 0x60 - RFBPrivilege ctor stores
thisatprotocol + 0x68
The NtwStream pointer at offset 0x08 is set by InitHandShake, which
allocates a 0x628-byte NtwStream and stores it at this + 0x08.
Offset Size Type Field Notes
0x00 8 vtable* vtable ptr Points to 0x0022f850
0x08 8 NtwStream* network stream Buffered I/O layer (0x628 bytes)
0x10-0x1F (inherited) RMProtocol fields (not explicitly used by RFB)
0x20 4 uint mode_flags Bit 0 = capture mode
0x24 4 (padding)
0x28 8 FILE* capture_file For replay/capture mode
0x30 4 int operating_mode 0=normal, 1=replay, other=direct
0x34 4 (padding)
0x38 8 void* source_data File ptr or data ptr for replay
0x40 4 int rfb_major Always 3 (set by constructor)
0x44 4 int rfb_minor Always 8 (set by constructor)
0x48 1 uchar fw_protocol_flag Set by SetFWProtocol(); controls SetScreenUILang extra u32
0x49-0x4F (padding)
0x50 8 RFBMouse* mouse_handler Set by RFBMouse constructor (0x28 bytes)
0x58 8 RFBKeyboard* keyboard_handler Set by RFBKeyboard constructor (0xB8 bytes)
0x60 8 RFBScreen* screen_handler Set by RFBScreen constructor (0x2080 bytes)
0x68 8 RFBPrivilege* privilege_handler Set by RFBPrivilege constructor (0x120 bytes)
RFBProtocol Vtable (at 0x0022f850)
Index Offset Address Method
[0] 0x00 0x00117d60 ~RFBProtocol() (destructor, non-deleting)
[1] 0x08 0x00117dd0 ~RFBProtocol() (destructor, deleting)
[2] 0x10 0x00117fa0 InitHandShake(RMConnInfo_t)
[3] 0x18 0x00118190 Authenticate(Userinfo_t)
[4] 0x20 0x001183d0 ProtocolHandler()
[5] 0x28 0x00118230 ProcAlive(bool)
[6] 0x30 0x00118320 ProcSetScreenUI(int, int)
[7] 0x38 0x001183a0 ProcGetScreenUI()
RFBScreen Vtable (at 0x0022f8d0)
Verified from raw vtable bytes at 0x0022f8d0 (13 entries x 8 bytes).
Index Offset Address Method
[0] 0x00 0x00118710 ~RFBScreen() (non-deleting, D1)
[1] 0x08 0x001187c0 ~RFBScreen() (deleting, D0)
[2] 0x10 0x00118900 SetPowerOnOff(uchar) [sends type 0x1A]
[3] 0x18 0x00118850 ScreenUpdate(ScreenReqInfo_t) [sends type 0x03]
[4] 0x20 0x00118960 ScreenCalibration() [sends type 0x17]
[5] 0x28 0x001189b0 ScreenSetPosition(ScreenPos_t) [sends type 0x15]
[6] 0x30 0x00118b80 ScreenSetInfo(ScreenInfo_t) [sends type 0x32]
[7] 0x38 0x00118c10 ScreenGetInfo() [copies screen info struct, no network I/O]
[8] 0x40 0x00118d50 ScreenSetFrameBuff(uchar*, uchar*)
[9] 0x48 0x00118d40 ScreenGetFreame() [returns framebuffer ptr at this+0x38]
[10] 0x50 0x00118da0 GeFrontGround()
[11] 0x58 0x00118cb0 GetScreenUILang() [handles server type 0x3C]
[12] 0x60 0x00118d60 GetCursorPos() [sends type 0x19] / MixedCursor
Correction from previous analysis: Index [2] is SetPowerOnOff (0x00118900), NOT ScreenUpdate. Index [3] is ScreenUpdate (0x00118850), NOT ScreenGetRect. Index [7] is ScreenGetInfo (0x00118c10), NOT SetPowerOnOff. The power control JNI functions call vtable offset 0x10 (index [2]) which is indeed SetPowerOnOff.
RFBMouse Vtable (at 0x0022f9f0)
Index Offset Address Method
[0] 0x00 0x001194b0 ~RFBMouse() (non-deleting)
[1] 0x08 0x001194d0 ~RFBMouse() (deleting)
[2] 0x10 0x00119660 MouseAction(MouseInfo_t) [main entry: handles wheel repeat]
[3] 0x18 0x00119730 MouseSync() [sends type 0x07, value 0x0780]
[4] 0x20 0x00119780 MouseReset() [sends type 0x08 + mouse_type]
[5] 0x28 0x001197d0 MouseUpdateInfo() [sends type 0x37, clears config flag]
[6] 0x30 0x00119810 MouseGetPT() [returns 16 bytes from this+0x10]
[7] 0x38 0x00119820 MouseSetPT(PTerInfo_t) [sends type 0x36 + mode + type]
[8] 0x40 0x001198c0 ProcMouseInfo() [called by types 0x35, 0x37; reads 3 bytes]
[9] 0x48 0x00119910 MouseHotPlug() [sends type 0x3A, 1 byte only]
Note: Previous vtable mapping had SendMouse at index [2] and MouseAction at [3]. Corrected: index [2] is MouseAction (the JNI entry point which calls SendMouse internally), and the remaining methods follow the mangled symbol order. SendMouse at 0x001194f0 is called internally by MouseAction, not through vtable.
RFBMouse Object Layout (0x28 bytes)
Offset Size Type Field Notes
0x00 8 vtable* vtable ptr Points to 0x0022f9f0
0x08 8 RMProtocol* protocol Network stream access
0x10 1 byte config_received Set to 1 by ProcMouseInfo; cleared by MouseUpdateInfo
0x14 4 int mouse_mode From ProcMouseInfo byte 2; MouseReset sends it
0x18 4 int encryption_enabled From ProcMouseInfo byte 1; SendMouse checks != 0
0x1c 4 int additional_config From ProcMouseInfo byte 3
0x20 8 RFBKMCryto* crypto AES encryption instance (0x2110 bytes)
Constructor (RFBMouse::RFBMouse at 0x001193F0): Allocates RFBKMCryto (0x2110
bytes) at offset 0x20. Initializes mouse_mode=1 (offset 0x14) and
encryption_enabled=0 (offset 0x18 = unencrypted by default).
RFBKeyboard Vtable (at 0x00230290)
Index Offset Address Method
[0] 0x00 0x0011a800 ~RFBKeyboard() (non-deleting)
[1] 0x08 0x0011a760 ~RFBKeyboard() (deleting)
[2] 0x10 0x0011a5d0 KeyboardAction(KeyInfo_t) [calls processVK via vtable[9]]
[3] 0x18 0x0011a5f0 KeyboardSync() [stub, returns 1]
[4] 0x20 0x0011a600 KeyboardReset()
[5] 0x28 0x0011a610 KeyboardUpdateInfo() [sends type 0x35]
[6] 0x30 0x0011a650 KeyboardGetType()
[7] 0x38 0x0011a660 KeyboardSetType(KeyTypeInfo_t)
[8] 0x40 0x0011a720 ProcKeyboardInfo() [called by type 0x35]
[9] 0x48 0x0011a670 Sendkey(int keysym, int down_flag) [always cleartext]
[10] 0x50 0x0011a920 processVK(KeyInfo_t) [VK -> X11 keysym via 3 maps]
Note: The KeyboardAction at vtable[2] calls vtable[9] (offset 0x48) which is
processVK, not Sendkey. The decompiled code shows
(**(code **)(*param_1 + 0x48))(param_1, ...) where 0x48 is vtable offset for
processVK. After processVK returns the keysym, the caller invokes Sendkey
(vtable[9] at offset 0x48) to transmit the event.
RFBPrivilege Vtable (at 0x0022fcd0)
Index Offset Address Method
[0] 0x00 0x0011a0e0 ~RFBPrivilege() (non-deleting)
[1] 0x08 0x0011a100 ~RFBPrivilege() (deleting)
[2] 0x10 0x0011a120 ProcPrivilegeInfo() [called by type 0x39; reads 8+256 bytes]
[3] 0x18 0x0011a1b0 ExePrivilegeCtrl() [calls privilegeControl JNI helper]
[4] 0x20 0x0011a1f0 SendKickRequest(int, uchar*) [sends type 0x38, 73 bytes]
[5] 0x28 0x0011a1d0 ViewerConfig(int, uchar*) [stores session_id + config]
RFBPrivilege Object Layout (0x120 bytes)
Offset Size Type Field Notes
0x00 8 vtable* vtable ptr Points to 0x0022fcd0
0x08 8 RMProtocol* protocol Network stream access
0x10 4 int session_word_lo From ProcPrivilegeInfo
0x14 4 int session_word_hi From ProcPrivilegeInfo
0x18 256 u8[256] aes_key_material 256 bytes read in ProcPrivilegeInfo
0x118 8 RFBKMCryto* crypto AES instance (0x2110 bytes, allocated in ctor)
Constructor (RFBPrivilege::RFBPrivilege at 0x00119fa0/0x0011a030):
- Calls
RMPrivilege::RMPrivilege(base)parent constructor - Stores
thispointer at protocol+0x68 (so RFBProtocol can find it) - Allocates
RFBKMCryto(0x2110 bytes) and stores at this+0x118 - The
SetCipherKeymethod on this crypto instance is a no-op (returns 1)
RMConnection Vtable (at 0x0022dfd0)
Index Offset Address Method
[0] 0x00 0x00112870 ~RMConnection() (non-deleting)
[1] 0x08 0x001128b0 ~RMConnection() (deleting)
[2] 0x10 0x001128f0 ConnKeepActive()
[3] 0x18 0x001129e0 CheckVaildUser(Userinfo_t)
[4] 0x20 0x00112b40 QosControl()
[5] 0x28 0x00112b70 ConnEvenetHandle() [stub: returns 0]
RMDesktop Layout (Composition)
Offset Size Member
0x00 8 RFBKeyboard*
0x08 8 RFBMouse*
0x10 8 RFBScreen*
0x18 8 RFBPrivilege*
0x20 8 RFBProtocol* (parent)
...
0x30 -- (total allocated size = 0x30)
Security Analysis
Hardcoded Cryptographic Keys
AES-128-CBC key (libiKVM64.so, RFBKMCryto):
Key: 2B 7E 15 16 28 AE D2 A6 AB F7 15 88 09 CF 4F 3C
IV: 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
This is the NIST AES test vector key. Used for mouse event encryption only (keyboard events are always cleartext). The key is hardcoded in the binary -- any client or MITM can decrypt.
RC4 key (libSharedLibrary64.so, plug-in packet encryption):
ASCII: BX80570E3110Q814A447
Hex: 42 58 38 30 35 37 30 45 33 31 31 30 51 38 31 34 41 34 34 37
Hardcoded 20-byte key at 0x00136202. Only encrypts the initial plug-in packet (credentials + USB descriptors).
RC4 key (libiKVM64.so, AST2050 video stream decryption):
ASCII: fedcba9876543210
Hex: 66 65 64 63 62 61 39 38 37 36 35 34 33 32 31 30
Hardcoded 16-byte key at 0x00230500. Used to optionally decrypt the AST2050
(encoding 0x57) video tile bitstream. Key is cyclically expanded to 256 bytes
by Keys_Expansion() before RC4 KSA. Any client or MITM can decrypt.
Credential Handling
- KVM authentication: Username (24 bytes) and password (24 bytes) are sent
in plaintext over the TCP connection during the
Authenticatephase. The server sends a 24-byte "challenge" which the client reads but discards -- there is no challenge-response mechanism. ThecheckValidUserJNI function at 0x00119C60 copies credentials from JavaUserInfofields usingstrcpyinto fixed-size stack buffers (24 bytes for username, 96 bytes for password) -- potential buffer overflow if Java side provides longer strings. - Username and password are sent in plaintext (or RC4-encrypted with known key) in the VM plug-in packet
- VM credentials are stored in global memory at fixed addresses:
- Username:
DAT_0023f848(max 16 chars) - Password:
DAT_0023f85c(max 20 chars)
- Username:
- KVM credentials are passed through JNI and stored in the
UserInfobean
Input Validation Concerns
sprintfused extensively (potential format string issues)strcpy/strncpyused for string handling (buffer overflow risk)system()andpopen()imported by libSharedLibrary64.so (command injection risk)- No bounds checking on SCSI CDB parsing in some handlers
- ISO file reading trusts values from ISO 9660 volume descriptor at offset 0x8050
Network Security
- KVM connection: Modified RFB 3.8 over plain TCP (no TLS)
- VM connection: Plain TCP with optional RC4 (known key)
- HTTP upload: Plain HTTP with basic cookie auth (
SID=) - No certificate validation, no HMAC, no replay protection
- Keep-alive is unauthenticated
Software AES Implementation
The AES implementation in RFBKMCryto is a standalone software implementation (not using system OpenSSL/libcrypto). This means:
- No hardware acceleration (AES-NI)
- Potential for timing side-channels
- Custom implementation may have subtle bugs
Key Data Structures and Globals
libiKVM64.so Global Variables
| Symbol | Address | Type | Purpose |
|---|---|---|---|
connection |
BSS 0x1d2ce0 | ptr | RMConnection instance |
desktop |
BSS 0x04823740 | ptr | RMDesktop instance |
frameBuffer |
BSS 0x048236e8 | ptr | Direct ByteBuffer pointer |
frameBufferLength |
BSS 0x048236e0 | int | Frame buffer size |
cursorBuffer |
BSS 0x048236d0 | ptr | Cursor ByteBuffer pointer |
cursorBufferLength |
BSS 0x048236c8 | int | Cursor buffer size |
frameObj |
BSS 0x048236f0 | jobject | Java frame buffer reference |
cursorObj |
BSS 0x048236d8 | jobject | Java cursor buffer reference |
remoteVideo |
BSS | jobject | Java RemoteVideo reference |
jvm |
BSS | JavaVM* | JVM pointer for callbacks |
changeResolutionMid |
BSS 0x04823738 | jmethodID | Resolution change callback |
errorHandlerMid |
BSS 0x04823730 | jmethodID | Error handler callback |
genEventMid |
BSS 0x04823728 | jmethodID | Key event callback |
privilegeCtrlMid |
BSS | jmethodID | Privilege control callback |
addClipBoundsMid |
BSS 0x04823710 | jmethodID | Dirty rect callback |
setViewerConfigMid |
BSS | jmethodID | Config callback |
setNormalStartMid |
BSS | jmethodID | Normal start callback |
getScreenUILangMid |
BSS 0x04823708 | jmethodID | OSD lang callback |
getQuickCursorMid |
BSS 0x04823700 | jmethodID | Quick cursor callback |
connInfoIpFid |
BSS 0x1d2cd8 | jfieldID | ConnInfo.ip field |
connInfoPortFid |
BSS 0x1d2cd0 | jfieldID | ConnInfo.port field |
g_config |
BSS 0x04823748 | int | Viewer config value |
g_session_id |
DATA 0x001310cc | int | Session ID |
buildtime |
DATA 0x001310d0 | char[] | Build timestamp |
capsLock_status |
BSS 0x04823694 | int | CapsLock state |
numLock_status |
BSS | int | NumLock state |
scrollLock_status |
BSS | int | ScrollLock state |
capsLock_LEDorig |
DATA 0x001310c4 | int | Original CapsLock LED |
catchKey |
DATA 0x001310c8 | int | Catch key flag |
filterFlag |
BSS 0x048236b0 | int | Input filter flag |
flag |
BSS 0x048236bc | int | General flag |
count |
BSS 0x048236c0 | int | Frame counter |
captureFile |
BSS 0x04823628 | ptr | Screen capture file |
captureName |
BSS 0x04823640 | ptr | Capture filename |
ast_Buffer |
BSS 0x1d35c0 | ptr | AST decoder buffer |
byTileYuv |
BSS 0x1d32c0 | byte[] | Tile YUV buffer |
Cr_tab |
BSS 0x1d2aa0 | int[] | Cr→R lookup table |
Cb_tab |
BSS 0x1d28a0 | int[] | Cb→B lookup table |
Cr_Cb_green_tab |
BSS 0x1b28a0 | int[] | Cr/Cb→G lookup table |
HTAC |
BSS 0x001311c0 | struct | AC Huffman table |
HTDC |
BSS 0x00171320 | struct | DC Huffman table |
HUFFMANCODE |
DATA 0x0012faa0 | byte[] | Huffman code constants |
AC_CHROMINANCE_HUFFMANCODE |
DATA 0x00130320 | byte[] | AC chrominance table |
AC_LUMINANCE_HUFFMANCODE |
DATA 0x001303e0 | byte[] | AC luminance table |
DC_CHROMINANCE_HUFFMANCODE |
DATA 0x00130480 | byte[] | DC chrominance table |
DC_LUMINANCE_HUFFMANCODE |
DATA 0x001304c0 | byte[] | DC luminance table |
G_funct |
BSS 0x1d32b0 | ptr | Global function pointer |
G_outBuf |
BSS 0x1d32a8 | ptr | Global output buffer |
G_QT_TableSelection |
BSS 0x1d32a4 | int | Quantization table select |
G_txb / G_tyb |
BSS 0x1d32bc/b8 | int | Tile position tracking |
libSharedLibrary64.so Key Globals
See MOUNT_PROTOCOL.md for the java_vsui structure (0x0023f560) and
per-device structure layout.
| Symbol/Address | Size | Purpose |
|---|---|---|
DAT_0023f560 (java_vsui) |
0x2A0 | Shared Java↔native configuration (see layout below) |
st_VMMainInfo (0x0023f800) |
0xD0 | VM main info struct (cleared by InitVMSW) |
DAT_0023f803 |
1 | Encryption enabled flag (always 1 after JASWInit) |
DAT_0023f804 |
1 | SID authentication flag (always 1 after JASWInit) |
DAT_0023f805-08 |
4 | SID bytes (4 bytes from rand() seeded with time()) |
DAT_0023f810 |
8 | Per-device structure base pointer (from VMInfoCalloc) |
DAT_0023f818 |
8 | File-backed storage function table pointer |
DAT_0023f820 |
8 | Socket function pointer table (indexed by conn type) |
DAT_0023f830 |
8 | Physical storage function table pointer |
DAT_0023f848 |
16 | Username string (max 16 chars, from StoreVMInfoFromGUI) |
DAT_0023f85c |
20 | Password string (max 20 chars, from StoreVMInfoFromGUI) |
DAT_0023f874 |
~72 | BMC IP address string (IPv6 bracket-stripped) |
DAT_0023f8bc |
4 | Connection type (0=IPv4, 1=IPv6, from DetermineTCPType) |
DAT_0023f8c0 |
4 | Descriptor mode flag (always 1 after SetupVMInfoBetSWAndFW) |
DAT_0023f8c4 |
4 | VM device count |
vuDevRespData (0x0023e240) |
824 | Hardcoded USB descriptor table (8 entries at 0x67 stride) |
st_VSDevConfigDescriptor (0x0023e580) |
540 | Device config descriptor table |
st_VSIADDescriptor (0x0023e7a0) |
60 | IAD descriptor table (5 entries) |
java_vsui structure layout (0x0023f560, 0x2A0 bytes):
| Offset | Size | Field | Set by |
|---|---|---|---|
| 0x000 | 20 | Username (temp) | JASetUNamePwdIPPort (via strcpy, unbounded) |
| 0x014 | 20 | Password (temp) | JASetUNamePwdIPPort (via strcpy, unbounded) |
| 0x02C | 72 | IP address | JASetUNamePwdIPPort (via memcpy, length-bounded) |
| 0x074 | 512 | Image file path | VUSPlugIn |
| 0x294 | 4 | Platform version | JASWInit |
| 0x298 | 1 | Company ID | JASWInit |
| 0x299 | 1 | Board ID | JASWInit |
Per-device structure (allocated at DAT_0023f810, stride = 0x15920):
Slots: devCount + 1 (extra slot for temporary status query connections).
Round 5 Verification Results
The following items were independently re-verified by decompiling all relevant
functions from Ghidra. All decompilations were performed on the binary
/liblinux_x86_64__V1.0.5/libiKVM64.so-730126.
ProtocolHandler Dispatch Table (VERIFIED)
The jump table at 0x00127790 was read as raw bytes and decoded. It contains exactly 61 entries (types 0x00 through 0x3C), each a 32-bit signed offset relative to the table base address. The default handler is at 0x001183f5 (returns the type byte to the caller without consuming more data).
The 7 non-default handlers are confirmed at these exact target addresses:
| Type | Target | Handler |
|---|---|---|
| 0x00 | 0x0011849a | Calls RFBScreen::ScreenDecode via vtable[0x58] |
| 0x04 | 0x001184de | Calls RFBScreen::ScreenCursorPosProc via vtable |
| 0x16 | 0x00118488 | Inline: StreamRead8() (reads 1 status byte) |
| 0x35 | 0x0011844d | Calls RFBKeyboard::ProcKeyboardInfo at offset 0x40, then falls through to 0x37 handler |
| 0x37 | 0x00118457 | Calls RFBMouse::ProcMouseInfo at offset 0x40 |
| 0x39 | 0x00118479 | Calls RFBPrivilege::ProcPrivilegeInfo at vtable[0x10] |
| 0x3C | 0x0011846a | Calls RFBScreen::GetScreenUILang via vtable[0x58] |
The raw jump table hex at 0x00127790 (first 7 non-default offsets, all others are the default 0xFFFF0C65 = -62363):
Type 0x00: 0xFFFF0D0A -> 0x0011849A
Type 0x04: 0xFFFF0D4E -> 0x001184DE
Type 0x16: 0xFFFF0CF8 -> 0x00118488
Type 0x35: 0xFFFF0CBD -> 0x0011844D
Type 0x37: 0xFFFF0CC7 -> 0x00118457
Type 0x39: 0xFFFF0CE9 -> 0x00118479
Type 0x3C: 0xFFFF0CDA -> 0x0011846A
Confirmed: Type 0x04 IS cursor position (not screen control). The handler
calls ScreenCursorPosProc which reads cursor_x, cursor_y, cursor_width,
cursor_height, cursor_type, and optionally cursor pixel data. Previous
documentation was correct on this point.
Authentication Flow (VERIFIED)
RFBProtocol::Authenticate at 0x00118190 decompilation confirms:
StreamRead(auStack_28, 0x18)-- reads 24-byte challenge into LOCAL variableStreamWrite(&stack0x00000008, 0x18)-- writes 24 bytes from username parameterStreamWrite(&stack0x00000020, 0x18)-- writes 24 bytes from password parameterStreamRead32()-- reads auth_result (u32)
The challenge is read into auStack_28[32] (local stack variable) and NEVER
referenced again. It is genuinely discarded. The username and password
parameters come from the Userinfo_t struct passed by value on the stack.
The checkValidUser JNI at 0x00119c60 copies username with strcpy(local_a8, __src)
into a 24-byte buffer and password with strcpy(local_90, __src_00) into a 96-byte
buffer. This is a buffer overflow vulnerability if the Java side provides strings
longer than 24/96 bytes respectively.
Security finding confirmed: Credentials are sent in plaintext. The 24-byte challenge is theater -- it exists in the protocol but is completely ignored by the client implementation.
Connection Handshake (VERIFIED)
All handshake functions were decompiled. Key findings:
-
InitHandShake(0x00117fa0): Allocates NtwStream (0x628 bytes), stores at this+0x08. Calls Connect(), ProcVersion(), ProcSecurity(). Returns 0 on any failure, 1 on success. -
ProcVersion(0x00117e40): Reads 12 bytes, parses withsscanf("RFB %03d.%03d\n"), compares against this+0x40 (=3) and this+0x44 (=8). Echoes the same 12 bytes back. -
ProcSecurity(0x00117f10): Reads count (u8), then readscountsecurity types (u8 each) in a loop. The LAST type read is selected and sent back (1 byte). Note: the variable holding the selected type (unaff_R13B) is the register that holds the last value from the loop -- confirming "last type wins" behavior. -
Authenticate(0x00118190): See above. -
ProcClientInit(0x00117ee0): Sends single byte 0x00 (shared_flag = exclusive). -
ProcServerInit(0x00118040): Reads standard RFB ServerInit (2+2+16+4+name bytes), all discarded. Then reads 4 bytes padding (skip), session_id (u32), and 4 bytes of aes_key_seed (read as 4 individual StreamRead8 calls). Creates a temporary RFBPrivilege (0x120 bytes), calls ViewerConfig(session_id, &key_seed) on it via vtable[0x28], then immediately destroys it via vtable[0x00].
GetDecoder (VERIFIED)
RMDecoder::GetDecoder at 0x00112560 is a singleton factory. The global
DecoderHandle is checked first -- if non-NULL, it is returned immediately.
Otherwise a switch on the encoding type:
- 0x57 or 0x58:
ASTVideoDecoder(0x58 bytes) - 0x59:
HermonVideoDecoder(0x58 bytes) - 0x60:
YarkonVideoDecoder(0x60 bytes) - 0x61:
Pilot3VideoDecoder(0x458 bytes)
There is NO default case and no fallback. Unknown encoding types result in
DecoderHandle remaining NULL, and the function returns NULL.
JNI Function Behavior (VERIFIED)
| JNI Function | Behavior |
|---|---|
changeLEDstate (0x00120cd0) |
Purely local X11. Opens display, simulates key press/release for CapsLock (VK 0x39->0xFFE5), ScrollLock (VK 0x47->0xFF14), or NumLock (VK 0x53->0xFF7F) using XTestFakeKeyEvent, closes display. NO network I/O. |
updateInfo (0x00120be0) |
Calls MouseUpdateInfo() via vtable on desktop+0x08 (RFBMouse), vtable offset 0x28. Sends type 0x37 on wire. |
changeScreenInfo (0x00120c00) |
Calls ScreenGetRect() then ScreenSetInfo() via vtable on desktop+0x10 (RFBScreen). Sends type 0x32 if resolution changed. Also refreshes JNI buffer references. |
doCatch (0x00121570) |
Sets global filterFlag = param_3. No message processing. |
catchLoop (0x001216c0) |
Empty no-op function (contains only return). |
refresh (0x00121510) |
JNI MonitorEnter/MonitorExit on frameObj, resets flag. No network I/O. |
sendPrivilegeCtrl (0x00121970) |
Extracts 64-byte array from Java via GetByteArrayRegion(0, 0x40), calls SendKickRequest(action, data) via vtable on desktop+0x18 (RFBPrivilege) offset 0x20. Sends type 0x38 (73 bytes). |
Round 6 Final Sweep Results
The following items were verified and discovered by decompiling all relevant
functions from Ghidra. All decompilations were performed on the binary
/liblinux_x86_64__V1.0.5/libSharedLibrary64.so-161223.
Byte-Level Wire Format Verification (ALL CONFIRMED CORRECT)
Every critical packet format was verified by decompiling the function that constructs it. All documented byte offsets in MOUNT_PROTOCOL.md are accurate:
- PlugIn (FillUSBPlugInPkt at 0x001167c0): 52 bytes + USB descriptors
- SetEP (FillSetEPCMDPkt at 0x00116234): 16 bytes, 3 hardcoded endpoints
- KeepAlive (FillKeepAlivePkt at 0x0011b1c2): 12 bytes, payload 0xFFFFFFFF
- DevStatus (Core_GetDevStatusFromFW at 0x0011741e): 8-byte query, type=0x08
- HTTP Port (GetHttpPortFromFW at 0x0011cdd4): 8-byte query, type=0x0A (leaks socket)
- UnMount (Core_VM_SendUnMountPkt at 0x0011bdf2): 8 bytes, type=5
- PDU Header (State_Fn_RX_PDU_TAG at 0x0011b24c): type=BE, length=LE
- CSW (FillCSW at 0x00119fe8): 21 bytes, both IN and OUT paths verified
- Data PDU (FillDataPDUTag at 0x00119d10): 8-byte header, IN direction only
Key Discoveries
-
Agent 4 SID/USB ID correction: Offsets 0x2C-0x2F in PlugIn packet are SID bytes (from
GetRandomTimeStamp), NOT USB vendor/product IDs as Agent 4 incorrectly labeled. MOUNT_PROTOCOL.md correctly identifies these as SID bytes. -
VMInfoCalloc extra slot:
calloc((devCount+1), 0x15920)allocates an extra slot beyond the device count, used as a temporary connection slot for status queries byCore_GetDevStatusFromFW. -
SetupVMInfoBetSWAndFW descriptor reset: Before mount, all HID keyboard/ mouse descriptors and all 5 IAD descriptors are explicitly disabled. Only the current device's descriptor is re-enabled by
MtMethod_Media. -
StoreVMInfoFromGUI validation: Username < 17 chars, password < 21 chars. IPv6 addresses have brackets stripped. However,
JASetUNamePwdIPPortcopies credentials via unboundedstrcpyBEFORE this validation occurs. -
Floppy WRITE(10) 32-bit limitation: Uses
fseek(32-bit) while READ(10) usesfseeko64(64-bit). Latent bug for images > 2GB. -
FillErrorData logic: CSW status is PASS (0x00) if both ASC and ASCQ are zero; otherwise FAIL (0x01). No finer-grained status mapping.
-
SCSI handlers fully enumerated: ISO has 26 opcodes, floppy has 10. Newly documented handlers: MECHANISM STATUS (0xBD), START STOP UNIT (0x1B), SET READ AHEAD (0xA7), SET CD SPEED (0xBB), READ BUFFER (0x3C).
New Function Addresses
| Address | Function | Purpose |
|---|---|---|
| 0x001178c8 | InitVMSW | Clear st_VMMainInfo, init queues, generate SID |
| 0x00120bb2 | VMInfoCalloc | calloc((devCount+1), 0x15920) |
| 0x00117b92 | SetState_ReadPDU | Reset state to 0x01, expect 8-byte header |
| 0x0011ced4 | SetupVMInfoBetSWAndFW | Pre-mount setup with verified address |
| 0x0011f958 | UI_Mount_VM | Full mount orchestration |
| 0x0011d28c | StoreVMInfoFromGUI | Credential validation and storage |
| 0x0011667c | UI_PrepareVMResource | Resource preparation and memory allocation |
| 0x0011b24c | State_Fn_RX_PDU_TAG | PDU header parser and dispatch |
| 0x0011ac68 | State_Fn_RX_CBW | CBW parsing state |
| 0x00117cc8 | State_Fn_RX_BULK_OUT_DATA | Bulk OUT data receive state |
| 0x0011b694 | State_Fn_RX_MOUNT_DEV_STATUS | Mount status response handler |
| 0x001288be | MachanismStatus | MECHANISM STATUS (0xBD) handler |
| 0x00128aa0 | Startstopunit | START STOP UNIT (0x1B) handler |
| 0x00128378 | SetReadAhead | SET READ AHEAD (0xA7) handler |
| 0x00128e80 | SetCDSpeed | SET CD SPEED (0xBB) handler |
| 0x0012a7aa | ReadBuffer | READ BUFFER (0x3C) handler |
Areas for Further Investigation
Completed (Final Round)
-
[DONE] ScreenDecode exact wire format --
RFBScreen::ScreenDecodeat 0x00119090 fully decompiled. Exact StreamRead sequence after type 0x00: StreamReadSkip(3), StreamRead16 x4 (x, y, w, h), StreamRead32 (encoding), StreamRead32 (frame_number), StreamRead32 (data_length), StreamRead(buf, data_length). Total: 23 bytes header + data_length. The 3-byte skip covers standard RFB padding(1) + num_rects(2). Width/height are signed 16-bit and abs()'d before decode. See KVM_PROTOCOL.md for complete byte-level docs. -
[DONE] Complete client-to-server message enumeration -- All 22 functions containing
StreamWrite8calls were identified via code search. 19 unique client-to-server message type bytes found (one StreamWrite8 is ProcClientInit which writes 0x00 as shared_flag, not a message type; two StreamWrite8 calls are for encrypt_flag and button_mask subfields). The previously missing type is 0x35 (KeyboardUpdateInfo) -- a 1-byte client request for keyboard+mouse config, sent byRFBKeyboard::KeyboardUpdateInfoat 0x0011a610. Complete list: 0x03, 0x04, 0x05, 0x07, 0x08, 0x15, 0x16, 0x17, 0x19, 0x1A, 0x32, 0x35, 0x36, 0x37, 0x38, 0x3A, 0x3B, 0x3C, 0x3D. -
[DONE] KeepAlive (type 0x16) details -- Server sends type 0x16 + 1 byte (status). The ProtocolHandler inline handler at 0x00118488 calls StreamRead8() and discards the result (not stored or checked). The status byte has no significance. Client ACK:
RFBProtocol::ProcAlive(bool)at 0x00118230 sends type 0x16 + the bool parameter. ThesendKeepAliveAckJNI at 0x00121a00 always passes 1 (true) via vtable offset 0x28. A client should always send0x16 0x01in response. -
[DONE] Object allocation sizes -- All sizes verified from operator_new calls in constructors: NtwStream=0x628, RFBProtocol=0x70, RFBScreen=0x2080, RFBMouse=0x28, RFBKeyboard=0xB8, RFBPrivilege=0x120, RFBKMCryto=0x2110, RMConnection=0x10, RMDesktop=0x30, TcpSocket=0x60, ErrMsg=0x44. The RMDesktop constructor (0x0011e0f0) allocates all sub-objects: RFBKeyboard (0xB8), RFBMouse (0x28), RFBScreen (0x2080), RFBPrivilege (0x120).
-
[DONE] X11 indicator state bit mapping correction -- The init JNI at 0x0011fbd0 reads XkbGetIndicatorState and tests: bit 0 (& 1) = scrollLock, bit 1 (& 2) = numLock, bit 2 (& 4) = capsLock. Previous documentation had these reversed (bit 0 = CapsLock). Corrected in KVM_PROTOCOL.md.
High Priority (Previously Completed)
-
[DONE] RFBProtocol::ProtocolHandler dispatch table -- Fully mapped. Jump table at 0x00127790 has 61 entries (types 0x00-0x3C). Only 7 have non-default handlers: 0x00 (FramebufferUpdate), 0x04 (CursorPosition), 0x16 (KeepAlive), 0x35 (KeyboardInfo+MouseInfo), 0x37 (MouseInfo only), 0x39 (PrivilegeInfo), 0x3C (GetScreenUILang). All handler code is inline within the ProtocolHandler function body. Previous documentation had incorrect type IDs (0x31, 0x33 don't exist; 0x38 is client-only).
-
[DONE] Video decoder internals (ast_jpeg / 0x58) -- The AST2100+ decoder (
ast_jpegclass) uses a JPEG-like scheme with 4:2:0 YCbCr subsampling: 4 Y blocks + 1 Cb + 1 Cr per tile (16x16 pixels). Fixed Huffman tables (not embedded in stream). Tile command stream uses 2-bit command types: same-position or new-position, with two QT table selections. 16-byte header provides quantization levels, tile sizes, and total size. See KVM_PROTOCOL.md for full wire format.
2b. [DONE] AST2050 decoder (ast2100 class, encoding 0x57) -- Fully
decompiled. Uses a hybrid VQ + Huffman + IDCT decoder with optional RC4
decryption. RC4 key is the hardcoded string "fedcba9876543210" (at
0x00230500). The 4-byte header provides mode, quality, and chip revision
(0x01A6 = 16x16 tiles, 0x01BC = 8x8 tiles). Tile commands are encoded as
4-bit nibble codes in the bitstream. VQ uses a 4-entry COLOR_CACHE palette.
See KVM_PROTOCOL.md for complete wire format documentation.
-
[DONE] system() and popen() call sites --
system()PLT stub at 0x001143b0,popen()PLT stub at 0x00112ae0. Both are called from the FAT filesystem emulation (Linux_TFATFileSystemImage_*) and physical device scanning (Linux_ScanPhyStor_*) code paths. Nosystem()/popen()calls were found in the main protocol engine, SCSI handlers, or mount path. The calls appear limited to folder mount file operations and hardware enumeration. Security risk is mitigated by the fact that user-controlled file paths do not directly reach these calls (they go throughStoreVMInfoFromGUIvalidation first), but the folder mount path should still be considered a potential command injection surface. -
[DONE] Authentication flow --
RFBProtocol::Authenticateat 0x00118190 uses ATEN-proprietary plaintext auth: reads 24-byte challenge (discarded), sends 24-byte username + 24-byte password in cleartext, reads 4-byte result. No VNC DES challenge-response or other standard security type handling. The security type negotiation is a formality -- the client blindly selects the last type offered.
Medium Priority
-
Physical device scanning -- The
Linux_ScanPhyStor_*functions enumerate hardware devices on the client machine. What paths are scanned? Are there symlink/TOCTOU issues? -
FAT filesystem emulator -- The
TFATFileSystem_*family implements a complete FAT16 filesystem in memory for folder mounting. The complexity here is a potential source of bugs (path traversal, buffer overflows in filename handling, etc.). -
[DONE] Error message / exception flow -- The
ErrMsgclass is a 0x44-byte struct with error_code (int) + error_message (char[64]). Error codes: -1 (socket closed/failed), -2 (timeout), -3 (multi-session not supported), -4 (file closed), -5 (file corrupted). Exceptions are thrown in TcpSocket read/write and video decoders, caught in runImage/checkValidUser/keepActive JNI functions. TheerrorHandler(int)JNI callback (registered at init) propagates error codes to Java. Socket timeout is 30 seconds. -
[DONE] Hermon and Yarkon decoders -- Fully decompiled. Hermon uses raw 16bpp/8bpp tiles with simple pixel format conversion. Yarkon uses Huffman/RLE compression with RFB Hextile sub-encoding for incremental updates. Both throw "Not Support Multi-Session!" for encoding 0x10001. See KVM_PROTOCOL.md for complete wire formats.
-
[DONE] Screen recording subsystem -- The
screenRecording*exports use a simple file-based capture/playback mechanism.startCaptureopens a file and sets a flag; the ProtocolHandler's type 0x00 handler writes received frame data to the capture file when the flag is set. Playback reads from a file via TcpSocket's file mode (offset 0x44 != 0). Recording is raw protocol data (not AVI). The Java side usesAVIOutputStreamfor the actual AVI recording.Huffman_Compress/Uncompressare used by the Yarkon decoder, not directly by the recording subsystem.
Low Priority
-
[DONE] Keyboard LED synchronization -- The
keyboardActionJNI function at 0x00121000 opens/closes an X11 display connection for each lock key event. It reads the indicator state viaXkbGetIndicatorState(display, 0x100, &state): bit 0 = ScrollLock, bit 1 = NumLock, bit 2 = CapsLock. If the lock is not active, the VK code is modified with 0xFF in the low byte. The X11 connection is ephemeral (opened, read, closed) -- no persistent display handle. -
IPv6 support -- The
Linux_VMTCPConnect_IPv6_Defaultfunction and bracket/scope stripping logic in the Java side. IPv6 parsing is notoriously error-prone. -
HTTP upload protocol -- The multipart/form-data construction and response parsing in
MtMethod_WebISO/MtMethod_UploadIMA. Cookie handling (SID=) may have injection issues. -
[DONE] USB descriptor assembly --
FillUSBPlugInPktat 0x001167c0 fully decompiled and verified byte-by-byte. The descriptor mode flag (DAT_0023f8c0, always 1 afterSetupVMInfoBetSWAndFW) controls idProduct (0x2222 vs 0x1111). 8 entries fromvuDevRespDatatable are assembled, with Entry 1 (Configuration Descriptor) dynamically including Interface + Endpoint descriptors. Before mount, all HID/IAD descriptors are disabled and only the current device's Mass Storage interface is enabled.
Remaining Open Questions (from final round)
-
Server-side KeepAlive status byte: The server sends a status byte with each KeepAlive (type 0x16), but the client discards it. Does the BMC send different values (e.g., 0 for idle, 1 for active)? Only observable via packet capture against a live BMC.
-
Type 0x35 dual-purpose: The type 0x35 is used bidirectionally -- the client sends it as a 1-byte config request, the server sends it as a 6-byte config message. This asymmetry is unusual. Verify with packet captures whether the BMC correctly distinguishes the two directions by context.
-
frame_number semantics: The first frame has frame_number=1 (triggers resolution change notification), subsequent frames have frame_number=0. Are there other frame_number values? Does it ever increment beyond 1? Only observable from live traffic.
-
PrivilegeInfo session_word meaning: The (lo=1, hi=4) comparison in ProcPrivilegeInfo triggers SetThreadNormaleStart(0). What does this mean? Is (1,4) a "session takeover" indicator? What are other observed values?
-
Server behavior on unknown client types: What happens if the client sends a message type the BMC does not expect? Does the BMC disconnect, ignore, or desynchronize?
-
Nagle's algorithm: The
enableNagles()function in libiKVM64.so is a no-op stub. Similarly, libSharedLibrary64.so never setsTCP_NODELAY--send()calls use flags=0. This means Nagle's algorithm is active on both the KVM and VM connections, which could cause latency issues for small messages. A real client implementation should consider setting TCP_NODELAY. -
SIGPIPE handling (VM library): libSharedLibrary64.so does not set
MSG_NOSIGNALonsend()and has nosignal(SIGPIPE, SIG_IGN)handler (unlike libiKVM64.so which hasBrokenPipe_handle). A broken pipe on the VM connection would deliver SIGPIPE and could crash the JVM.
Additional Discovered Functions and Addresses
Authentication path:
| Function | Address | Purpose |
|---|---|---|
RFBProtocol::InitHandShake |
0x00117fa0 | TCP connect + version + security negotiation |
RFBProtocol::Authenticate |
0x00118190 | ATEN plaintext auth + ClientInit + ServerInit |
RFBProtocol::ProcVersion |
0x00117e40 | RFB version exchange |
RFBProtocol::ProcSecurity |
0x00117f10 | Security type negotiation |
RFBProtocol::ProcClientInit |
0x00117ee0 | Send shared_flag = 0x00 |
RFBProtocol::ProcServerInit |
0x00118040 | Read server init + ATEN extensions |
RFBProtocol::SetFWProtocol |
0x00118530 | Set firmware protocol flag byte |
Server message handlers (called from ProtocolHandler):
| Function | Address | Server Type | Purpose |
|---|---|---|---|
RFBScreen::ScreenDecode |
0x0010E7E8 (DEFAULT) / 0x00119090 (IMPORTED) | 0x00 | Video frame decode |
RFBScreen::ScreenCursorPosProc |
0x0010E7F8 (DEFAULT) / 0x00118FB0 (IMPORTED) | 0x04 | Cursor position + image |
RFBKeyboard::ProcKeyboardInfo |
0x0011A720 | 0x35 | Keyboard mode/type config |
RFBMouse::ProcMouseInfo |
0x001198C0 | 0x35, 0x37 | Mouse mode/type/encrypt config |
RFBPrivilege::ProcPrivilegeInfo |
0x0011A120 | 0x39 | Session + AES key material |
RFBPrivilege::ExePrivilegeCtrl |
0x0011A1B0 | -- | Forward privilege data to Java |
RFBPrivilege::ViewerConfig |
0x0011A1D0 | -- | Store session_id + config |
RFBScreen::GetScreenUILang |
0x00118CB0 | 0x3C | OSD language response |
Client message senders:
| Function | Address | Client Type | Purpose |
|---|---|---|---|
RFBScreen::ScreenUpdate |
0x00118850 | 0x03 | Send FramebufferUpdateRequest (10 bytes) |
RFBKeyboard::Sendkey |
0x0011A670 | 0x04 | Send KeyEvent (18 bytes, always cleartext) |
RFBMouse::SendMouse |
0x001194F0 | 0x05 | Send PointerEvent (encrypted or clear) |
RFBMouse::MouseSync |
0x00119730 | 0x07 | Send MouseSync (value 0x0780) |
RFBMouse::MouseReset |
0x00119780 | 0x08 | Send MouseReset (+ mouse_type) |
RFBScreen::ScreenSetPosition |
0x001189B0 | 0x15 | Send screen position |
RFBProtocol::ProcAlive |
0x00118230 | 0x16 | Send KeepAlive ACK |
RFBScreen::ScreenCalibration |
0x00118960 | 0x17 | Trigger screen recalibration |
RFBScreen::GetCursorPos |
0x00118D60 | 0x19 | Request cursor position from server |
RFBScreen::SetPowerOnOff |
0x00118900 | 0x1A | Power control (off/on/reset/soft-off) |
RFBScreen::ScreenSetInfo |
0x00118B80 | 0x32 | Send resolution |
RFBKeyboard::KeyboardUpdateInfo |
0x0011A610 | 0x35 | Request keyboard+mouse config (1 byte) |
RFBMouse::MouseSetPT |
0x00119820 | 0x36 | Set pointer type (mode + type) |
RFBMouse::MouseUpdateInfo |
0x001197D0 | 0x37 | Request mouse config from server |
RFBPrivilege::SendKickRequest |
0x0011A1F0 | 0x38 | Send kick request (73 bytes) |
RFBMouse::MouseHotPlug |
0x00119910 | 0x3A | Mouse hot-plug reset (1 byte) |
RFBProtocol::ProcQos |
0x00118290 | 0x3B | Send QoS parameters (3 x u32) |
RFBProtocol::ProcGetScreenUI |
0x001183A0 | 0x3C | Request OSD language (1 byte) |
RFBProtocol::ProcSetScreenUI |
0x00118320 | 0x3D | Set OSD language (conditional u32) |
JNI helper functions:
| Function | Address | Purpose |
|---|---|---|
sendViewerConfig |
0x0011FB40 | Send g_config + g_session_id to Java |
storeViewerConfig |
0x001216A0 | Store session_id and config bytes (strncpy 4 bytes) |
SetThreadNormaleStart |
0x00121640 | Notify Java of normal/non-normal session |
privilegeControl |
0x00121580 | Forward privilege info (lo, hi, 256-byte key) to Java |
getScreenUILangConfig |
0x00121B80 | Forward OSD lang (2 x u32) to Java |
addClipBounds |
0x0011FAA0 | Forward dirty rectangles (4 x int) to Java |
getQuickCursor |
(called from ScreenCursorPosProc) | Notify Java of cursor update |
Error handling functions:
| Function | Address | Purpose |
|---|---|---|
ErrMsg typeinfo |
0x0022DF70 | C++ exception type (0x44 bytes: int code + char[64] msg) |
ErrMsg typeinfo-name |
0x00122415 | "6ErrMsg" mangled name |
TcpSocket::read |
0x00117310 | Throws ErrMsg on socket errors (-1, -2, -4, -5) |
TcpSocket::write |
0x00117530 | Throws ErrMsg on socket errors (-1, -2) |
NtwStream Implementation Details
The NtwStream class provides buffered network I/O with mutex-based write
batching. The object layout includes a pthread_mutex_t at the beginning,
a 1518-byte internal write buffer, and pointers to the underlying socket.
Object layout (partial):
Offset Size Description
0x00 varies pthread_mutex_t (write lock, ~40 bytes on Linux x86-64)
0x28 0x5F0 Internal write buffer (1520 bytes = offset 0x618 - 0x28)
0x618 8 Write cursor pointer (points into buffer at 0x28..0x618)
0x620 8 Socket vtable pointer (StreamSocket/TcpSocket instance)
Buffer size: The internal write buffer spans from offset 0x28 to 0x618
(1520 bytes, or 0x5F0). This is based on the capacity check in StreamWrite():
if ((this - __dest) + 0x616 < param_2) where __dest = *(this + 0x618).
This equates to an Ethernet MTU-sized buffer (1518 bytes is standard Ethernet
MTU + 2 bytes for alignment).
StreamWriteStart (0x0010f370): Simply calls pthread_mutex_lock() on the
mutex at the beginning of the NtwStream object. This locks the stream for an
atomic write batch.
StreamWriteFlush (0x0010f380): Sends buffered data and releases the lock:
- If write cursor != buffer start (i.e., there is buffered data):
- Calls the socket's
writevirtual method (vtable offset 0x20) with the buffer contents from 0x28 through the cursor position - Resets write cursor to buffer start (0x28)
- Calls the socket's
- Calls
LeaveCriticalSection()which wrapspthread_mutex_unlock()
StreamWrite (0x0010f1f0): Buffers data or flushes if buffer is full:
- If the data fits in the remaining buffer space:
memcpy()into buffer and advance the cursor - If data exceeds remaining space:
- Flush current buffer directly via socket write
- Send the new data directly via socket write (bypasses buffer)
- Reset cursor to buffer start
StreamRead (0x0010f0a0): Reads exactly param_2 bytes from the socket:
- Loops calling the socket's
readvirtual method (vtable offset 0x18) - Accumulates partial reads until all bytes are received or an error occurs
- Returns total bytes read (should equal
param_2on success)
StreamReadSkip (0x0010f180): Allocates a temporary buffer, reads N bytes
into it via StreamRead(), then frees the buffer. Simple but wasteful -- it
allocates and zeroes memory just to discard the bytes.
StreamWriteSkip (0x0010f3f0): Same pattern as StreamReadSkip -- allocates
a zeroed buffer and writes it via StreamWrite(). This sends N zero bytes.
StreamRead8/16/32: Thin wrappers around StreamRead:
StreamRead8(): Reads 1 byte, returns it directlyStreamRead16(): Reads 2 bytes, returns big-endian u16StreamRead32(): Reads 4 bytes, returns big-endian u32
StreamWrite8/16/32: Thin wrappers around StreamWrite:
StreamWrite8(val): Writes 1 byteStreamWrite16(val): Writes 2 bytes big-endian (high byte first)StreamWrite32(val): Writes 4 bytes big-endian (MSB first)
Socket layer (TcpSocket):
TcpSocket::read(0x00117310): Usesselect()with 30-second timeout +recv(). Throws ErrMsg exceptions: -1 (closed), -1 (failed), -2 (timeout). Also supports file replay mode (offset 0x44 != 0) usingfread().TcpSocket::write(0x00117530): Usesselect()with 30-second timeout +send(). Throws ErrMsg exceptions: -1 (closed), -1 (failed), -2 (timeout).TcpSocket::EstablishConnection(0x00117770): Usesgetaddrinfo()+socket()+connect()for client mode;bind()+listen()+accept()for server mode (offset 0x5c != 0). CallsenableNagles()which is a no-op stub (Nagle's algorithm is not disabled -- TCP_NODELAY is never set).TcpSocket::EndSock(0x00117aa0): Callsshutdown(fd, SHUT_RDWR)thenclose(fd). Manages a reference count (RefSocketCount) for shared listen socket cleanup.
Connection Teardown
destory [sic] (Java_tw_com_aten_ikvm_ui_RemoteVideo_destory at
0x0011ffd0):
The destory() JNI method performs complete session cleanup:
- Saves
desktopandconnectionglobal pointers to local variables - Sets both global pointers to NULL immediately (prevents double-free)
- If
desktopwas non-NULL:- Calls
RMDesktop::~RMDesktop()which destroys the composition:- Destroys the RFBScreen (includes video decoder cleanup)
- The destructor chain destroys keyboard, mouse, screen, privilege objects
- Frees the RMDesktop memory via
operator delete
- Calls
- If
connectionwas non-NULL:- Calls the deleting destructor via vtable (offset 0x08) which:
- Destroys the RFBProtocol (including NtwStream)
- NtwStream destructor destroys the TcpSocket
- TcpSocket destructor calls
EndSock(): a.shutdown(socket_fd, SHUT_RDWR)-- half-close both directions b.close(socket_fd)-- release file descriptor c. DecrementsRefSocketCount; if zero, also shuts downListenfd - Frees the TcpSocket memory
- Calls the deleting destructor via vtable (offset 0x08) which:
There is no graceful disconnect message. The connection is torn down by simply shutting down the TCP socket. There is no RFB "disconnect" or "bye" message sent to the server. The server detects the disconnect via TCP RST or FIN when the socket is closed.
Server disconnect handling: When the server disconnects, the next
TcpSocket::read() call will either:
- Return 0 (EOF) -> throws ErrMsg(-1, "Read: Socket closed")
- Return -1 (error) -> throws ErrMsg(-1, "Socket Read Failed")
- Timeout after 30 seconds -> throws ErrMsg(-2, "Socket Read Timeout")
The C++ exception propagates up through the JNI call stack. The Java-side
catch handler in runImage() / catchLoop() receives the error code via
the errorHandler(int) callback and terminates the session.
AST2050 Decoder Internals (ast2100 class)
Function addresses:
| Function | Address | Purpose |
|---|---|---|
ast2100::ast2100 (C1) |
0x0011b5f0 | Constructor |
ast2100::ast2100 (C2) |
0x0011b5c0 | Constructor (delegating) |
ast2100::~ast2100 (D1) |
0x0011ca10 | Destructor |
ast2100::~ast2100 (D2) |
0x0011c9e0 | Destructor (delegating) |
ast2100::PsudoStreamSwap16 |
0x0011b620 | Byte-swap u16 (big-endian) |
ast2100::GetINFData |
0x0011b640 | Initialize decoder info defaults |
ast2100::InitParameter |
0x0011b6f0 | Set tile/mode params from chip revision |
ast2100::SetBuffer |
0x0011b7e0 | Set framebuffer dimensions and mode |
ast2100::Keys_Expansion |
0x0011b830 | Expand RC4 key to 256 bytes (cyclic) |
ast2100::DecodeRC4_setup |
0x0011b870 | RC4 KSA (Key Scheduling Algorithm) |
ast2100::RC4_crypt |
0x0011b8f0 | RC4 PRGA - XOR decrypt/encrypt |
ast2100::WORD_hi_lo |
0x0011b960 | Byte order utility |
ast2100::prepare_range_limit_table |
0x0011b970 | Build clamping table |
ast2100::lookKbits |
0x0011ba20 | Peek N bits from bitstream |
ast2100::skipKbits |
0x0011ba40 | Consume N bits from bitstream |
ast2100::getKbits |
0x0011bad0 | Get N bits from bitstream |
ast2100::init_QT |
0x0011bb10 | Init quantization table pointers |
ast2100::set_quant_table |
0x0011bb40 | Set quantization table entries |
ast2100::load_quant_table |
0x0011bbb0 | Load quantization table (Y) |
ast2100::load_quant_tableCb |
0x0011bd50 | Load quantization table (Cb/Cr) |
ast2100::load_advance_quant_table |
0x0011bf70 | Load advance mode QT |
ast2100::load_advance_quant_tableCb |
0x0011c0e0 | Load advance mode QT (Cb/Cr) |
ast2100::process_Huffman_data_unit |
0x0011c430 | Huffman decode one 8x8 block |
ast2100::init_jpg_table |
0x0011c6c0 | Initialize JPEG Huffman/QT tables |
ast2100::SetOptions |
0x0011c770 | Parse 4-byte header, init decoder state |
ast2100::FreeQT |
0x0011c990 | Free quantization tables |
ast2100::init_JPG_decoding |
0x0011c940 | Initialize JPEG decoding state |
ast2100::updatereadbuf |
0x0011ce30 | Advance bitstream read position |
ast2100::YUVToRGB |
0x0011cec0 | YCbCr -> BGRX32 color conversion |
ast2100::Decompress |
0x0011d150 | Huffman+IDCT decompress one tile |
ast2100::VQ_Decompress |
0x0011d490 | VQ decompress one tile |
ast2100::MoveBlockIndex |
0x0011d590 | Advance tile position, record dirty rect |
ast2100::VQ_Initialize |
0x0011d670 | Initialize 4-entry COLOR_CACHE |
ast2100::decode |
0x0011d6a0 | Main decode entry - tile command loop |
ast2100::get_decoderID |
0x0011df60 | Return decoder identifier |
ast2100::get_ImageData |
0x0011df00 | Return decoded image data pointer |
ast2100::PsudoStreamSwap32 |
0x0011df30 | Byte-swap u32 |
RC4 key details:
- ASCII:
fedcba9876543210(16 bytes at 0x00230500, null-terminated) - The key is expanded to 256 bytes by
Keys_Expansion()which repeats the key cyclically:key[i] = key[i % strlen(key)]for i = 0..255 DecodeRC4_setup()is a standard RC4 KSA: initializes S-box [0..255], then permutes based on expanded keyRC4_crypt()is a standard RC4 PRGA: XOR each data byte with keystream
RC4 purpose: The RC4 encryption is used to decrypt the tile bitstream data
when the encrypt_flag (this+0x101f6d) is set. It appears to be an optional
obfuscation layer that some ASPEED AST2050 firmware versions enable. The first
frame check (this+0x101e78 == 0) triggers key setup; subsequent frames reuse
the RC4 state. The encryption flag can change between frames.
Hermon/Yarkon/Pilot3 Decoder Internals
HermonVideoDecoder functions:
| Function | Address | Purpose |
|---|---|---|
HermonVideoDecoder::HermonVideoDecoder (C1) |
0x0010fc70 | Constructor |
HermonVideoDecoder::~HermonVideoDecoder (D1) |
0x0010fd20 | Destructor |
HermonVideoDecoder::GetDecodedFrame |
0x0010fd50 | Return framebuffer ptr |
HermonVideoDecoder::ConvertVierwerPixelFormat |
0x0010fd60 | 16bpp/8bpp -> 32bpp |
HermonVideoDecoder::SetRect |
0x0010fe30 | Copy 16x16 tile to framebuffer |
HermonVideoDecoder::Decode |
0x0010fee0 | Main decode: full or incremental |
HermonVideoDecoder::MixedCursor |
0x00110120 | Cursor compositing |
YarkonVideoDecoder functions:
| Function | Address | Purpose |
|---|---|---|
YarkonVideoDecoder::YarkonVideoDecoder (C1) |
0x001101c0 | Constructor |
YarkonVideoDecoder::~YarkonVideoDecoder (D1) |
0x00110280 | Destructor |
YarkonVideoDecoder::GetDecodedFrame |
0x001102b0 | Return framebuffer ptr |
YarkonVideoDecoder::AssignRectColour |
0x001102c0 | Fill rect with solid color |
YarkonVideoDecoder::ConvertVierwerPixelFormat |
0x00110380 | RGB555 -> BGRX32 |
YarkonVideoDecoder::SetRect |
0x00110400 | Copy tile to framebuffer |
YarkonVideoDecoder::HextileDecoder |
0x001104a0 | RFB Hextile tile decode |
YarkonVideoDecoder::Decode |
0x00110630 | Main: Huffman/RLE + hextile |
YarkonVideoDecoder::MixedCursor |
0x00110850 | Cursor compositing |
Huffman_Compress |
0x00121f40 | Huffman compression (for capture) |
Huffman_Uncompress |
0x001221d0 | Huffman decompression |
RLE_Uncompress |
0x0011a520 | RLE decompression with escape chars |
Pilot3VideoDecoder functions:
| Function | Address | Purpose |
|---|---|---|
Pilot3VideoDecoder::Pilot3VideoDecoder (C1) |
0x00110900 | Constructor |
Pilot3VideoDecoder::~Pilot3VideoDecoder (D1) |
0x001109e0 | Destructor |
Pilot3VideoDecoder::GetDecodedFrame |
0x00110a20 | Return framebuffer ptr |
Pilot3VideoDecoder::ConvertVierwerPixelFormat |
0x00110a30 | Multi-format pixel convert |
Pilot3VideoDecoder::SetRect |
0x00110c20 | Copy 32x32 tile to framebuffer |
Pilot3VideoDecoder::Decode |
0x00110d20 | Main: multi-mode RLE + planar decode |
Pilot3VideoDecoder::MixedCursor |
0x00112490 | Cursor compositing |
Pilot3 object layout (0x458 bytes):
Offset Size Description
0x0C 2 Screen width (short)
0x0E 2 Screen height (short)
0x28 8 Working framebuffer pointer (converted pixels)
0x30 0x400 Color palette (256 entries x 4 bytes = 1024 bytes)
0x430 8 Decompression buffer pointer (8 MB)
0x438 8 Planar conversion buffer pointer (8 MB)
0x440 8 Input data pointer (compressed data)
0x448 8 Output framebuffer pointer (display)
0x450 4 Compressed data size
0x454 4 Total pixel data size (width * height * 4)
Screen Recording Subsystem
Global state variables:
| Variable | Address | Purpose |
|---|---|---|
startCapture |
0x04923680 | State machine: 0=idle, 1=started, 2=recording, -1=stopping |
captureFile |
0x04923628 | FILE* for capture output |
captureName |
0x04923640 | Filename string for capture |
screenSource |
(BSS) | Playback source: 0=none, 1=file loaded |
sourceFile |
(BSS) | FILE* for playback source |
sourceName |
(BSS) | Filename string for playback |
playbackStatus |
(BSS) | Playback state: 0=stopped, 1=playing, 2=stopped |
isReplayRecord |
(BSS) | Replay record flag |
Recording flow:
screenRecordingStartCapture(filename): Opens file for writing, setsstartCapture=1, stores filename incaptureName- In
RFBProtocol::ProtocolHandler(), whenthis->0x20 & 1(capture flag): the FramebufferUpdate data is written tocaptureFileviafwrite() screenRecordingStopCapture(): SetsstartCapture=-1(stopping)
Playback flow:
screenRecordingLoad(filename): Opens file for reading, setsscreenSource=1,playbackStatus=1,isReplayRecord=1- ProtocolHandler Mode 1: Copies
sourceFilepointer to the TcpSocket's file mode (offset 0x44), causing reads to come from the file instead of the network socket screenRecordingStop(): SetsplaybackStatus=2screenRecordingUnload(): Closes file, resetsscreenSource=0
Recording format: The recording is raw RFB protocol data -- the exact bytes
received from the server (FramebufferUpdate messages). This is NOT an AVI or
video format. The Java side (VideoRecorder class using
ch.randelshofer.media.avi.AVIOutputStream) handles conversion to AVI format
for user-facing recording. The native recording is for protocol-level replay.
Huffman_Compress (0x00121f40): Standard Huffman coding implementation:
- Build frequency table from input bytes
- Build Huffman tree by repeatedly combining lowest-frequency nodes
- Generate variable-length codes for each byte value
- Sort codes by frequency (bubble sort)
- Encode input using generated codes Returns compressed size minus start position.
Huffman_Uncompress (0x001221d0): Tree-based decompression:
- Reads the Huffman tree structure from the compressed data header
- For each output byte, traverses the tree bit-by-bit
- Leaf nodes contain the decoded byte values Both are used by the Yarkon decoder for its compressed data, not by the screen recording subsystem directly.
ProtocolHandler jump table:
| Item | Address | Notes |
|---|---|---|
| Jump table base | 0x00127790 | 61 entries, 4 bytes each (244 bytes) |
| Default handler | 0x001183F5 | Returns type byte, 54 entries point here |
| Type 0x00 target | 0x0011849A | Inline in ProtocolHandler |
| Type 0x04 target | 0x001184DE | Inline in ProtocolHandler |
| Type 0x16 target | 0x00118488 | Inline in ProtocolHandler |
| Type 0x35 target | 0x0011844D | Inline in ProtocolHandler |
| Type 0x37 target | 0x00118457 | Inline (falls through from 0x35) |
| Type 0x39 target | 0x00118479 | Inline in ProtocolHandler |
| Type 0x3C target | 0x0011846A | Inline in ProtocolHandler |
Ghidra Analysis Notes
- Both binaries are loaded and analyzed in the Ghidra project
- libiKVM64.so binary name:
/liblinux_x86_64__V1.0.5/libiKVM64.so-730126 - libSharedLibrary64.so binary name:
/liblinux_x86_64__V1.0.5/libSharedLibrary64.so-161223 - C++ symbol demangling works well due to preserved .dynsym entries
- Vtable analysis provides class hierarchy information
- Many Java
.classfiles are also loaded (useful for cross-referencing JNI) - Some duplicate function entries exist (DEFAULT vs IMPORTED sources) at different addresses -- the IMPORTED versions are the actual implementations
- The ProtocolHandler jump table uses PC-relative offsets (each entry is a signed 32-bit offset from the table base at 0x00127790). Ghidra's decompiler could not recover the switch statement ("WARNING: Could not recover jumptable... Too many branches") so manual analysis of the raw bytes was required
- RFBProtocol member offsets (0x50-0x68) are set by the constructors of
RFBMouse, RFBKeyboard, RFBScreen, and RFBPrivilege respectively (each
writes its
thispointer back into the protocol object)
Keyboard Subsystem Analysis
Key finding: Keyboard events are NEVER encrypted. Despite having an
RFBKMCryto* allocated at RFBKeyboard+0x20, the Sendkey() function
never calls any encryption function. Only RFBMouse::SendMouse() uses AES.
RFBKeyboard Object Layout (total size ~0xB8):
Offset Size Type Field Notes
0x00 8 vtable* vtable ptr
0x08 8 RMProtocol* protocol Network stream access
0x10 1 byte config_received Set to 1 by ProcKeyboardInfo
0x14 4 uint keyboard_type From ProcKeyboardInfo
0x18 4 uint keyboard_mode From ProcKeyboardInfo
0x20 8 RFBKMCryto* crypto AES instance (UNUSED by Sendkey)
0x28 48 std::map vk_table_1 Table 1: printable chars (48 entries)
0x58 48 std::map vk_table_2 Table 2: function/control keys (89 entries)
0x88 48 std::map vk_table_3 Table 3: scan code map (32 entries)
processVK Key Translation Data Tables:
Table 1 data: 0x0022FD20 (48 x 8-byte records, 384 bytes) Table 2 data: 0x0022FEA0 (89 x 8-byte records, 712 bytes) Table 3 data: 0x00230180 (32 x 8-byte records, 256 bytes)
Each record is 8 bytes: [value:u32][key:u8, extended_flag:u8, pad:u16]
Table 2 uses the extended_flag byte: if non-zero, 0x100 is added to the key. Table 3 also uses the extended_flag byte in the same way.
Power Action Codes
Confirmed from JNI function decompilation:
| JNI Function | Address | Action Code | Description |
|---|---|---|---|
setPowerOff |
0x00121A70 | 0 | Hard power off |
setPowerOn |
0x00121A50 | 1 | Power on |
setPowerReset |
0x00121AB0 | 2 | Hard reset |
setSoftPowerOff |
0x00121A90 | 3 | ACPI soft off |
getPowerStatus |
0x00121AD0 | (misnamed) | Calls MouseGetPT (returns mouse config) |
All call through vtable: desktop->screen->SetPowerOnOff(action) where
SetPowerOnOff is RFBScreen vtable index 7 (offset 0x38) at 0x00118900.
RFBScreen Object Layout
The RFBScreen object is approximately 0x2080 bytes. Allocated by operator_new(0x2080).
Constructor (RFBScreen::RFBScreen at 0x001185d0):
- Calls
RMScreen::RMScreen(param_1)parent constructor - Sets
frame_number(offset 0x50) = 0 - Stores
thispointer atparam_1 + 0x60(RFBProtocol.screen_handler) - Sets framebuffer ptr (0x38) = 0
- Sets cursor_data_valid (0x55) = 0
- Sets vtable to 0x0022F8D0
- Allocates 0x600000 bytes (6,291,456 = ~6 MB) for compressed_data_buf (stored at 0x40)
- Zeroes out cursor fields at 0x2058-0x206F (24 bytes)
Offset Size Type Field Notes
0x00 8 vtable* vtable ptr 0x0022F8D0
0x08 8 RMProtocol* protocol Set by RMScreen parent ctor
0x10 4 int screen_width Pixels (from ScreenDecode)
0x14 4 int screen_height Pixels (from ScreenDecode)
0x18 4 int stored_width For change detection (ScreenSetInfo)
0x1C 4 int stored_height For change detection (ScreenSetInfo)
0x20 4 uint encoding_type 0x57-0x61 (from ScreenDecode)
0x28 8 uint64 update_blocks Dirty rect tracking
0x38 8 uchar* framebuffer Decoded pixels (BGRX), set via ScreenSetFrameBuff
0x40 8 uchar* compressed_data_buf Incoming encoded data (0x600000 bytes allocated)
0x48 8 uchar* cursor_buffer Cursor Java ByteBuffer, set via ScreenSetFrameBuff
0x50 4 int frame_number Sequence counter (0 initially)
0x54 1 byte resolution_changed Notify flag (set when frame_number==1, first frame)
0x55 1 byte cursor_data_valid Cursor available (set when cursor_type==1 received)
0x56 varies uchar[] cursor_pixel_data Inline cursor image (width*height*2 bytes)
0x2058 4 uint cursor_x Hardware cursor X position
0x205C 4 uint cursor_y Hardware cursor Y position
0x2060 4 int cursor_width Cursor image width
0x2064 4 int cursor_height Cursor image height
0x2070 4 uint cursor_extra Additional cursor info
0x2078 8 RMDecoder* current_decoder Active video decoder (from GetDecoder)
ast_jpeg Decoder State (0x588 bytes)
Offset Size Type Field Notes
0x08 8 uchar* output_buffer 16bpp or 32bpp framebuffer
0x1C 4 int cb_tile_w Cb tile width
0x20 4 int cb_tile_h Cb tile height
0x24 4 int y_tile_w Y tile width
0x28 4 int y_tile_h Y tile height
0x30 8 void* temp_buffer Freed after decode
0x38 4 int advance_mode From header bit 31
0x3C 4 int cbcr_quant Chrominance quant level
0x40 4 int advance_sign Negated if advance_mode set
0x450 4 int (zero init)
0x454 4 int tile_x Current tile X index
0x458 4 int tile_y Current tile Y index
0x45C 4 int prev_tile_x
0x460 4 int prev_tile_y
0x468 8 uint64 decoded_pixels Running count
0x470 8 uint64 (bitstream state)
0x478 8 uint64 codebuf Current code buffer
0x480 8 uint64 newbuf Next buffer
0x48C 4 int newbits Bits remaining
0x49C 4 uint orig_width
0x4A0 4 uint orig_height
0x4BC 4 uint cbcr_tile_size From header
0x518 8 ulong* buffer_ptr Data buffer pointer
0x520 8 uchar* decoded_pixels_buf 32bpp decoded tile data
0x528 8 uint64 buffer_index Current position in buffer
0x530 8 long aligned_width Width aligned to tile boundary
0x538 8 long aligned_height Height aligned to tile boundary
0x540 8 long padded_height
0x548 8 long padded_width
0x550 1 byte (config)
0x551 1 byte Y_DC_selector = 1
0x552 1 byte Y_AC_selector = 1
0x553 1 byte Y_DC_idx = 0
0x554 1 byte Cb_DC_idx = 1
0x555 1 byte Cr_DC_idx = 1
0x556 1 byte Y_AC_idx = 0
0x557 1 byte Cb_AC_idx = 1
0x558 1 byte Cr_AC_idx = 1
0x55A 2 short Y_DC_prev DC coefficient accumulator (Y)
0x55C 2 short Cb_DC_prev DC coefficient accumulator (Cb)
0x55E 2 short Cr_DC_prev DC coefficient accumulator (Cr)
0x564 4 int tile_pixel_size 16 or other
0x568 4 int R multiplier Pixel format
0x56C 4 int G multiplier
0x570 4 int B multiplier
0x574 4 int R shift
0x578 4 int G shift
0x57C 4 int B shift
Additional Discovered Functions
Keyboard subsystem:
| Function | Address | Purpose |
|---|---|---|
RFBKeyboard::Sendkey(int, int) |
0x0011A670 | Send key event (always cleartext) |
RFBKeyboard::processVK(KeyInfo_t) |
0x0011A920 | VK -> X11 keysym translation |
RFBKeyboard::KeyboardAction(KeyInfo_t) |
0x0011A5D0 | Entry: calls processVK via vtable |
RFBKeyboard::KeyboardSync() |
0x0011A5F0 | Stub (returns 1) |
RFBKeyboard::KeyboardReset() |
0x0011A600 | Reset keyboard state |
RFBKeyboard::KeyboardUpdateInfo() |
0x0011A610 | Send type 0x35 request |
RFBKeyboard::KeyboardGetType() |
0x0011A650 | Get keyboard type |
RFBKeyboard::KeyboardSetType(KeyTypeInfo_t) |
0x0011A660 | Set keyboard type |
RFBKeyboard::RFBKeyboard(RMProtocol*) |
0x0011ABF0 / 0x0011AF50 | Constructor (populates maps) |
Java_..._keyboardAction |
0x00121000 | JNI entry: lock key + processVK |
RFBScreen internals:
| Function | Address | Purpose |
|---|---|---|
RFBScreen::ScreenDecode() |
0x00119090 | Receive + decode FramebufferUpdate |
RFBScreen::ScreenUpdate(ScreenReqInfo_t) |
0x00118850 | Send FramebufferUpdateRequest |
RFBScreen::ScreenGetFreame() |
0x00118D40 | Return framebuffer pointer |
RFBScreen::ScreenGetRect(ScreenReqInfo_t, uchar*) |
0x00118A20 | Copy rect from framebuffer |
RFBScreen::ScreenSetFrameBuff(uchar*, uchar*) |
0x00118D50 | Set frame/cursor buffer ptrs |
RFBScreen::ScreenSetInfo(ScreenInfo_t) |
0x00118B80 | Send resolution (type 0x32) |
RFBScreen::ScreenCursorPosProc() |
0x00118FB0 | Handle cursor position (type 0x04) |
RFBScreen::GeFrontGround() |
0x00118DA0 | Get cursor composite buffer |
RFBScreen::GetCursorPos() |
0x00118D60 | Send type 0x19 (request cursor) |
RFBScreen::ScreenGetInfo() |
0x00118C10 | Get current screen info struct |
Video decoder chain:
| Function | Address | Purpose |
|---|---|---|
RMDecoder::GetDecoder(Decodeinfo, uchar*, uchar*) |
0x00112560 | Decoder factory (singleton) |
ASTVideoDecoder::Decode(UpdateBlocks_t&) |
0x0010F8B0 | AST decode entry (dispatches to ast_jpeg or ast2100) |
ASTVideoDecoder::GetDecodedFrame() |
0x0010F8A0 | Return decoded frame pointer |
ast_jpeg::decode(ASTDecodeInfo&, UpdateBlocks_t&) |
0x0011EA70 | AST2100+ JPEG decode |
ast_jpeg::SetOptions(ASTDecodeInfo&) |
0x0011E280 | Configure from frame header |
ast_jpeg::YUVToRGB(...) |
0x0011E4E0 | YCbCr -> BGRX conversion |
ast_jpeg::MoveBlockIndex() |
0x0011E700 | Advance tile X/Y index |
ast_jpeg::ScreenResolution() |
0x0011E770 | Write decoded tile to output buffer |
ast_jpeg::updatereadbuf(...) |
0x0010E048 | Advance bitstream read pointer |
ast_jpeg::setScreenInfo(...) |
0x0011F900 | Configure screen parameters |
ast_jpeg::get_buffer() |
0x0011F8C0 | Get compressed data buffer |
ast_jpeg::get_ImageData() |
0x0011F8D0 | Get decoded image data |
ast_jpeg::get_decoderID() |
0x0011F990 | Get decoder ID |
ast_jpeg::PsudoStreamSwap16(ushort) |
0x0011F940 | 16-bit byte swap |
ast_jpeg::PsudoStreamSwap32(uint) |
0x0011F960 | 32-bit byte swap |
ast_jpeg::flag |
0x00230FB0 | Static: init_jpg_table needed flag |
AES encryption:
| Function | Address | Purpose |
|---|---|---|
RFBKMCryto::EnCryto(char*, char*, int) |
0x00117050 | Encrypt wrapper |
RFBKMCryto::DeCryto(...) |
0x001149C0 | Decrypt wrapper |
RFBKMCryto::RFB_AES128_EventCryto(uchar*, uchar*) |
0x00116E30 | AES-128-CBC event encrypt |
RFBKMCryto::SW_AES_CBC(mode, keysize, input, blocks, key, output, iv) |
0x0010E2E8 | Software AES-CBC |
RFBKMCryto::aes_encrypt(aes_context*, uchar*, uchar*) |
0x00114F20 | AES block encrypt |
RFBKMCryto::aes_set_key(aes_context*, uchar*, int) |
0x0010E278 | Set key (128/192/256) |
RFBKMCryto::aes_gen_tables() |
0x001149D0 | Generate S-boxes |
RFBKMCryto::SetCipherKey(...) |
0x0011E250 | Configure cipher key |
RFBKMCryto::RSb |
0x001225E0 | Reverse S-box table |
Power control:
| Function | Address | Action Code | Purpose |
|---|---|---|---|
setPowerOn |
0x00121A50 | 1 | Power on |
setPowerOff |
0x00121A70 | 0 | Hard power off |
setSoftPowerOff |
0x00121A90 | 3 | ACPI soft power off |
setPowerReset |
0x00121AB0 | 2 | Hard reset |
getPowerStatus |
0x00121AD0 | -- | Misnamed: calls MouseGetPT() (returns mouse config) |
RFBScreen::SetPowerOnOff(uchar) |
0x00118900 | -- | Send type 0x1A + action byte |