Final review pass cross-referencing server RE with client protocol spec and client RE. Key corrections and findings: - Type 0x07 is MouseSync/ReSync, NOT ClientCutText — dispatches through DeviceManager to MouseDevice::ReSync ioctl - Differential frame tile format includes 4 bytes padding (uninitialized stack data from CopyTileData) - Differential-to-full fallback when diff data exceeds frame size - Hermon sub-header byte 1 is color depth flag (0=RGB555, else 8bpp) - ServerInit firmware bug: width/height are sent in reversed order - Type 0x35 firmware bug: server sends 3 bytes but client expects 6 (missing mouse info bytes) - KeepAlive broadcast dispatches as type 0x39 PrivilegeInfo with hi=7 for LED sync - libutility.so function signatures: UtilAuthUser returns 0-9 on success (>=10 failure), UtilGetNowPowerStatus returns u8 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
119 KiB
Supermicro X9 BMC Firmware Reverse Engineering Notes
Firmware: SMT_X9_339.bin
BMC SoC: Nuvoton WPCM450 ("Hermon")
Architecture: ARM 32-bit Little Endian
Kernel: Linux 2.6.17.WB_WPCM450.1.3
Compiler: GCC 3.4.4
Extraction
Firmware extracted to original_binaries/server/180000/ — this is
the root filesystem. Web content at original_binaries/server/B80000/.
Key Binaries
ikvmserver — KVM/VNC Server
- Path:
original_binaries/server/180000/bin/ikvmserver - Ghidra name:
/180000/bin/ikvmserver-bd7cb9 - Type: ARM 32-bit LE ELF executable, stripped, C++ with virtual dispatch
- Functions: 471
- Default port: 5900 (configurable via persistent storage)
- Init script:
/etc/init.d/ikvmd
ikvmserver Function Map
| Address | Name / Purpose | Notes |
|---|---|---|
| 0x0000a1e0 | entry |
Calls __libc_start_main with main at 0xa2d4 |
| 0x0000a2f0 | Preview thread handler | Triggered by SIGUSR2, spawns 0xa33c |
| 0x0000a33c | Preview capture thread | Captures frame, saves /tmp/Snapshot.bmp |
| 0x0000a4fc | KillAllSessions | Kills all if >1 session, sends cmd=6 "Change Port" |
| 0x0000a9c0 | ResetIPHandler | Prints "ResetIP_Handler", cleans up, exit(0) |
| 0x0000a9dc | SIGUSR1 handler (session kill) | Reads /tmp/killsess: -1=all, -2=reset, else by username |
| 0x0000acbc | Main accept loop | Spawns per-connection threads |
| 0x0000aea8 | Global init (device manager) | Creates DeviceManager at DAT_0001e560 |
| 0x0000b10c | CancelThread(pthread_t*) | pthread_cancel + pthread_detach |
| 0x0000b140 | SetCancelState | PTHREAD_CANCEL_ENABLE + ASYNCHRONOUS |
| 0x0000b160 | Init recursive mutex | pthread_mutexattr_settype(RECURSIVE) |
| 0x0000b1b0 | TriggerPortChange | Writes "-1" to /tmp/killsess, sends SIGUSR1 to self |
| 0x0000b254 | Read IKVM port from PS | Default 5900 if empty |
| 0x0000b2bc | Install SIGUSR1 handler | sigaction(10=SIGUSR1, handler) |
| 0x0000b2e8 | (nop) | Empty function, called before signal setup |
| 0x0000b2ec | Install SIGUSR2 handler | sigaction(12=SIGUSR2, handler) |
| 0x0000b318 | SaveMouseMode(mode) | Writes mode to /nv/MouseMode |
| 0x0000b63c | DeviceManager constructor | Allocates VCD, Keyboard, Mouse sub-objects |
| 0x0000b7b4 | DeviceManager::~dtor | |
| 0x0000b7e4 | DeviceManager::~dtor (deleting) | |
| 0x0000b824 | DeviceManager::SetCursorShape | Delegates to MouseDevice vtable+0x08 |
| 0x0000b860 | DeviceManager::SendKeyEvent | Delegates to KeyboardDevice vtable+0x1c |
| 0x0000b880 | DeviceManager::GetMouseCursorPos | Delegates to MouseDevice vtable+0x14 |
| 0x0000b8b8 | DeviceManager::SendMouseEvent | Delegates to MouseDevice vtable+0x1c |
| 0x0000b8d4 | DeviceManager::SetMouseMode | Delegates to MouseDevice vtable+0x1c |
| 0x0000b8f0 | DeviceManager::SendKeyboardEvent | Delegates to KeyboardDevice vtable+0x1c |
| 0x0000b90c | DeviceManager::GetKeyboardInfo | Delegates to KeyboardDevice vtable+0x14 |
| 0x0000b948 | DeviceManager::CaptureFrame | Delegates to VideoCaptureDevice vtable+0x08 |
| 0x0000b9c0 | DeviceManager::SaveSnapshot | Delegates to VideoCaptureDevice vtable+0x28 |
| 0x0000ba3c | DeviceManager::ReleaseCapture | Delegates to VideoCaptureDevice vtable+0x0c |
| 0x0000ba78 | DeviceManager::CaptureForClient | Delegates to VideoCaptureDevice vtable+0x10 |
| 0x0000baf0 | DeviceManager::VideoControl | Delegates to VideoCaptureDevice vtable+0x1c |
| 0x0000bb60 | DeviceManager::GetVideoInfo | Delegates to VideoCaptureDevice vtable+0x20 |
| 0x0000bb84 | DeviceManager::UnlockCapture | Delegates to VideoCaptureDevice vtable+0x24 |
| 0x0000bd08 | VideoCaptureDevice constructor | Opens /dev/vcd, inits VCD hw, allocs frame buf |
| 0x0000befc | VideoCaptureDevice::RemoveSession | Clears session slot from slot table |
| 0x0000bf58 | VideoCaptureDevice::UnlockCapture | Calls RemoveSession, unlocks mutex |
| 0x0000bf70 | VideoCaptureDevice::SaveSnapshot | Writes /tmp/Snapshot.bmp (RGB555 BMP) |
| 0x0000c134 | FindActiveSlot | Scans 4 capture slots for active one |
| 0x0000c168 | BuildNoSignalFrame | Returns encoding 0x59, negative w/h, len=0 |
| 0x0000c198 | FindSlotBySession | Looks up capture slot by session ID |
| 0x0000c1d0 | AllocateSlot | Assigns session to free capture slot |
| 0x0000c220 | BuildEmptyFrame | Returns encoding 0x59, 1x1, len=10 |
| 0x0000c240 | GetResolution | Reads w/h from VCD, caps to 1280×1024 |
| 0x0000c2ac | BuildFullFrame | Full-frame Hermon packet with 10-byte header |
| 0x0000c404 | CopyTileData | Copies 16×16 tile from VCD framebuffer |
| 0x0000c4c4 | BuildDifferentialFrame | Tile-based dirty-rect Hermon packet |
| 0x0000c72c | RetryCapture | Retries WaitForValidResolution 3×, returns no-signal |
| 0x0000c770 | VideoCaptureDevice::CaptureFrame | Main capture entry point |
| 0x0000cc08 | KeyboardDevice constructor | Opens /dev/keyboard, vtable at 0x14674 |
| 0x0000cd04 | KeyboardDevice::SendKeyEvent | Writes 8-byte key event to /dev/keyboard |
| 0x0000cdac | KeyboardDevice::stub_0c | Returns 1 (unused) |
| 0x0000cdb4 | KeyboardDevice::stub_10 | Returns 1 (unused) |
| 0x0000cdbc | KeyboardDevice::GetKeyboardInfo | Copies 2 words from +0x14,+0x18 to output |
| 0x0000cdd4 | KeyboardDevice::SetKeyboardInfo | Stores 2 words into +0x14,+0x18 |
| 0x0000cde4 | KeyboardDevice::GetKbLed | ioctl ATEN_KEYBOARD_IOCGETKBLED (0x80046205) |
| 0x0000ce5c | InitConnectionDescriptor | Stores socket fd, DeviceManager ptr, clears state |
| 0x0000ce74 | NotifyPeersSessionEvent | Sends cmd to peers with ≥ this session's privilege |
| 0x0000cf58 | NotifyPeersStatusUpdate | Sends own status to peers with ≥ own privilege |
| 0x0000d0c8 | CleanupRFBProtocol(rfb_proto) | Removes session from list, notifies peers, deletes Auth/Stream |
| 0x0000d194 | CleanupRFBProtocol2(rfb_proto) | Variant cleanup: removes from list, notifies, destroys Stream |
| 0x0000d050 | SendMessageToSession | Finds session by thread ID, sends mq message |
| 0x0000d260 | EnforceMaxSessions | Kills oldest session if >4 active sessions |
| 0x0000d334 | Per-session handler (main thread fn) | InitHandShake → Authenticate → ProtocolHandler |
| 0x0000d600 | GetPrivInfoPtr(session_ctx) | Returns session_ctx+0x4c (privilege info) |
| 0x0000d608 | CleanupSessionMQ(session_ctx) | Deletes priv info, closes/unlinks mq |
| 0x0000d6e0 | ReceiveFromMQ(session_ctx, buf) | mq_receive into buffer |
| 0x0000d740 | IsKbdMouseExclusive | Returns 0 if any session has kbd/mouse perm |
| 0x0000d800 | GetAcceptThreadId | Finds own session node by pthread_self(), returns thread_id at +0x0c |
| 0x0000d810 | FindSessionNode(thread_id) | Iterates linked list for matching thread ID |
| 0x0000d870 | GetSessionInfo(out, thread_id) | Copies 0x48-byte session info from linked list |
| 0x0000d8e8 | BroadcastMessage(session, target_tid, msg) | Looks up target, sends via mq_send |
| 0x0000d950 | KillSessionByThreadId(list, tid) | Gets session info, sends mq msg, signals thread |
| 0x0000d9d0 | InitSessionContext(ctx, accept_cb, tid) | Allocs 0x164-byte priv struct, creates mq, adds to list |
| 0x0000daa8 | InitSessionContext2(ctx, param, tid) | Variant of InitSessionContext used in accept loop |
| 0x0000dba0 | InitSessionList (global ctor) | Initializes DAT_0001e58c/DAT_0001e590 circular list |
| 0x0000dbd8 | FreeLinkedList2 | Walks list, operator_delete each node |
| 0x0000dc04 | (nop) | Empty function |
| 0x0000dc10 | SharedMemoryInit | Attaches GlobalVar/NVRAM/PS shm, registers PID |
| 0x0000dd28 | SetPermissions(username, priv, level) | Sets 4 permission bytes based on IPMI privilege |
| 0x0000ddf4 | GetVideoPermission(priv_info) | Returns byte at +0x155 |
| 0x0000de04 | GetKbdMousePermission(priv_info) | Returns byte at +0x156 |
| 0x0000de14 | GetKickPermission(priv_info) | Returns byte at +0x157 |
| 0x0000de24 | GetPowerPermission(priv_info) | Returns byte at +0x158 |
| 0x0000de30 | DecrementSessionCount(username) | Decrements /tmp/sess_ counter, logs to SEL |
| 0x0000e1b4 | LoadViewerLang(priv_info) | Reads /nv/IKVMViewerLang for user, stores +0x15c/+0x160 |
| 0x0000e2f0 | AuthenticateFromWebSession | Reads /tmp/sess_, SSO auth, tracks session count |
| 0x0000e638 | AuthBackend | Tries web session SSO first, falls back to UtilAuthUser |
| 0x0000e710 | Get power status wrapper | Calls UtilGetNowPowerStatus() |
| 0x0000e81c | MouseDevice constructor | Opens /dev/mouse, vtable at 0x14a5c |
| 0x0000e920 | MouseDevice::SendMouseEvent | Writes 20-byte mouse event to /dev/mouse |
| 0x0000e9b0 | MouseDevice::ReSync | ioctl ATEN_MOUSE_IOCRESYNC (0x40046204) |
| 0x0000e9d8 | MouseDevice::HotPlug | ioctl ATEN_MOUSE_IOCHOTPLUG (0x40046201) |
| 0x0000ea00 | MouseDevice::GetCursorPos | ioctl ATEN_MOUSE_IOCGETMODE (0x80046202), returns pos |
| 0x0000ea40 | MouseDevice::SetMouseMode | Sets mode (1=abs, 2=rel, 3=passthrough), ioctls, saves /nv/MouseMode |
| 0x0000eae0 | MouseDevice::DoHotPlug | ioctl ATEN_MOUSE_IOCHOTPLUG (0x40046201) |
| 0x0000eb3c | Handler: type 0x08 SetEncoding | Reads 1B mouse_mode, calls SetMouseMode(mode,0) |
| 0x0000eb78 | Handler: type 0x34 SetEncoding2 | Reads 2B, calls DevMgr vtable+0x20 (nop stub) |
| 0x0000ebcc | Handler: type 0x36 SetEncoding3 | Reads 2B, calls SetMouseMode(mode, extra) |
| 0x0000ec20 | Handler: type 0x3a RequestRefresh | Calls DoHotPlug (USB mouse re-enumerate) |
| 0x0000ec40 | GetFilterLen (payload sizes for skip) | Returns bytes to skip for filtered msgs |
| 0x0000eb28 | ReadSecurityResult | Reads 1 byte from client (security handshake ack) |
| 0x0000ed88 | VersionExchange | Sends "RFB 003.008\n", reads client version |
| 0x0000ee0c | Handler: type 0x3e GetSessionStatus | Sends back type 0x3E |
| 0x0000ee5c | Handler: type 0x3c GetViewerLang | Sends back type 0x3C |
| 0x0000eeac | Handler: type 0x35 GetKbdMouseInfo | Sends LED status + mouse mode (3 bytes) |
| 0x0000ef10 | Handler: type 0x16 KeepAlive ACK | Echoes back type 0x16 + status byte |
| 0x0000ef58 | Handler: type 0x19 GetCursorPos | CaptureForClient + cursor data as type 0x04 |
| 0x0000f058 | SecurityTypeNegotiation | Sends count=1, type=0x10 (VNC auth), reads client choice |
| 0x0000f098 | InitHandShake (vtable entry) | Calls VersionExchange then SecurityTypeNegotiation |
| 0x0000f0cc | Handler: type 0x33 GetVideoInfo | Sends back type 0x33 |
| 0x0000f130 | PermissionFilter | Returns true (deny) if session lacks privilege for msg type |
| 0x0000f1ec | SendServerInit | Standard RFB ServerInit + ATEN extension fields |
| 0x0000f37c | Authenticate (vtable entry) | Challenge/response + auth backend + ServerInit |
| 0x0000f44c | Handler: type 0x32 VideoControl3 | Reads 2×u16, calls DevMgr::VideoControl |
| 0x0000f4dc | Handler: type 0x18 VideoControl2 | Reads u16, calls DevMgr::VideoControl(-1, val) |
| 0x0000f55c | Handler: type 0x14 VideoControl | Reads u16, calls DevMgr::VideoControl(val, -1) |
| 0x0000f5dc | Handler: type 0x07 MouseSync/ReSync | Reads u16, calls DevMgr vtable+0x0c → MouseDevice::ReSync |
| 0x0000f610 | Handler: type 0x03 FBUpdateRequest | Captures video, sends type 0x00 |
| 0x0000f7a8 | SkipFilteredPayload | Calls StreamSkipBytes for denied messages |
| 0x0000f7b0 | Handler: type 0x15 (2×u32, ignored) | Reads 2×u32 BE, discards both |
| 0x0000f7d0 | Handler: type 0x05 PointerEvent | AES decrypt, then DevMgr::SetCursorShape(x,y,btns) |
| 0x0000f8c4 | Handler: type 0x04 KeyEvent | AES decrypt, then DevMgr::SendKeyboardEvent(key,down) |
| 0x0000f994 | Handler: type 0x37 GetSessionInfo | Sends back type 0x37 + power status |
| 0x0000fa10 | Handler: type 0x3b SetBandwidth | 3×u32 bandwidth params |
| 0x0000fa54 | Handler: type 0x38 SendKickRequest | Reads target_tid(u32), cmd(u32), reason(64B), forwards via mq |
| 0x0000faa4 | Handler: type 0x3d SetViewerLang | Saves to /nv/IKVMViewerLang |
| 0x0000fae4 | Handler: type 0x1a PowerControl | 0=off, 1=on, 2=reset, 3=soft off |
| 0x0000fb70 | KeepAlive broadcast to peers | Sends power status change to all sessions via mq |
| 0x000108b8 | ProtocolHandler (main dispatch) | Queue processing → read type → permission check → dispatch |
| 0x0001069c | ProcessCommandQueue | Drains mq + pending list, sends type 0x39 or throws disconnect |
| 0x00010b34 | ReceiveAndEnqueueMQ | mq_receive, enqueue to pending list or throw exception |
| 0x00010bec | FreeLinkedList3 | Walks list, operator_delete each node (9 refs) |
| 0x00010244 | RFBProtocol constructor | 0x28-byte object with vtable at 0x14b00 |
| 0x00010e7c | Auth constructor | Sets vtable to 0x14c64 (Auth base) |
| 0x00010fa8 | AESCrypto constructor | Calls aes_gen_tables, sets vtable to 0x14cb4 |
| 0x000110c0 | AES CBC multi-block wrapper | param2: 0=encrypt, 1=decrypt; param5=block_count |
| 0x00011020 | AES multi-block ECB (unused) | Encrypt/decrypt without CBC, unreferenced |
| 0x00011268 | AES-128-CBC decrypt (hardcoded key) | Key at 0x14cf0, IV at 0x14ce0 |
| 0x000112d8 | AESCrypto::Decrypt (vtable thunk) | Calls 0x11268, returns 1 |
| 0x000112e8 | AES-128-CBC encrypt (hardcoded key) | Same key/IV |
| 0x00011368 | VCD: WaitForVSync | Polls VCD status register for vsync |
| 0x00011548 | VCD: SetLineStride | Rounds up to power-of-2, writes to reg+0x08 |
| 0x0001152c | VCD: GetCaptureReady | Checks bit 15 of 0xF001003c |
| 0x000115a4 | VCD: GetBytesPerPixel | Mode 0→2(RGB555), 0x10→1, 0x20→1, 0x30→4 |
| 0x00011718 | VCD: GetHorizontalRes | Reads from 0xF0030310/0xF0030314 |
| 0x0001178c | VCD: GetVerticalRes | Reads from 0xF0030320/0xF0030324 |
| 0x0001184c | VCD: GetPixelClock | Computes from 0xF0030340..0xF0030348 |
| 0x000118bc | VCD: SetCaptureResolution | Packs w/h into reg+0x0c, max 1280×1024 |
| 0x00011948 | VCD: SetCaptureMode | Writes mode bits to reg+0x14 bits[18:16] |
| 0x00011988 | VCD: SetColorDepth | Writes to reg+0x5c bits[14:8] |
| 0x000119cc | VCD: SetDifferentialMode | Toggles bit 11 in reg+0x14 |
| 0x00011a24 | VCD: SetHorDeciRatio | Horizontal decimation in reg+0x5c[3:0] |
| 0x00011a68 | VCD: SetVerDeciRatio | Vertical decimation in reg+0x5c[7:4] |
| 0x00011aac | VCD: SetTileSize | Tile size in reg+0x5c[31:28] |
| 0x00011aec | VCD: ResetCapture | Triggers capture cycle via reg+0x18 |
| 0x00011bfc | VCD: Init | Opens /dev/vcd, validates hardware, configures registers |
| 0x00011da0 | VCD: SetFrameBufferAddr | Writes address to reg+0x00, offset to reg+0x04 |
| 0x00011df0 | VCD: VerifyLineStride (unused) | Writes reg+0x08, verifies read-back |
| 0x00011e64 | VCD: VerifyCaptureWindow (unused) | Writes reg+0x10, verifies [10:0] and [26:16] |
| 0x00011ed8 | VCD: SetSourceSelect | Selects video input (0-4) via reg+0x14 |
| 0x00011f6c | VCD: SetPixelFormat | 0=2bpp(RGB555), 1=1bpp(8bit), 2=1bpp |
| 0x000120d4 | VCD: WaitForCaptureDone | select()+read() on /dev/vcd fd, 100ms timeout |
| 0x0001209c | VCD: IsCaptureComplete | Checks bits 0,3 in reg+0x1c |
| 0x000121b4 | VCD: TriggerFullCapture | Sets reg+0x18=0, triggers, waits for done |
| 0x0001219c | VCD: TriggerDiffCapture | Sets reg+0x18=0x20, triggers diff capture |
| 0x000121c0 | VCD: VerifyCaptureRes | Checks captured resolution matches expected |
| 0x000124d4 | VCD: IsVideoPresent | True if any VGA sync signals detected |
| 0x00012524 | VCD: WaitForValidResolution | Detects video mode, prints "[Hermon]" status |
| 0x00012808 | VCD: GetResolutionInfoPtr | Returns DAT_0001e594 (resolution struct) |
| 0x00012830 | VCD: GetLineStride | Returns low 16 bits of reg+0x08 |
| 0x000127ec | VCD: IsDisplayOff | True if DAT_f0030300 == 0x86 |
| 0x0001221c | VCD: SetColorLookup (unused) | Palette→RGB555 conversion, writes reg+0x24 |
| 0x00012794 | VCD: PrintStatus (unused/debug) | Prints vcd_stat register value |
| 0x00012a6c | VCD: OpenDevice | open("/dev/vcd", O_RDONLY), singleton |
| 0x00012dc8 | StreamReadBytes(stream, buf, len) | |
| 0x00012e18 | StreamRead8(stream) → u8 | |
| 0x00012e38 | StreamRead16BE(stream) → u16 | Big-endian |
| 0x00012e68 | StreamRead32BE(stream) → u32 | Big-endian |
| 0x00012ea8 | StreamSkipBytes(stream, count) | Alloc+read+free (wasteful) |
| 0x00012eec | StreamWriteBytes(stream, buf, len) | Buffered, flushes at 0x5F0 bytes |
| 0x00012f94 | StreamWrite32BE(stream, val) | Big-endian |
| 0x00012fd8 | StreamWrite16BE(stream, val) | Big-endian |
| 0x0001300c | StreamWrite8(stream, val) | |
| 0x00013038 | StreamFlush(stream) | Also unlocks stream mutex |
| 0x0001308c | StreamWriteZeros(stream, count) | Alloc+memset+write+free |
| 0x000130f4 | ByteSwap16 (for decrypted AES data) | |
| 0x00013128 | Set bandwidth limits on stream | |
| 0x00013180 | InstallCrashSignalHandlers | Installs SIG_IGN for SIGPIPE, handlers for 15 signals |
| 0x00012c90 | Stream constructor (0x610 bytes) | |
| 0x00012d88 | Stream::Disconnect(stream) | Calls transport vtable dtor via +0x60c |
| 0x00012da8 | Stream::Connect(stream) | Calls transport vtable connect via +0x60c |
| 0x000132f0 | Socket recv wrapper | select() + recv, 60s timeout |
| 0x00013464 | Socket send wrapper | select() + send, 60s timeout, MSG_NOSIGNAL |
| 0x00013628 | Accept wrapper | select() + accept |
| 0x000136cc | Socket setup (bind/listen/connect) | SO_SNDBUF/SO_RCVBUF = 0x40000 (256KB) |
| 0x000135dc | (unused socket utility) | Socket-related helper, 0 references |
| 0x00013944 | Connection establishment | getaddrinfo + server/client mode |
| 0x000139f4 | CloseConnection(stream) | shutdown+close socket, manages ref count at DAT_0001e5a8 |
| 0x00013d70 | TimeDiffMicros | Computes time difference in microseconds |
| 0x00013db0 | MicrosToTimeval | Converts microseconds to {tv_sec, tv_usec} |
| 0x00013dec | BandwidthCounterReset | Zeros counter fields |
| 0x00013e00 | BandwidthCounterInit(obj, limit, period) | Initializes bandwidth counter object |
| 0x00013e1c | BandwidthCounterUpdate | Updates byte/time counters |
| 0x00013e38 | BandwidthCounterApply | Used by stream write to enforce limits |
| 0x00013f38 | BandwidthThrottle(counter, bytes) | Sleeps via select() to enforce bandwidth limit |
| 0x0000a2d4 | main() | Calls CreateServerSocket then MainAcceptLoop |
| 0x0000a2ec | SharedMemoryInit (thunk to 0xdc10) | GlobalVar/NVRAM/PS shared memory attach, registers PID |
| 0x0000abdc | KillAllSessionsInner | Iterates pending list, cancels threads, frees resources |
| 0x0000af04 | FreeLinkedList | Walks std::list, operator_delete on each node |
| 0x0000af30 | RemoveListNode | std::_List_node_base::unhook + operator_delete |
| 0x0000af54 | SpawnDetachedThread | pthread_create + pthread_detach wrapper |
| 0x0000af7c | (nop) | Empty function |
| 0x0000af84 | InstallSignalHandler(signum, handler) | sigaction wrapper (no SA_RESTART) |
| 0x0000afac | ThreadExit | pthread_exit(NULL) |
| 0x0000afb8 | CreateMessageQueue | mq_open(name, O_CREAT|O_RDWR|O_EXCL, 0666) |
| 0x0000b04c | DestroyMessageQueue | mq_close + mq_unlink |
| 0x0000b074 | MQSend | mq_send wrapper |
| 0x0000b0c4 | MQReceive | mq_receive wrapper, gets max msg size from attrs |
| 0x0000b1a0 | SleepMs(ms) | usleep(ms * 1000) |
ikvmserver Data Addresses
| Address | Size | Content |
|---|---|---|
| 0x00014518 | 0x48 | DeviceManager vtable (18 entries) |
| 0x00014578 | 0x2c | VideoCaptureDevice vtable (11 entries) |
| 0x00014674 | 0x20 | KeyboardDevice vtable (8 entries) |
| 0x00014a5c | 0x20 | MouseDevice vtable (8 entries) |
| 0x00014b00 | 0x14 | RFBProtocol vtable (derived) |
| 0x00014b50 | 0x14 | Protocol vtable (base, pure virtual) |
| 0x00014c64 | 0x0c | Auth vtable (3 entries: dtor, dtor2, Authenticate) |
| 0x00014ca0 | 0x0c | RFBAuth vtable (3 entries: dtor, dtor2, Authenticate=pure_virtual) |
| 0x00014cb4 | 0x18 | AESCrypto vtable (6 entries, +0x10=Decrypt thunk) |
| 0x00014ce0 | 0x10 | AES IV: 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f |
| 0x00014cf0 | 0x60 | AES key material (96 bytes, NIST test vectors) |
First 16 bytes used: 2b 7e 15 16 28 ae d2 a6 ab f7 15 88 09 cf 4f 3c |
||
| 0x00014e40 | 0x18 | Stream vtable |
| 0x0001d524 | var | BMP file header template (for /tmp/Snapshot.bmp) |
| 0x0001e4ac | 0x04 | ikvmSessCount — current IKVM session count |
| 0x0001e4b0 | 0x04 | VCD register block pointer (mmap'd hardware) |
| 0x0001e4e8 | 0x04 | VCD file descriptor (/dev/vcd) |
| 0x0001e4f0 | 0x04 | at_p_St_GlobalVar — GlobalVar shared memory pointer |
| 0x0001e4f4 | 0x04 | at_p_St_PS — Platform-Specific shared memory pointer |
| 0x0001e560 | 0x18 | DeviceManager global instance (6 words) |
| 0x0001e578 | 0x08 | Accept loop: pending connection linked list (std::list) |
| 0x0001e580 | 0x04 | Global: server socket object pointer |
| 0x0001e588 | 0x01 | Global: cached power status byte (for change detection) |
| 0x0001e58c | 0x08 | Global: active session linked list head (std::list, circular) |
| 0x0001e594 | 0x04 | VCD resolution info struct pointer |
| 0x0001e59c | 0x04 | VCD initialized flag (singleton guard) |
| 0x0001e5a0 | 0x04 | VCD hardware revision (from 0xF0010000 >> 24) |
| 0x0001e5a4 | 0x04 | VCD device opened flag (singleton guard) |
| 0x0001e5a8 | 0x04 | Server connection reference count |
| 0x0001e5ac | 0x04 | Server listen socket file descriptor |
ikvmserver RTTI Strings
| String | Class |
|---|---|
12Stream |
Stream (buffered socket I/O) |
8Protocol |
Protocol (base, pure virtual) |
11RFBProtocol |
RFBProtocol (derived) |
4Auth |
Auth (base) |
7RFBAuth |
RFBAuth (derived) |
8AESCrypto |
AESCrypto (base) |
38AESCryto |
AESCryto [sic] (derived, note typo) |
ikvmserver Object Layouts
RFBProtocol (0x28 bytes, vtable at 0x14b00):
| Offset | Field |
|---|---|
| +0x00 | vtable ptr → 0x14b00 |
| +0x04 | Pointer to DeviceManager global (→ &DAT_0001e560) |
| +0x08 | Stream* (0x610-byte object) |
| +0x0c | Auth* (4-byte object, vtable at 0x14c64, Authenticate thunk → AuthBackend) |
| +0x10 | SessionContext* (0x50-byte object, see below) |
| +0x14 | Pending command linked list head (std::list of queued mq commands) |
| +0x1c | RFB major version (parsed from client) |
| +0x20 | RFB minor version (parsed from client) |
| +0x24 | AESCrypto* (0x104-byte object, vtable at 0x14cb4) |
DeviceManager (0x18 bytes, global at 0x1e560):
| Offset | Field |
|---|---|
| +0x00 | vtable (0x14518) |
| +0x04 | Width (default 640 = 0x280) |
| +0x08 | Height (default 480 = 0x1e0) |
| +0x0c | MouseDevice* (0x1c bytes, opens /dev/mouse) |
| +0x10 | KeyboardDevice* (0x1c bytes, opens /dev/keyboard) |
| +0x14 | VideoCaptureDevice* (0x84 bytes, opens /dev/vcd) |
KeyboardDevice (0x1c bytes):
| Offset | Field |
|---|---|
| +0x00 | vtable (0x14674) |
| +0x04 | (padding/unused) |
| +0x08 | (padding/unused) |
| +0x0c | Back-pointer to DeviceManager |
| +0x10 | File descriptor for /dev/keyboard (-1 before open) |
| +0x14 | Keyboard info word 1 (returned by GetKeyboardInfo) |
| +0x18 | Keyboard info word 2 (returned by GetKeyboardInfo) |
MouseDevice (0x1c bytes):
| Offset | Field |
|---|---|
| +0x00 | vtable (0x14a5c) |
| +0x04 | (padding/unused) |
| +0x08 | (padding/unused) |
| +0x0c | Back-pointer to DeviceManager |
| +0x10 | File descriptor for /dev/mouse (-1 before open) |
| +0x14 | Mouse mode (1=absolute, 2=relative+scroll, 3=passthrough) |
| +0x18 | Mouse extra param (set by SetMouseMode param_3) |
VideoCaptureDevice (0x84 bytes):
| Offset | Field |
|---|---|
| +0x00 | vtable (0x14578) |
| +0x04..+0x20 | (internal state) |
| +0x24 | Back-pointer to DeviceManager |
| +0x28 | pthread_mutex_t (recursive, protects capture) |
| +0x40 | Current resolution info pointer |
| +0x44 | Frame data buffer pointer (malloc'd ~7.5MB) |
| +0x48 | Encoding type (0x59) |
| +0x4c | Bytes per pixel (1 or 2) |
| +0x50..+0x7f | Capture slot table (4 slots × 0x0c bytes each) |
| +0x80 | Active session count |
Capture Slot (0x0c bytes, 4 entries at VideoCaptureDevice+0x50):
| Offset | Field |
|---|---|
| +0x00 | Session ID (0 = free) |
| +0x04 | Mode/flags |
| +0x08 | Valid flag (1 = has captured frame) |
CaptureResult (0x20 bytes, returned by CaptureFrame on stack):
| Offset | Field |
|---|---|
| +0x00 | (unused) |
| +0x04 | Encoding type (always 0x59) |
| +0x08 | Width (pixels, or negative for no-signal) |
| +0x0c | Height (pixels, or negative for no-signal) |
| +0x10 | (unused) |
| +0x14 | (unused) |
| +0x18 | Subrects info |
| +0x1c | Data length (bytes) |
Stream (0x610 bytes):
| Offset | Field |
|---|---|
| +0x00 | pthread_mutex_t (locked before write, unlocked by Flush) |
| +0x18 | Write buffer (0x5F0 bytes) |
| +0x3c | Write bandwidth counter object |
| +0x40 | Read bandwidth counter object |
| +0x44 | Socket file descriptor |
| +0x48 | Server mode flag |
| +0x608 | Write buffer current position pointer |
| +0x60c | Transport vtable pointer (for actual send/recv) |
SessionContext (0x50 bytes, allocated in InitSessionContext at 0xd9d0):
| Offset | Field |
|---|---|
| +0x00 | Thread ID (pthread_self()) |
| +0x04 | Message queue descriptor (mqd_t from mq_open) |
| +0x08..+0x47 | Message queue name string (64 bytes, format "/%x" with thread ID) |
| +0x48 | mq_attr.mq_maxmsg (0x104 = 260) |
| +0x4c | PrivilegeInfo* (pointer to 0x164-byte struct, see below) |
PrivilegeInfo (0x164 bytes, allocated in InitSessionContext):
| Offset | Field |
|---|---|
| +0x00..+0x3f | Username (64 bytes, used as key for session file lookup) |
| +0x40..+0x53 | Username copy (from web session file) |
| +0x54..+0x153 | Display name (256 bytes, from web session file) |
| +0x154 | IPMI privilege level (1=callback, 2=user, 3=operator, 4=admin) |
| +0x155 | Video permission (0=denied, 1=granted) |
| +0x156 | Keyboard/mouse permission (0=denied, 1=granted) |
| +0x157 | Kick/admin permission (0=denied, 1=granted) |
| +0x158 | Power control permission (0=denied, 1=granted) |
| +0x15c | Viewer language param 1 (u32, from /nv/IKVMViewerLang) |
| +0x160 | Viewer language param 2 (u32, from /nv/IKVMViewerLang) |
Session Linked List Node (std::list node at DAT_0001e58c, 0x60-byte data):
| Offset | Field |
|---|---|
| +0x00 | next pointer (circular, head=DAT_0001e58c) |
| +0x04 | prev pointer |
| +0x08 | pthread_self() of session thread |
| +0x0c | Thread ID (from accept) |
| +0x10..+0x57 | Message queue info (0x48 bytes, copy of SessionContext+0x04) |
| +0x58 | (additional session info, copy of SessionContext mq attrs) |
| +0x5c | PrivilegeInfo* pointer (copy of SessionContext+0x4c) |
Startup and Initialization
Global Constructors (before main)
C++ global constructors run before main(), registered via __cxa_atexit:
- FUN_0000caf0 — Initializes
std::ios_base::Initat DAT_0001e584 (C++ iostream) - FUN_0000dba0 — Initializes the session linked list at DAT_0001e58c/0x1e590 as an empty circular list
- FUN_0000aea8 — Initializes the pending connection list at DAT_0001e578/0x1e57c as an empty circular list, then constructs the global DeviceManager at DAT_0001e560 (calls FUN_0000b63c)
DeviceManager Construction (FUN_0000b63c)
The DeviceManager constructor at 0xb63c allocates and initializes three sub-objects:
operator_new(0x84)→ VideoCaptureDevice (FUN_0000bd08): opens/dev/vcd, initializes VCD hardware, allocates frame bufferoperator_new(0x1c)→ KeyboardDevice (FUN_0000cc08): opens/dev/keyboardoperator_new(0x1c)→ MouseDevice (FUN_0000e81c): opens/dev/mouse
Sets DeviceManager+0x01 = 640 (width), DeviceManager+0x02 = 480 (height).
main() Function (0xa2d4)
The main function is minimal (5 ARM instructions):
int main(int argc, char **argv) {
CreateServerSocket(); // FUN_0000a430
MainAcceptLoop(argc); // FUN_0000acbc (never returns)
return 0;
}
No command-line arguments are processed — argc is passed through to the accept loop but only used as a parameter slot for SpawnDetachedThread.
Shared Memory Initialization (FUN_0000dc10 / thunk at 0xa2ec)
Called from CreateServerSocket (FUN_0000a430) via thunk_FUN_0000dc10. Attaches three shared memory segments:
GlobalVar_Shm_AT(&at_p_St_GlobalVar)— global variables shared between all BMC processesNVRAM_Shm_AT()— persistent NVRAM storagePS_Shm_AT(&at_p_St_PS)— platform-specific configuration (contains IKVM port at +0x1b42)
On success, registers this process's PID in the global variable table:
- Stores PID at
GlobalVar[0x24d4 + counter*4](process registry array) - Increments counter at
GlobalVar[0x2534] - Also stores PID at
GlobalVar[0x2724](ikvmserver-specific PID slot)
On failure at any step, prints to stderr and returns without initializing.
CreateServerSocket (FUN_0000a430)
Allocates a 4-byte server socket object (via operator_new(4)), calls SharedMemoryInit, and stores the result at DAT_0001e580 (the global server socket pointer). Uses singleton pattern — only allocates if DAT_0001e580 is NULL.
Main Accept Loop (FUN_0000acbc)
The accept loop performs initialization steps then enters an infinite loop:
- Read IKVM port: Calls FUN_0000b254 which reads
PS_Shm[0x1b42]as a 16-bit port number. Returns 5900 if the value is 0. - Install signal handlers:
- FUN_0000b2ec:
sigaction(SIGUSR2=12, FUN_0000a2f0)— preview capture handler - FUN_0000b2bc:
sigaction(SIGUSR1=10, FUN_0000a9dc)— session kill handler - Both handlers use
SA_NODEFERflag (0x40000000)
- FUN_0000b2ec:
- Accept loop (infinite
do-while):- Allocates a connection descriptor (
operator_new(0x20)) with a pthread mutex - Allocates a connection info struct (
operator_new(0x14)) - Calls
InitConnectionDescriptor(FUN_0000ce5c): stores socket fd and DeviceManager pointer - Spawns a detached accept thread via
SpawnDetachedThread(entry at LAB_0000a620) - Polls in a loop with 1-second sleep until the accept thread sets the ready flag (descriptor+0x04 != 0)
- Prints "GetVaildSession=true" [sic] and unlocks the descriptor mutex
- Allocates a connection descriptor (
The accept thread (at LAB_0000a620) calls the socket accept wrapper, and on success, the per-session handler (FUN_0000d334) proceeds with the RFB handshake.
Signal Handlers
SIGUSR2 — Preview Capture (FUN_0000a2f0):
Creates a non-detached thread (via pthread_create) running FUN_0000a33c. The thread:
- Calls
DeviceManager::CaptureFramewith incremental=-1 (forced full capture) - If capture data length > 0: calls
DeviceManager::SaveSnapshot→ writes/tmp/Snapshot.bmp - If no video signal:
cp -f /web/images/blank.bmp /tmp/Snapshot.bmp - Calls
DeviceManager::ReleaseCaptureandDeviceManager::UnlockCapture
SIGUSR1 — Session Kill (FUN_0000a9dc):
Reads /tmp/killsess file, dispatches based on content:
"-1"→KillAllSessions(FUN_0000a4fc): sends cmd=6 "Kill Session Due to Change Port" to all sessions (only if >1 active)"-2"→ResetIPHandler(FUN_0000a9c0): calls cleanup (FUN_0000a998), thenexit(0)- Otherwise → username match: iterates session list, compares with PrivilegeInfo+0x40, sends cmd=3 "Kill Session Due to Web Logout" to matching session
- Deletes
/tmp/killsessafter processing
ResetIPHandler (FUN_0000a9c0):
Prints "ResetIP_Handler", calls FUN_0000a998 which kills all sessions in the pending connection list and frees the server socket object at DAT_0001e580, then calls exit(0).
Signal Handler Setup Details
Two patterns for signal installation:
- FUN_0000b2bc/FUN_0000b2ec: Uses
sigactionwithsa_flags = SA_NODEFER(0x40000000). Used for SIGUSR1/SIGUSR2. - FUN_0000af84: Uses
sigactionwithsa_flags = 0. General-purpose signal installer. - FUN_0000b2e8: No-op, called between signal setup calls.
Video Capture and Encoding Pipeline
Architecture Overview
The video pipeline has three layers:
- DeviceManager (global singleton at 0x1e560) — facade that delegates to sub-objects
- VideoCaptureDevice (0x84 bytes at DeviceManager+0x14) — manages capture slots and builds frame packets
- VCD hardware layer (functions at 0x11368..0x12a6c) — direct register manipulation of the Nuvoton WPCM450 VCD block
The DeviceManager also holds a KeyboardDevice (+0x10, opens /dev/keyboard) and MouseDevice (+0x0c, opens /dev/mouse), but these are not part of the video pipeline.
DeviceManager Vtable (0x14518, 18 entries)
| Offset | Address | Method | Delegates to |
|---|---|---|---|
| +0x00 | 0xb7b4 | ~DeviceManager() | — |
| +0x04 | 0xb7e4 | ~DeviceManager() (deleting) | — |
| +0x08 | 0xb824 | SendMouseEvent(x, y, buttons) | MouseDevice vtable+0x08 (0xe920) |
| +0x0c | 0xba5c | MouseReSync — delegates to MouseDevice vtable+0x0c (0xe9b0) | ioctl ATEN_MOUSE_IOCRESYNC |
| +0x10 | 0xb880 | GetMouseCursorPos() → (x, y) | MouseDevice vtable+0x14 (0xea00) |
| +0x14 | 0xb8b8 | SetMouseMode(mode, ...) | MouseDevice vtable+0x18 (0xea40) |
| +0x18 | 0xb8f0 | SendKeyboardEvent(key, down) | KeyboardDevice vtable+0x08 (0xcd04) |
| +0x1c | 0xb90c | GetKeyboardInfo() → (info, status) | KeyboardDevice vtable+0x14 (0xcdbc) |
| +0x20 | 0xb944 | (nop, returns) | — |
| +0x24 | 0xb948 | CaptureFrame(result, ...) | VCD vtable+0x08 (0xc770) |
| +0x28 | 0xb9c0 | SaveSnapshot(data, ...) | VCD vtable+0x28 (0xbf70) |
| +0x2c | 0xba3c | ReleaseCapture() | VCD vtable+0x0c |
| +0x30 | 0xba78 | CaptureForClient(result, ...) | VCD vtable+0x10 |
| +0x34 | 0xbaf0 | VideoControl(params...) | VCD vtable+0x1c |
| +0x38 | 0xbb60 | GetVideoInfo() → (w, h) | VCD vtable+0x20 |
| +0x3c | 0xbb84 | UnlockCapture() | VCD vtable+0x24 (0xbf58) |
| +0x40 | 0xb8d4 | DoHotPlug() | MouseDevice vtable+0x1c (0xeae0) |
| +0x44 | 0xb860 | GetKbLed() → status byte | KeyboardDevice vtable+0x1c (0xcde4) |
VideoCaptureDevice Vtable (0x14578, 11 entries)
| Offset | Address | Method |
|---|---|---|
| +0x00 | 0xbe9c | ~VideoCaptureDevice() |
| +0x04 | 0xbec8 | ~VideoCaptureDevice() (deleting) |
| +0x08 | 0xc770 | CaptureFrame — main capture entry point |
| +0x0c | 0xcad0 | ReleaseCapture |
| +0x10 | 0xcac0 | CaptureForClient (incremental with cursor) |
| +0x14 | 0xca4c | (stub, returns 1) |
| +0x18 | 0xca54 | (stub, returns 1) |
| +0x1c | 0xca5c | VideoControl — applies video parameters |
| +0x20 | 0xca98 | GetVideoInfo — returns current resolution |
| +0x24 | 0xbf58 | UnlockCapture — releases session slot, unlocks mutex |
| +0x28 | 0xbf70 | SaveSnapshot — writes /tmp/Snapshot.bmp |
KeyboardDevice Vtable (0x14674, 8 entries)
| Offset | Address | Method |
|---|---|---|
| +0x00 | 0xcce4 | ~KeyboardDevice() |
| +0x04 | 0xccf4 | ~KeyboardDevice() (deleting) |
| +0x08 | 0xcd04 | SendKeyEvent(keycode, down_flag) — writes 8 bytes to /dev/keyboard |
| +0x0c | 0xcdac | (stub, returns 1) |
| +0x10 | 0xcdb4 | (stub, returns 1) |
| +0x14 | 0xcdbc | GetKeyboardInfo() — returns 2 words from +0x14,+0x18 |
| +0x18 | 0xcdd4 | SetKeyboardInfo(a, b) — stores into +0x14,+0x18 |
| +0x1c | 0xcde4 | GetKbLed() — ioctl 0x80046205 (ATEN_KEYBOARD_IOCGETKBLED) |
MouseDevice Vtable (0x14a5c, 8 entries)
| Offset | Address | Method |
|---|---|---|
| +0x00 | 0xe900 | ~MouseDevice() |
| +0x04 | 0xe910 | ~MouseDevice() (deleting) |
| +0x08 | 0xe920 | SendMouseEvent(x, y, buttons) — writes 20 bytes to /dev/mouse |
| +0x0c | 0xe9b0 | ReSync() — ioctl 0x40046204 (ATEN_MOUSE_IOCRESYNC) |
| +0x10 | 0xe9d8 | HotPlug() — ioctl 0x40046201 (ATEN_MOUSE_IOCHOTPLUG) |
| +0x14 | 0xea00 | GetCursorPos() — ioctl 0x80046202 (ATEN_MOUSE_IOCGETMODE), returns pos |
| +0x18 | 0xea40 | SetMouseMode(mode, param) — sets mode, ioctls, saves /nv/MouseMode |
| +0x1c | 0xeae0 | DoHotPlug() — ioctl 0x40046201 (ATEN_MOUSE_IOCHOTPLUG) |
VCD Hardware Interface
The WPCM450 SoC's Video Capture Device (VCD) is accessed through memory-mapped registers and a character device:
Device: /dev/vcd (major 25), opened read-only. Used for:
select()— poll for capture completion (100ms timeout)read()— 1-byte read to acknowledge capture done
VCD Control Register Block (pointed to by DAT_0001e4b0, mapped from kernel module):
| Offset | Purpose | Notes |
|---|---|---|
| +0x00 | Frame buffer base address | Written by SetFrameBufferAddr |
| +0x04 | Frame buffer offset | Second buffer for diff capture |
| +0x08 | Line stride | Packed: low16=stride, high16=stride |
| +0x0c | Capture resolution | Packed: [26:16]=width, [10:0]=height |
| +0x14 | Mode/control register | Bits: [18:16]=mode, [15:14]=source, [11]=diff, [5:4]=pixfmt |
| +0x18 | Trigger register | Write 0=start full, 0x20=start diff, read for done |
| +0x1c | Status register | Bit 0=done, bit 3=valid, bit 30=error, etc. |
| +0x5c | Extended control | [31:28]=tile_size, [14:8]=color_depth, [7:4]=ver_deci, [3:0]=hor_deci |
VCD Video Status Registers (memory-mapped at 0xF003xxxx):
| Address | Purpose |
|---|---|
| 0xF0030300 | VCD mode status (bit 7: digital/analog) |
| 0xF0030310 | Horizontal active (low byte) |
| 0xF0030314 | Horizontal active (high bits [2:0]) |
| 0xF0030320 | Vertical active (low byte) |
| 0xF0030324 | Vertical active (high bits [2:0]) |
| 0xF0030340 | Pixel clock divider |
| 0xF0030344 | Pixel clock multiplier |
| 0xF0030348 | Pixel clock post-divider |
| 0xF0030358 | Display timing info |
| 0xF0030364 | Sync polarity flags |
| 0xF0030370 | Horizontal front porch |
VCD Frame Buffers (physical addresses mapped into process space):
| Address | Purpose |
|---|---|
| 0xF0010000 | VCD identification register (revision in bits [31:24]) |
| 0xF001003c | VCD control register (direct HW access) |
| 0xF0010040 | VCD status register (bit 0: vsync) |
| 0xF0010060 | VCD extended control (for Hermon rev ≥ 8) |
| 0xF0020000 | Tile dirty bitmap (used for differential encoding) |
| 0xF0100000 | Frame buffer slot 0 (pixel data) |
| 0xF0400000 | Frame buffer slot 1 |
| 0xF0700000 | Frame buffer slot 2 |
| 0xF0A00000 | Frame buffer slot 3 |
Each frame buffer slot is 0x300000 bytes (3MB), supporting up to 1280×1024 at 2 bytes/pixel.
Capture Flow (FBUpdateRequest → Response)
-
Client sends FBUpdateRequest (type 0x03):
[u8 incremental] — 0=full, 1=incremental [u16 x-position] [u16 y-position] [u16 width] [u16 height] -
Server calls DeviceManager::CaptureFrame (vtable+0x24 → 0xb948 → 0xc770):
- Acquires the capture mutex
- Looks up or allocates a capture slot for this session (max 4 concurrent sessions)
- If session already has a valid frame (slot.valid == 1), reuses the existing capture
- Otherwise, configures VCD hardware:
- Sets frame buffer address:
slot_index * 0x300000 + 0x6400000 - Sets pixel format: 1=8bpp or 2=RGB555
- Calls WaitForValidResolution (detects connected display mode)
- Triggers capture (full or differential based on
incrementalflag) - Waits for capture completion via select()/read() on /dev/vcd
- Sets frame buffer address:
-
Resolution detection (WaitForValidResolution, 0x12524):
- Reads horizontal/vertical active pixels from VCD status registers
- Detects digital vs analog mode from bit 7 of 0xF0030300
- Prints
[Hermon] vcd_mode = 0x%x, %s modeand[Hermon] digital mode: h=%d, v=%d - Caps resolution to max 1280×1024
- Updates VCD line stride (power-of-2 rounding)
-
Frame building — two paths based on
incrementalflag:Full frame (incremental=0, via BuildFullFrame at 0xc2ac):
- Copies entire frame buffer from VCD memory to the output buffer
- If width is 512-aligned, uses single memcpy
- Otherwise, copies line-by-line using stride
- Builds 10-byte Hermon sub-header (see below)
Differential frame (incremental=1, via BuildDifferentialFrame at 0xc4c4):
- Reads tile dirty bitmap from 0xF0020000
- Bitmap is organized as rows of 32-bit words, each bit = one 16×16 tile
- For each dirty tile, copies 16×16 pixel data via CopyTileData (0xc404)
- Each tile entry in the output:
[4 bytes padding][u8 tile_y][u8 tile_x][16×16 pixel data] - Builds 10-byte Hermon sub-header with tile count
Differential-to-full fallback: CaptureFrame (0xc770) compares the differential data length against the full frame size (
bpp × width × height + 10). If the differential data would be larger or equal, it discards the differential result and falls back to a full frame capture instead. This prevents wasteful transmission when most tiles are dirty.No-signal (no VGA input detected):
- Returns width=0xFFFFFD80 (-640 signed), height=0xFFFFFE20 (-480 signed), data_length=0
- Client should display a "no signal" indicator
Empty frame (slot allocation failed):
- Returns width=1, height=1, data_length=10
-
Server sends FramebufferUpdate response (type 0x00):
[u8 type=0x00] [u8 padding=0x00] [u16 num_rects=1] [u16 x=0] [u16 y=0] [u16 width] — from CaptureResult [u16 height] — from CaptureResult [u32 encoding=0x59] — ATEN Hermon encoding [u32 subrects_info] — tile count or mode info [u32 data_length] — total byte count of frame data [data_length bytes] — frame data with Hermon sub-header -
Server calls DeviceManager::ReleaseCapture (vtable+0x2c → 0xba3c) to release resources.
Hermon Frame Data Format (Encoding 0x59)
The frame data (after the RFB rectangle header) contains a 10-byte Hermon sub-header followed by pixel data:
Full frame (sub-header byte 0 = 0x01):
[u8 0x01] — full frame marker
[u8 mode] — capture slot mode = color depth flag (0=RGB555/2bpp, non-zero=8bpp)
[u8 0x12] — magic bytes (constant)
[u8 0x34]
[u8 0x56]
[u8 0x78]
[u8 size_hi_byte] — high byte of (data_length >> 8)
[u8 size_lo_byte] — low byte of data_length
[u8 len_hi] — data_length >> 8
[u8 len_lo] — data_length & 0xff
[raw pixel data] — width × height × bpp bytes, row-major
Differential frame (sub-header byte 0 = 0x00):
[u8 0x00] — differential frame marker
[u8 mode] — capture slot mode = color depth flag (0=RGB555/2bpp, non-zero=8bpp)
[u32 tile_count] — number of dirty tiles (big-endian)
[u32 data_length] — total data length (big-endian)
[tile entries...]
The mode byte (sub-header byte 1) is the capture slot's mode field from
CaptureSlot+0x04, which corresponds to the VCD pixel format setting. The client's
HermonVideoDecoder uses this as the color_depth_flag parameter: 0 means 16bpp
RGB555 (2 bytes/pixel), non-zero means 8bpp indexed (1 byte/pixel).
Each tile entry:
[4 bytes padding] — uninitialized (from CopyTileData stack, not meaningful)
[u8 tile_y] — tile row (tile coordinates, not pixels)
[u8 tile_x] — tile column
[16 rows × row_bytes] — 16 rows of pixel data, each row = 16 × bpp bytes
Tile dimensions are 16×16 pixels. For 2bpp (RGB555), each tile row = 32 bytes (0x20). For 1bpp (8-bit), each tile row = 16 bytes (0x10).
Pixel Formats
The VCD supports multiple pixel formats, selected by VCD register +0x14 bits [5:4]:
| Mode bits | Bpp | Format | Notes |
|---|---|---|---|
| 0x00 | 2 | RGB555 | Default, 16-bit per pixel |
| 0x10 | 1 | 8-bit indexed | Palette mode |
| 0x20 | 1 | 8-bit indexed | Alternate palette mode |
| 0x30 | 4 | 32-bit RGBA | High color (rarely used) |
RGB555 is the default and most common format. The byte order in the frame buffer follows the VCD hardware (little-endian), but the client's Hermon decoder handles the byte swapping.
VideoControl Handlers
Three client message types control video parameters, all delegating to DeviceManager::VideoControl (vtable+0x34 → 0xbaf0 → VCD vtable+0x1c):
| Type | Handler | Client sends | Effect |
|---|---|---|---|
| 0x14 | 0xf55c | [u16 param] |
VideoControl(param, -1) — sets first video parameter |
| 0x18 | 0xf4dc | [u16 param] |
VideoControl(-1, param) — sets second video parameter |
| 0x32 | 0xf44c | [u16 p1][u16 p2] |
VideoControl(p1, p2) — sets both parameters |
A value of -1 (0xFFFF) means "don't change" for that parameter.
Preview/Snapshot Mechanism
The web interface triggers video preview captures via SIGUSR2:
- Web CGI (
CapturePreview.cgi) sends SIGUSR2 to the ikvmserver process - Signal handler at 0xa2f0 spawns a detached thread running 0xa33c
- The thread calls DeviceManager::CaptureFrame with incremental=-1 (forced full capture)
- If capture succeeds (data_length > 0), calls DeviceManager::SaveSnapshot to write
/tmp/Snapshot.bmp— a raw RGB555 BMP with flipped row order - If no video signal, copies
/web/images/blank.bmpto/tmp/Snapshot.bmp - Calls DeviceManager::ReleaseCapture and DeviceManager::UnlockCapture to clean up
Keyboard Input Path
Client → Server Wire Format (Type 0x04 KeyEvent)
The KeyEvent handler at 0xf8c4 reads the first byte after the type byte to determine the encryption mode:
Unencrypted mode (first byte == 0x00):
[u8 type=0x04]
[u8 0x00] — encryption flag: plaintext
[u8 down_flag] — 0=release, 1=press
[2 bytes skip]
[u32 keycode] — USB HID keycode (big-endian)
[9 bytes skip] — total payload = 17 bytes after type
Encrypted mode (first byte != 0x00):
[u8 type=0x04]
[u8 !0x00] — encryption flag: AES-encrypted
[16 bytes ciphertext] — AES-128-CBC encrypted block
— total payload = 17 bytes after type
Decrypted plaintext (16 bytes):
[u8 down_flag] — byte 0: 0=release, 1=press, 2=press-and-release
[2 bytes unused] — bytes 1-2
[u8 keycode_b3] — bytes 3-6: keycode as big-endian u32
[u8 keycode_b2] — (after ByteSwap16, the u32 has each u16 pair byte-swapped)
[u8 keycode_b1]
[u8 keycode_b0]
[9 bytes unused] — bytes 7-15
After decryption, ByteSwap16 at 0x130f4 is applied to the keycode u32. This function performs a 32-bit byte reordering: ABCD → BADC (swapping bytes within each 16-bit half). This corrects for the byte order difference between the big-endian network format and the format expected by the kernel driver.
The handler then calls DeviceManager::SendKeyboardEvent (vtable+0x18 → 0xb8f0 → KeyboardDevice::SendKeyEvent at 0xcd04).
KeyboardDevice::SendKeyEvent (0xcd04)
Writes an 8-byte record to /dev/keyboard:
[u32 keycode] — USB HID keycode
[u32 down_flag] — 0=release, 1=press
When down_flag == 2 (press-and-release), the function writes twice: first with down_flag=1 (press), then with down_flag=0 (release). This implements single-shot key events.
usb_hid.ko Keyboard Processing (keyboardAction)
The kernel module's keyboardAction function receives the 8-byte record from the write buffer (copied as kb_work_buffer entries of 0x14 bytes via copy_from_user). It processes the keycode and down_flag:
Modifier keys (USB HID keycodes 0xE0-0xE7) are mapped to the modifier byte (byte 0) of kb_code:
| Keycode | Modifier Bit | Key |
|---|---|---|
| 0xE0 | 0x01 | Left Control |
| 0xE1 | 0x02 | Left Shift |
| 0xE2 | 0x04 | Left Alt |
| 0xE3 | 0x08 | Left GUI |
| 0xE4 | 0x10 | Right Control |
| 0xE5 | 0x20 | Right Shift |
| 0xE6 | 0x40 | Right Alt |
| 0xE7 | 0x80 | Right GUI |
On key press, the corresponding bit is ORed in. On release, the bit is ANDed out.
Toggle keys (CapsLock=0x39, NumLock=0x53, ScrollLock=0x47) have special handling. When pressed, the driver checks the current LED status (kb_led_status). If the LED for that key is already on, the keypress is suppressed (no HID report sent). This avoids a toggle conflict — the host OS manages LED state and the driver tracks it. The special keycodes 0xFF39, 0xFF53, 0xFF47 perform the inverse: suppressed when LED is off.
Regular keys (all others, masked to 0xFF) are inserted into kb_code slots 2-7 via km_insert on press, and removed via km_remove on release. The kb_code buffer supports up to 6 simultaneous non-modifier keys (standard USB HID boot protocol).
After updating kb_code, the 8-byte report is copied to hid_test_rep_data and sent via _usb_device_send_data(dev, 1, data, 8) on USB endpoint 1.
USB HID Keyboard Report Format (8 bytes, EP1)
Byte 0: Modifier bitmap (E0-E7 keys, see table above)
Byte 1: Reserved (always 0)
Bytes 2-7: Up to 6 simultaneous keycodes (0 = no key)
This is the standard USB HID Boot Keyboard Report as defined by the HID report descriptor at hid_report_keyboard (0x170bf, 64 bytes). The descriptor declares:
- Usage Page: Generic Desktop (0x01), Usage: Keyboard (0x06)
- 8 modifier bits (Usage 0xE0-0xE7, 1-bit each)
- 1 reserved byte
- 5 LED output bits (Usage 0x01-0x05)
- 6 keycode bytes (Usage 0x00-0x8B)
Mouse/Pointer Input Path
Client → Server Wire Format (Type 0x05 PointerEvent)
The PointerEvent handler at 0xf7d0 reads the first byte after the type byte to determine encryption:
Unencrypted mode (first byte == 0x00):
[u8 type=0x05]
[u8 0x00] — encryption flag: plaintext
[u8 button_mask] — RFB button mask
[u16 x_position] — big-endian, pixel coordinates
[u16 y_position] — big-endian, pixel coordinates
[11 bytes skip] — total payload = 17 bytes after type
Encrypted mode (first byte != 0x00):
[u8 type=0x05]
[u8 !0x00] — encryption flag: AES-encrypted
[16 bytes ciphertext] — AES-128-CBC encrypted block
— total payload = 17 bytes after type
Decrypted plaintext (16 bytes):
[u8 button_mask] — byte 0: RFB button mask
[u8 x_hi] — bytes 1-2: x position as big-endian u16
[u8 x_lo]
[u8 y_hi] — bytes 3-4: y position as big-endian u16
[u8 y_lo]
[11 bytes unused] — bytes 5-15
After decryption, ByteSwap16 is applied to the x and y values individually. ByteSwap16 swaps bytes within each 16-bit half of a u32, converting from big-endian network order to native little-endian.
The handler calls DeviceManager::SendMouseEvent (vtable+0x08 → 0xb824 → MouseDevice::SendMouseEvent at 0xe920).
Button Mask Encoding
The RFB button mask uses standard VNC encoding:
| Bit | Button |
|---|---|
| 0 | Left button |
| 1 | Middle button |
| 2 | Right button |
| 3 | Scroll up |
| 4 | Scroll down |
MouseDevice::SendMouseEvent (0xe920)
Writes a 20-byte record to /dev/mouse:
[u32 x_position] — pixel X coordinate
[u32 y_position] — pixel Y coordinate
[u32 button_mask] — RFB button mask
[u32 display_width] — from DeviceManager+0x04
[u32 display_height] — from DeviceManager+0x08
The MouseDevice reads display_width and display_height from its back-pointer to the DeviceManager. These are used by the kernel module for absolute→relative coordinate conversion.
usb_hid.ko Mouse Processing (mouseAction)
The mouseAction function receives the 20-byte record and dispatches based on mousetype and hid_support_flag:
Mode 1: Absolute Mouse (mousetype == 1 or hid_support_flag == 1 with mousetype != 3):
Coordinates are scaled to the 0-32767 range expected by the absolute HID descriptor:
scaled_x = (x << 7) / resolutionX — maps to 0-32767 range
scaled_y = (y << 7) / resolutionY — maps to 0-32767 range
The result is packed into a 6-byte absolute mouse report:
Byte 0: Button mask (bit 0=left, bit 1=middle, bit 2=right — note: bits 1 and 2 are swapped from RFB)
Byte 1: X low byte (scaled_x bits [7:0])
Byte 2: X high bits (scaled_x bits [11:8] in low nibble, fractional in high nibble)
Byte 3: Y low byte
Byte 4: Y high bits
Byte 5: Scroll wheel (signed 8-bit: button_mask bit 3 → +1, bit 4 → -1, else 0)
Sent via _usb_device_send_data(dev, 2, data, 6) on USB endpoint 2.
Mode 2: Relative Mouse with Scroll (mousetype == 2):
Coordinates are converted to relative deltas from the previous position (xOld, yOld). Large deltas are broken into multiple USB reports, each clamped to the -127..+127 range. The driver iteratively moves toward the target in steps, waiting for each USB transfer to complete before sending the next.
4-byte relative mouse report with scroll:
Byte 0: Button mask (same remapping as absolute)
Byte 1: X delta (signed 8-bit, -127 to +127)
Byte 2: Y delta (signed 8-bit, -127 to +127)
Byte 3: Scroll wheel (signed 8-bit)
Sent as 4 bytes on EP2.
Mode 3: Raw Passthrough (mousetype == 3):
X and Y are treated as raw signed byte values (sign-extended from 8-bit). Used for direct PS/2 passthrough. 4-byte report format, same as mode 2.
Default (hid_support_flag == 1, mousetype != 3): Standard Relative Mouse:
Same relative delta logic as mode 2, but sends 3-byte reports (no scroll wheel).
Mouse Button Remapping (RFB → USB HID)
The kernel module remaps the RFB button mask bits:
| RFB bit | USB HID bit | Button |
|---|---|---|
| 0 (0x01) | 0 (0x01) | Left |
| 2 (0x04) | 1 (0x02) | Middle |
| 1 (0x02) | 2 (0x04) | Right |
| 3 (0x08) | scroll +1 | Scroll up |
| 4 (0x10) | scroll -1 | Scroll down |
Note: RFB has right=bit1, middle=bit2 but USB HID has middle=bit1, right=bit2.
Mouse Synchronization (mouseSync)
Called when the display resolution changes (prev_resX != resolutionX || prev_resY != resolutionY). Resets the internal tracking position to (0,0):
- In absolute mode (mousetype=1): sends a single report with position (0, 1, 0, 1) — near top-left
- In relative modes (mousetype=2 or 3): sends 16 reports of (-127, -127) deltas to force the cursor to the top-left corner, then resets
xOld = yOld = 0
USB HID Mouse Report Descriptors
Relative Mouse (hid_report_mouse at 0x1714e, 52 bytes):
- Usage Page: Generic Desktop (0x01), Usage: Mouse (0x02)
- 3 buttons (1-bit each) + 5-bit padding
- X, Y, Wheel: 8-bit signed each (-127 to +127)
- Total report: 3 or 4 bytes (3 buttons + X + Y [+ Wheel])
Absolute Mouse (hid_report_absolute_mouse at 0x170ff, 79 bytes):
- Usage Page: Generic Desktop (0x01), Usage: Mouse (0x02)
- 3 buttons (1-bit each) + 5-bit padding
- X, Y: 16-bit unsigned each (0 to 32767, logical and physical range)
- Wheel: 8-bit signed (-127 to +127)
- Total report: 6 bytes
The active descriptor is selected by mousetype via the wpcm450_hotplug ioctl, which updates the HID descriptor length field and triggers a USB disconnect/reconnect sequence (500ms disconnect + 1500ms reconnect delay) to force the host to re-enumerate.
AES Encryption for Input Events
Encryption Toggle
The first byte after the message type byte serves as an encryption flag:
0x00: Plaintext mode — remaining fields are read directly from the stream- Non-zero: Encrypted mode — next 16 bytes are an AES-128-CBC ciphertext block
Both keyboard (type 0x04) and mouse (type 0x05) events use this same scheme. The payload is always 17 bytes after the type byte: 1 flag byte + 16 bytes (either plaintext fields or ciphertext).
AES Decrypt Path
When encrypted, the handler:
- Reads 16 bytes of ciphertext from the stream
- Calls AESCrypto vtable+0x10 (thunk at
0x112d8→FUN_00011268) - FUN_00011268 copies the static IV (
00 01 02 .. 0f) and key material (2b 7e 15 16 ..) to stack-local buffers - Calls the CBC wrapper at
0x110c0withmode=1(decrypt),key_size_param=0(→ 128-bit),block_count=1 - The CBC wrapper calls
aes_set_keyfrom libipmicrypt.so, thenaes_decryptfor the single block, and XORs the result with the IV
The decrypted 16-byte plaintext is placed in a local buffer, and the handler extracts the relevant fields from it.
ByteSwap16 (0x130f4)
Applied to multi-byte values after decryption. Performs a byte reorder on a 32-bit value:
Input: byte3 byte2 byte1 byte0 (as 32-bit word)
Output: byte0 byte3 byte2 byte1 (CONCAT31 of shuffled bytes)
More precisely, for a u32 value param_2:
- Result byte 0 =
param_2 >> 24(original MSB) - Result byte 1 =
param_2 >> 8(original byte 1) - Result byte 2 =
param_2 >> 16(original byte 2) - Result byte 3 =
param_2(original LSB)
This effectively byte-swaps within each 16-bit half: [B1 B0 B3 B2] → [B0 B1 B2 B3]. It converts from big-endian wire format (where each u16 is in network byte order) to the native ARM little-endian representation.
AESCrypto Vtable (0x14cb4, 6 entries)
| Offset | Address | Method |
|---|---|---|
| +0x00 | 0x10fdc | ~AESCrypto() |
| +0x04 | 0x10ff4 | ~AESCrypto() (deleting) |
| +0x08 | 0x11018 | (unused multi-block ECB wrapper) |
| +0x0c | 0x11358 | (related to VCD/sync) |
| +0x10 | 0x112d8 | Decrypt — thunk: calls 0x11268 (AES-128-CBC decrypt), returns 1 |
| +0x14 | (data) | (not a function pointer) |
CBC Wrapper (0x110c0)
FUN_000110c0(this, mode, key_size_index, input, block_count, key_material, output, iv)
mode: 0=encrypt, 1=decryptkey_size_index: 0 → 128-bit (0*64+128=128), 1 → 192-bit, 2 → 256-bitblock_count: number of 16-byte blocks- For encrypt: XORs plaintext with IV/previous ciphertext, then ECB encrypts
- For decrypt: ECB decrypts, then XORs with IV/previous ciphertext
USB HID Device Interface (usb_hid.ko)
Device Files
| Device | Minor | USB Endpoint | Direction | Purpose |
|---|---|---|---|---|
/dev/keyboard |
0 | EP1 (IN) | BMC → Host | Virtual USB keyboard |
/dev/mouse |
1 | EP2 (IN) | BMC → Host | Virtual USB mouse |
Both are character devices opened with O_RDWR by ikvmserver. Data flows from ikvmserver → kernel module → USB controller → host USB port.
Write Interface
write(fd, buf, len) queues a work item:
- For
/dev/keyboard(minor 0): copieslenbytes (expected 8) intokb_work_buffer[slot](0x14 bytes per slot), queueskeyboardActiontokb_queue - For
/dev/mouse(minor 1): copieslenbytes (expected 0x14=20) intoms_work_buffer[slot], queuesmouseActiontoms_queue - Work queue has 60 (0x3c) slots, wrapping to 0 when full
- Each work item is protected by a semaphore (
kb_work_count_mutex/ms_work_count_mutex)
Ioctl Interface
The ioctl dispatch table at keyboradmouse_ioctl_rountin [sic] (0x17184) maps 5 ioctls:
| Ioctl Code | Direction | Handler | Purpose |
|---|---|---|---|
| 0x40046201 | Write | wpcm450_hotplug |
USB disconnect/reconnect (re-enumeration) |
| 0x80046202 | Read | wpcm450_get_mode |
Get current mousetype (4 bytes) |
| 0x40046203 | Write | wpcm450_set_mode |
Set mousetype (4 bytes) |
| 0x40046204 | Write | wpcm450_resync |
Queue mouse position resync |
| 0x80046205 | Read | wpcm450_getkbled |
Get keyboard LED status (1 byte) |
Ioctl magic number: 0x62 ('b'), size field: 4 bytes.
USB HID Endpoints
| Endpoint | Type | Direction | Max Packet | Content |
|---|---|---|---|---|
| EP0 | Control | Both | 64 | Standard USB enumeration |
| EP1 | Interrupt IN | BMC→Host | 8 | Keyboard reports (8 bytes) |
| EP2 | Interrupt IN | BMC→Host | 8 | Mouse reports (3-6 bytes) |
ISR completion callbacks have swapped naming (a bug in the source):
wpcm450_service_ep1handles mouse completion (releasesmouse_data_mutex, setsms_isr_flag)wpcm450_service_ep2handles keyboard completion (releaseskeyboard_data_mutex, setskb_isr_flag)
Keyboard Write Data → USB HID Report
ikvmserver writes 8 bytes:
[u32 keycode] — USB HID usage code (0x00-0x8B for regular, 0xE0-0xE7 for modifiers)
[u32 down_flag] — 0=release, 1=press
The keyboardAction work handler:
- Checks modifier range (0xE0-0xE7): sets/clears the corresponding bit in
kb_code[0] - Checks toggle keys (0x39 CapsLock, 0x47 ScrollLock, 0x53 NumLock): conditionally handles based on
kb_led_status - Regular keys: inserts into
kb_code[2..7]on press, removes on release - Copies the 8-byte
kb_codetohid_test_rep_data - Sends via
_usb_device_send_data(dev, 1, data, 8)on EP1
Mouse Write Data → USB HID Report
ikvmserver writes 20 bytes:
[u32 x_position] — absolute pixel X
[u32 y_position] — absolute pixel Y
[u32 button_mask] — RFB-format button mask
[u32 display_width] — current display width (for scaling)
[u32 display_height] — current display height (for scaling)
The mouseAction work handler stores display_width → resolutionX and display_height → resolutionY globals. If resolution changed since last call, it triggers mouseSync to reset position tracking.
The output depends on mousetype (see Mouse/Pointer Input Path section above).
Mouse Mode Persistence
The mouse mode is persisted to /nv/MouseMode by FUN_0000b318 (SaveMouseMode) in ikvmserver, called from MouseDevice::SetMouseMode. The SetMouseMode handler:
- Prints "MouseSetPT: info.mode=%d !!"
- If mode changed, saves to
/nv/MouseMode - Issues ioctl 0x40046203 (
wpcm450_set_mode) to update the kernel driver - Issues ioctl 0x40046201 (
wpcm450_hotplug) to trigger USB re-enumeration
Keyboard LED Status
The kb_led_status byte in usb_hid.ko tracks LED state set by the host OS via USB SET_REPORT:
| Bit | LED |
|---|---|
| 0 | Num Lock |
| 1 | Caps Lock |
| 2 | Scroll Lock |
This is read by ikvmserver via ioctl 0x80046205 and reported to clients in:
- GetKbdMouseInfo (type 0x35) response
- KeepAlive broadcast (when LED status changes)
GetKbdMouseInfo Handler (type 0x35, 0xeeac)
Calls DeviceManager::GetKeyboardInfo (vtable+0x1c → KeyboardDevice vtable+0x14) which returns two u32 words from KeyboardDevice+0x14 and +0x18. The low bytes of each are sent:
Server → Client:
[u8 type=0x35]
[u8 info_byte_1] — low byte of KeyboardDevice+0x18
[u8 info_byte_0] — low byte of KeyboardDevice+0x14
These values track keyboard/mouse state set via SetKeyboardInfo (KeyboardDevice vtable+0x18).
MouseSync/ReSync Handler (type 0x07, 0xf5dc)
Reads a u16 value (big-endian) via StreamRead16BE and delegates to DeviceManager vtable+0x0c (0xba5c), which dispatches to MouseDevice vtable+0x0c at 0xe9b0 (MouseDevice::ReSync). The ReSync function issues ioctl(fd, ATEN_MOUSE_IOCRESYNC, ¶m) to the kernel's usb_hid module, which queues a mouseSync work item to reset the mouse position tracking.
The u16 value from the client is always 0x0780 (1920), representing the maximum framebuffer width. This value is passed as the ioctl parameter but is not used by the kernel driver's wpcm450_resync handler.
Note: Despite the function map entry's similarity to standard RFB ClientCutText (type 6), this is NOT clipboard functionality. The standard RFB type 6 (ClientCutText) does not exist in the ATEN protocol. Type 0x07 is an ATEN-proprietary MouseSync message, confirmed by tracing the vtable dispatch chain to MouseDevice::ReSync.
Authentication Flow
Handshake Sequence
The full connection handshake is driven by the RFBProtocol vtable (0x14b00):
- InitHandShake (vtable+0x08 →
0xf098) calls VersionExchange then SecurityTypeNegotiation - Authenticate (vtable+0x0c →
0xf37c) performs challenge/response then sends ServerInit - ProtocolHandler (vtable+0x10 →
0x108b8) enters the main message dispatch loop
Version Exchange (0xed88)
Server → Client: "RFB 003.008\n" (12 bytes)
Client → Server: "RFB 003.008\n" (12 bytes)
Server parses client version with sscanf("RFB %03d.%03d\n"), stores major/minor in RFBProtocol+0x1c/+0x20. Returns success only if client sends exactly version 3.8.
Security Type Negotiation (0xf058)
Server → Client: [u8 count=1] [u8 type=0x10]
Client → Server: [u8 selected_type]
The server offers exactly one security type: 0x10 (VNC Authentication, type 16). The client's response byte is read but not validated — any value is accepted.
Authentication (0xf37c)
Server → Client: [24 bytes challenge] (random/uninitialized stack data)
Client → Server: [24 bytes username]
Client → Server: [24 bytes password]
The challenge is 24 bytes written from uninitialized stack memory (not cryptographically random). The client sends username and password in 24-byte fixed-size fields. These are NOT encrypted with the challenge — the challenge is essentially a no-op. The ATEN protocol deviates from standard VNC auth here.
After receiving credentials, the server calls Auth::Authenticate(auth, username, password, priv_info_ptr) via the Auth vtable. The Auth vtable at 0x14c64 entry +0x08 points to a thunk at 0x10f48 that strips the this pointer and calls AuthBackend(username, password, priv_info_ptr) at 0xe638.
On auth success:
Server → Client: [u32 security_result=0] (0 = OK)
Client → Server: [u8 ack] (read but ignored)
Server → Client: [ServerInit message]
On auth failure, the function returns false, and the session handler at 0xd334 throws an "Authentication failed" exception that terminates the connection.
Authentication Backend (0xe638)
Two authentication paths, tried in order:
-
Web Session SSO (0xe2f0): Checks for
/tmp/sess_<username>. If the file exists:- Opens with advisory lock (
flock(LOCK_EX)) - Reads:
<auth_status> <username> <privilege> <display_name> <counter> - Increments the counter (session reference count) and writes it back
- Copies username to PrivilegeInfo+0x40, display name to +0x54
- Sets PrivilegeInfo+0x154 to the privilege level
- Calls SetPermissions and LoadViewerLang
- Tracks total session count in
/tmp/ikvmMaxSess - Logs to SEL via UtilSaveSysEventLog (event 6 if 5th session, event 3 otherwise)
- Returns success if privilege > 2 (operator or admin)
- Opens with advisory lock (
-
IPMI Local Auth: If no session file, calls
UtilAuthUser(username, password).- Returns a user index (0-9) on success, ≥10 on failure
- Looks up the user's privilege level from Platform-Specific shared memory:
PS_Shm[user_index * 0x40 + lan_channel + 0x25] & 0xf - LAN channel is obtained from
UtilSearchLanChannelNo(0, &channel) - Calls SetPermissions and LoadViewerLang
- Returns success if privilege == 4 (admin only for IPMI auth)
Permission Setup (0xdd28 — SetPermissions)
Permissions are set based on IPMI privilege level:
| Privilege | +0x155 Video | +0x156 Kbd/Mouse | +0x157 Kick | +0x158 Power |
|---|---|---|---|---|
| 1 (Callback) | 1 | 0 | 0 | 0 |
| 2 (User) | 1 | 0 | 0 | 0 |
| 3 (Operator) | 1 | * | 1 | 1 |
| 4 (Admin) | 1 | * | 1 | 1 |
| Other | 0 | 0 | 0 | 0 |
*Kbd/Mouse permission (*): Set to 1 only if no other active session already has kbd/mouse permission (exclusive access). Checked by IsKbdMouseExclusive at 0xd740 which scans the session linked list.
Auth Vtable Layout
The Auth object is only 4 bytes (just a vtable pointer). Auth vtable at 0x14c64:
| Offset | Address | Method |
|---|---|---|
| +0x00 | 0x10f28 | ~Auth() |
| +0x04 | 0x10f38 | ~Auth() (deleting) |
| +0x08 | 0x10f48 | Authenticate — thunk: drops this, calls AuthBackend(user, pass, priv) |
The RFBAuth vtable at 0x14ca0 has identical layout but the Authenticate entry is __cxa_pure_virtual — meaning RFBAuth does NOT override Auth's method. In practice, only the Auth base vtable (0x14c64) is used because the constructor at 0x10e7c sets the vtable to Auth base and never overwrites it to RFBAuth.
ServerInit Message (0xf1ec)
The ServerInit message extends standard RFB 3.8 with ATEN-specific fields:
[u16 framebuffer_width=0x01e0] — BE, 480 (swapped, should be 640; ATEN client ignores these)
[u16 framebuffer_height=0x0280] — BE, 640 (swapped, should be 480; ATEN client ignores these)
[u8 bits_per_pixel=32] — 0x20
[u8 depth=24] — 0x18
[u8 big_endian=0]
[u8 true_color=1]
[u16 red_max=0x00ff] — BE
[u16 green_max=0x00ff] — BE
[u16 blue_max=0x00ff] — BE
[u8 red_shift=16] — 0x10
[u8 green_shift=8] — 0x08
[u8 blue_shift=0] — 0x00
[3 bytes padding/zeros]
--- Standard RFB name ---
[u32 name_length] — BE, strlen("ATEN iKVM Server") = 16
[name_length bytes name] — "ATEN iKVM Server"
--- ATEN Extension ---
[4 bytes padding/zeros]
[u32 session_id] — BE, pthread_self() of this session thread
[u8 video_permission] — PrivilegeInfo+0x155
[u8 kbd_mouse_permission] — PrivilegeInfo+0x156
[u8 kick_permission] — PrivilegeInfo+0x157
[u8 power_permission] — PrivilegeInfo+0x158
The total ServerInit message is approximately 48 bytes (standard RFB fields + name + ATEN extension). The framebuffer dimensions (480×640) are hardcoded defaults — the actual resolution is communicated through FramebufferUpdate responses.
The session_id (pthread_self value) is used by the client to identify itself in kick requests and session management.
Session Lifecycle
Main Accept Loop (0xacbc)
- Read IKVM port from Platform-Specific shared memory (default 5900)
- Install SIGUSR2 handler (for preview captures)
- Install SIGUSR1 handler (for session kills)
- Enter infinite accept loop:
a. Allocate a connection descriptor (0x20 bytes with mutex)
b. Allocate a connection info struct (0x14 bytes)
c. Call
InitConnectionDescriptor— stores socket fd and DeviceManager ptr d. Spawn a detached accept thread (SpawnDetachedThreadat 0xaf54) e. Wait (polling with 1s sleep) until the accept thread signals readiness (sets flag at descriptor+0x04 to non-zero) f. Print "GetVaildSession=true" and unlock the descriptor mutex g. Loop back to accept next connection
The accept thread (entry at LAB_0000a620) calls the socket accept wrapper and on success proceeds to the per-session handler.
Per-Session Handler (0xd334)
Each accepted connection runs in its own detached thread:
- Create Stream: Allocates a 0x610-byte Stream object, connects it to the socket
- Stream handshake: Calls
Stream::Connect(vtable dispatch at 0x12da8) - Set ready flag: Signals the accept loop that this connection is active
- Create RFBProtocol: Allocates a 0x28-byte RFBProtocol object:
- Creates SessionContext (0x50 bytes) with POSIX message queue
- Creates Auth object (4 bytes)
- Creates AESCrypto object (0x104 bytes)
- Adds session to the global linked list at DAT_0001e58c
- InitHandShake (vtable+0x08): Version exchange + security negotiation
- Authenticate (vtable+0x0c): Challenge/response + auth + ServerInit
- Post-auth setup:
EnforceMaxSessionsat 0xd260: If >4 sessions, kills oldest with cmd=4 "Kill Session Due to reach the max session number"NotifyPeersStatusUpdate: Sends own info to all peers with ≤ own privilegeNotifyPeersSessionEvent: Sends connection event to all peers with ≥ own privilege
- ProtocolHandler (vtable+0x10): Enters infinite message dispatch loop
- Cleanup (on exception/disconnect):
- Exception is caught, session is removed from linked list
CleanupSessionMQat 0xd608: Frees PrivilegeInfo, closes and unlinks message queue
Session Kill Mechanisms
SIGUSR1 handler (0xa9dc) — triggered by external processes (web UI, CGI):
- Reads
/tmp/killsessfile content "-1": CallsKillAllSessions— sends cmd=6 "Kill Session Due to Change Port" to all sessions (only if >1 active)"-2": CallsResetIPHandler— cleans up server socket, exits process- Other: Matches the string against usernames in the session list (via
PrivilegeInfo+0x40), sends cmd=3 "Kill Session Due to Web Logout" to matching session
Kick request (message type 0x38, handler at 0xfa54):
- Client sends:
[u32 target_thread_id] [u32 kick_cmd] [64 bytes reason_string] - Server reads target_thread_id and kick_cmd, forwards the entire 0x104-byte message to the target session via its POSIX message queue
Disconnect commands via MQ: When a session receives a command with code 0 or 4 from its message queue, the ProcessCommandQueue function throws an exception that terminates the session thread.
Notify Peers
NotifyPeersSessionEvent (0xce74, param_2=event_type):
- Iterates the session linked list
- For each peer with IPMI privilege ≤ this session's privilege: sends a message containing
[event_type, thread_id, username, display_name]via the peer's message queue
NotifyPeersStatusUpdate (0xcf58, param_2=status_type):
- Iterates the session linked list
- For each peer whose thread_id differs from this session: if peer's privilege ≤ this session's privilege, sends status via message queue
Both functions format the message as: [u32 cmd] [256 bytes data] where data contains sprintf("%d %s %s", thread_id, username, display_name).
Message Queue IPC
Queue Creation
Each session thread creates a POSIX message queue in InitSessionContext (0xd9d0):
- Name:
"/<thread_id_hex>"(e.g.,/4001c460) - Flags:
O_CREAT | O_RDWR | O_EXCL(0x842) - Mode:
0666 - Max message size: 0x104 bytes (260)
- Max messages: 0x40 (64)
Message Format
Messages sent through queues are 0x104 bytes:
[u32 command_code] — identifies the message type
[256 bytes payload] — command-specific data
Known command codes:
| Code | Meaning | Source |
|---|---|---|
| 0 | Disconnect (immediate) | External kill |
| 1 | Session connected notification | NotifyPeersSessionEvent |
| 3 | Kill: web logout | SIGUSR1 handler, killsess by username |
| 4 | Kill: max sessions / kick request | EnforceMaxSessions, kick forwarding |
| 6 | Kill: port change | KillAllSessions |
| 7 | KeepAlive/power status broadcast | KeepAlive handler |
ProcessCommandQueue (0x1069c)
Called at the top of every ProtocolHandler iteration, before reading any client message:
- Try non-blocking mq_receive on the session's message queue
- If message received:
- Command 0: Throw exception → disconnect
- Command 4: Send type 0x39 response to client, then throw exception → disconnect
- Other: Enqueue in the pending command linked list (RFBProtocol+0x14)
- If no message in queue: drain the pending command list:
- For each pending command: send type 0x39 response to client with
[u32 count] [u32 cmd] [256 bytes payload] - If command is 4: throw exception after sending → disconnect
- For each pending command: send type 0x39 response to client with
- Return 1 (continue processing)
The type 0x39 client response format:
Server → Client:
[u8 type=0x39]
[u32 pending_count] — BE, number of remaining pending commands
[u32 command_code] — BE, the command code from the queue message
[256 bytes payload] — raw payload from the queue message
KeepAlive Broadcast (0xfb70)
Triggered when the server handles a KeepAlive message (type 0x16) from any client:
- Gets current keyboard/HID status via DeviceManager vtable+0x44 (KeyboardDevice dispatch)
- Compares with cached status byte at DAT_0001e588
- If status changed: sends command 7 to ALL sessions (including self) via MQ with payload bytes
[0x20, 0x20, 0x00, kb_led_status]starting at payload offset 0 (the 4-byte command code is 7, followed by 256 bytes of payload where only the first 4 are meaningful) - Updates the cached status byte
When ProcessCommandQueue dispatches this as a type 0x39 message, the client receives it as a PrivilegeInfo with session_word_lo derived from bytes 0-3 and session_word_hi=7 (LED sync). The Java privilegeCtrl callback interprets hi=7 as LED status sync: data[3] bits encode NumLock (bit 0), CapsLock (bit 1), ScrollLock (bit 2).
Permission System
Permission Filter (0xf130)
Called in the ProtocolHandler dispatch loop for every client message. Returns true to DENY access (causing the message payload to be skipped). Returns false to ALLOW the handler to process the message.
Permission checks by message type:
| Permission Byte | Message Types | Filter Logic |
|---|---|---|
| +0x155 (Video) | 0x03 FBUpdateRequest, 0x19 GetCursorPos, 0x33 GetVideoInfo, 0x14/0x18/0x32 VideoControl, 0x15 (ignored msg) | Denied if byte != 1 |
| +0x156 (Kbd/Mouse) | 0x04 KeyEvent, 0x05 PointerEvent, 0x07 MouseSync, 0x08 SetEncoding, 0x34/0x36 SetEncoding2/3, 0x35 GetKbdMouseInfo, 0x37 GetSessionInfo, 0x3a RequestRefresh | Denied if byte != 1 |
| +0x157 (Kick) | 0x38 SendKickRequest, 0x39 (response type, but checked) | Denied if byte != 1 |
| None (always allowed) | 0x16 KeepAlive, 0x1a PowerControl, 0x3b SetBandwidth, 0x3c GetViewerLang, 0x3d SetViewerLang, 0x3e GetSessionStatus | Always processed |
Permission byte value 1 means "granted" (filter returns false = don't deny), value 0 means "denied" (filter returns true = deny and skip). When denied, GetFilterLen (0xec40) returns the payload size so the server can skip the message body via StreamSkipBytes.
GetFilterLen (0xec40) — Message Payload Sizes
Returns the number of bytes to skip when a message is filtered (denied):
| Type | Payload Size | Description |
|---|---|---|
| 0x03 | 9 | FBUpdateRequest (1+2+2+2+2) |
| 0x04 | 17 | KeyEvent (1+2+4+8+2, with AES block) |
| 0x05 | 17 | PointerEvent (1+2+2+4+8, with AES block) |
| 0x07 | 2 | MouseSync (u16 value) |
| 0x08 | 1 | SetEncoding |
| 0x14 | 2 | VideoControl |
| 0x15 | 8 | (ignored message, 2×u32) |
| 0x16 | 1 | KeepAlive |
| 0x18 | 2 | VideoControl2 |
| 0x19 | 0 | GetCursorPos (no payload) |
| 0x32 | 4 | VideoControl3 (2×u16) |
| 0x33 | 0 | GetVideoInfo |
| 0x34 | 2 | SetEncoding2 |
| 0x35 | 0 | GetKbdMouseInfo |
| 0x36 | 2 | SetEncoding3 |
| 0x37 | 0 | GetSessionInfo |
| 0x38 | 72 | SendKickRequest (0x48: u32+u32+64B) |
| 0x39 | 0 | (privilege response, no client payload) |
| 0x3a | 0 | RequestRefresh |
RFBProtocol Vtable (0x14b00, 5 entries)
| Offset | Address | Method |
|---|---|---|
| +0x00 | 0xfe3c | ~RFBProtocol() |
| +0x04 | 0xfc2c | ~RFBProtocol() (deleting) |
| +0x08 | 0xf098 | InitHandShake (VersionExchange + SecurityTypeNegotiation) |
| +0x0c | 0xf37c | Authenticate (challenge/response + auth + ServerInit) |
| +0x10 | 0x108b8 | ProtocolHandler (main message dispatch loop) |
Protocol Base Vtable (0x14b50, 5 entries)
| Offset | Address | Method |
|---|---|---|
| +0x00 | 0x10d00 | ~Protocol() |
| +0x04 | 0x10c18 | ~Protocol() (deleting) |
| +0x08 | 0xa060 | InitHandShake = pure virtual |
| +0x0c | 0xa060 | Authenticate = pure virtual |
| +0x10 | 0xa060 | ProtocolHandler = pure virtual |
ProtocolHandler Dispatch Details
ProtocolHandler (0x108b8)
The main message dispatch loop runs in an infinite do-while(true):
loop:
1. ProcessCommandQueue(ctx) — drain MQ, send type 0x39, throw on disconnect cmds
2. type = StreamRead8(stream) — read 1-byte message type from client
3. denied = PermissionFilter(ctx, type)
4. if denied:
skip_len = GetFilterLen(ctx, type)
StreamSkipBytes(stream, skip_len)
goto loop
5. switch(type):
case 0x03: FBUpdateRequest(ctx)
case 0x04: KeyEvent(ctx)
case 0x05: PointerEvent(ctx)
case 0x07: MouseSync(ctx)
case 0x08: SetEncoding(ctx)
case 0x14: VideoControl(ctx)
case 0x15: IgnoredMessage(ctx) — reads 2×u32, discards
case 0x16: KeepAlive(ctx); KeepAliveBroadcast(ctx)
case 0x18: VideoControl2(ctx)
case 0x19: GetCursorPos(ctx)
case 0x1a: PowerControl(ctx)
case 0x32: VideoControl3(ctx)
case 0x33: GetVideoInfo(ctx)
case 0x34: SetEncoding2(ctx)
case 0x35: GetKbdMouseInfo(ctx)
case 0x36: SetEncoding3(ctx)
case 0x37: GetSessionInfo(ctx)
case 0x38: SendKickRequest(ctx)
case 0x3a: RequestRefresh(ctx)
case 0x3b: SetBandwidth(ctx)
case 0x3c: GetViewerLang(ctx)
case 0x3d: SetViewerLang(ctx)
case 0x3e: GetSessionStatus(ctx)
default: puts("ProtocolHandler: Error message type !!")
goto loop
The loop never exits normally. Termination is via C++ exception thrown by ProcessCommandQueue (on disconnect/kick commands) or by stream read failure.
Note: type 0x16 (KeepAlive) is the only message that calls two handlers sequentially: the KeepAlive echo handler followed by the KeepAlive broadcast to peers.
SetEncoding Handlers
Type 0x08 — SetEncoding (0xeb3c): Reads 1 byte (mouse mode). Calls DeviceManager::SetMouseMode(mode, 0).
Client → Server: [u8 type=0x08] [u8 mouse_mode]
Sets the mouse device to mode 1 (absolute), 2 (relative+scroll), or 3 (passthrough). The second parameter is hardcoded 0. Triggers USB hot-plug to re-enumerate with the appropriate HID descriptor.
Type 0x34 — SetEncoding2 (0xeb78): Reads 2 bytes. Calls DeviceManager vtable+0x20 (0xb944), which is a stub that returns immediately. The two bytes are effectively discarded.
Client → Server: [u8 type=0x34] [u8 param1] [u8 param2]
No-op on the server side. The client sends this but the server ignores the values.
Type 0x36 — SetEncoding3 (0xebcc): Reads 2 bytes. Calls DeviceManager::SetMouseMode(byte2, byte1).
Client → Server: [u8 type=0x36] [u8 extra_param] [u8 mouse_mode]
Sets mouse mode (second byte) and extra parameter (first byte). The extra parameter is stored at MouseDevice+0x18 but not used by the kernel driver. Triggers USB hot-plug.
RequestRefresh Handler (0xec20, type 0x3a)
No payload. Calls DeviceManager::DoHotPlug (vtable+0x40 → MouseDevice vtable+0x1c → 0xeae0), which issues ioctl(fd, ATEN_MOUSE_IOCHOTPLUG). This forces a USB disconnect/reconnect cycle on the virtual mouse, causing the host OS to re-enumerate the HID device.
Client → Server: [u8 type=0x3a]
GetSessionStatus Handler (0xee0c, type 0x3e)
Calls DeviceManager::GetKbLed (vtable+0x44) to get keyboard LED status, sends it back.
Client → Server: [u8 type=0x3e]
Server → Client: [u8 type=0x3e] [u8 kb_led_status]
The LED status byte has bits: 0=NumLock, 1=CapsLock, 2=ScrollLock.
GetSessionInfo Handler (0xf994, type 0x37)
Reads power status and mouse cursor info.
Client → Server: [u8 type=0x37]
Server → Client: [u8 type=0x37] [u8 mouse_extra_param] [u8 mouse_mode] [u8 power_status]
- Calls
UtilGetNowPowerStatus()→ u8 power status - Calls DeviceManager::GetMouseCursorPos (vtable+0x10) which does
ioctl(ATEN_MOUSE_IOCGETMODE)and returns {mouse_mode, extra_param} mouse_mode: 1=absolute, 2=relative+scroll, 3=passthroughpower_status: from IPMI power state query
SetBandwidth Handler (0xfa10, type 0x3b)
Reads 3 u32 values (big-endian) and configures the stream's bandwidth limiters.
Client → Server: [u8 type=0x3b] [u32 limit1] [u32 limit2] [u32 period]
Calls SetBandwidthLimits(stream, limit1, limit2, period) at 0x13128, which delegates to the stream's read and write bandwidth counter objects.
PowerControl Handler (0xfae4, type 0x1a)
Reads 1 byte (power code), dispatches to IPMI power utilities.
Client → Server: [u8 type=0x1a] [u8 power_code]
| Code | Action | Library Call |
|---|---|---|
| 0 | Hard power off | UtilPowerDown() |
| 1 | Power on | UtilPowerUp() |
| 2 | Hard reset | UtilPowerReset() |
| 3 | ACPI soft power off | UtilSoftPowerDown() |
| other | Error (prints "Error Power Control Request!!") | — |
No response is sent. The power action is performed and a debug message is printed.
SetViewerLang Handler (0xfaa4, type 0x3d)
Reads 2 u32 values (big-endian), stores in PrivilegeInfo+0x15c and +0x160, then updates the persistent /nv/IKVMViewerLang file.
Client → Server: [u8 type=0x3d] [u32 lang_param1] [u32 lang_param2]
The handler reads /nv/IKVMViewerLang line by line (format: username param1 param2), replaces the line matching the current session's username with the new values, writes to /tmp/TempViewerLang, then mv -f /tmp/TempViewerLang /nv/IKVMViewerLang. No response.
Type 0x15 Handler (0xf7b0)
Reads 2 u32 values (big-endian) and discards them. No response, no side effects.
Client → Server: [u8 type=0x15] [u32 unknown1] [u32 unknown2]
Purpose unknown. Payload is 8 bytes. Requires video permission.
Complete Message Type Reference
Client → Server Messages
| Type | Name | Payload (after type byte) | Permission | Handler |
|---|---|---|---|---|
| 0x03 | FBUpdateRequest | [u8 incremental] [u16 x] [u16 y] [u16 w] [u16 h] (9B) |
Video | 0xf610 |
| 0x04 | KeyEvent | [u8 enc_flag] [16B payload] (17B) |
Kbd/Mouse | 0xf8c4 |
| 0x05 | PointerEvent | [u8 enc_flag] [16B payload] (17B) |
Kbd/Mouse | 0xf7d0 |
| 0x07 | MouseSync | [u16 value=0x0780] (2B) |
Kbd/Mouse | 0xf5dc |
| 0x08 | SetEncoding | [u8 mouse_mode] (1B) |
Kbd/Mouse | 0xeb3c |
| 0x14 | VideoControl | [u16 param] (2B) |
Video | 0xf55c |
| 0x15 | (ignored) | [u32 val1] [u32 val2] (8B) |
Video | 0xf7b0 |
| 0x16 | KeepAlive | [u8 status] (1B) |
None | 0xef10 |
| 0x18 | VideoControl2 | [u16 param] (2B) |
Video | 0xf4dc |
| 0x19 | GetCursorPos | (none, 0B) | Video | 0xef58 |
| 0x1a | PowerControl | [u8 power_code] (1B) |
None | 0xfae4 |
| 0x32 | VideoControl3 | [u16 param1] [u16 param2] (4B) |
Video | 0xf44c |
| 0x33 | GetVideoInfo | (none, 0B) | Video | 0xf0cc |
| 0x34 | SetEncoding2 | [u8 param1] [u8 param2] (2B) |
Kbd/Mouse | 0xeb78 |
| 0x35 | GetKbdMouseInfo | (none, 0B) | Kbd/Mouse | 0xeeac |
| 0x36 | SetEncoding3 | [u8 extra] [u8 mouse_mode] (2B) |
Kbd/Mouse | 0xebcc |
| 0x37 | GetSessionInfo | (none, 0B) | Kbd/Mouse | 0xf994 |
| 0x38 | SendKickRequest | [u32 target_tid] [u32 cmd] [64B reason] (72B) |
Kick | 0xfa54 |
| 0x3a | RequestRefresh | (none, 0B) | Kbd/Mouse | 0xec20 |
| 0x3b | SetBandwidth | [u32 limit1] [u32 limit2] [u32 period] (12B) |
None | 0xfa10 |
| 0x3c | GetViewerLang | (none, 0B) | None | 0xee5c |
| 0x3d | SetViewerLang | [u32 param1] [u32 param2] (8B) |
None | 0xfaa4 |
| 0x3e | GetSessionStatus | (none, 0B) | None | 0xee0c |
Permission "None" means always allowed regardless of session privilege.
All multi-byte integers in client messages are big-endian. KeyEvent and PointerEvent have an additional AES encryption envelope (see AES Encryption section).
Server → Client Messages
| Type | Name | Payload | Trigger |
|---|---|---|---|
| 0x00 | FramebufferUpdate | See format below | Response to type 0x03 |
| 0x04 | CursorPos | [5×u32 fields] [optional cursor data] |
Response to type 0x19 |
| 0x16 | KeepAlive | [u8 status] |
Response to type 0x16 |
| 0x33 | VideoInfo | [u16 width] [u16 height] |
Response to type 0x33 |
| 0x35 | KbdMouseInfo | [u8 info2] [u8 info1] (2B only, see discrepancy note) |
Response to type 0x35 |
| 0x37 | SessionInfo | [u8 mouse_extra] [u8 mouse_mode] [u8 power_status] |
Response to type 0x37 |
| 0x39 | PrivilegeCommand | [u32 count] [u32 cmd] [256B payload] |
From ProcessCommandQueue |
| 0x3c | ViewerLang | [u32 param1] [u32 param2] |
Response to type 0x3c |
| 0x3e | SessionStatus | [u8 kb_led_status] |
Response to type 0x3e |
The ServerInit message (sent during auth) is documented in the Authentication section.
Server Response Formats
Type 0x00 — FramebufferUpdate
Sent in response to FBUpdateRequest (type 0x03). The ATEN-extended RFB FramebufferUpdate contains a single rectangle with encoding 0x59 (Hermon).
[u8 type = 0x00]
[u8 padding = 0x00] — 1 byte zeros
[u16 num_rects = 1] — BE, always 1
[u16 x = 0] — BE
[u16 y = 0] — BE
[u16 width] — BE, pixels (negative = no signal)
[u16 height] — BE, pixels (negative = no signal)
[u32 encoding = 0x59] — BE, ATEN Hermon
[u32 subrects_info] — BE, tile count (diff) or mode (full)
[u32 data_length] — BE, byte count of frame data
[data_length bytes] — Hermon frame data with 10-byte sub-header
Total header: 24 bytes before frame data.
Type 0x04 — CursorPos
Sent in response to GetCursorPos (type 0x19). The handler calls DeviceManager::CaptureForClient which returns cursor position and optional cursor shape data.
[u8 type = 0x04]
[u32 field1] — BE
[u32 field2] — BE
[u32 field3] — BE
[u32 field4] — BE
[u32 cursor_type] — BE (1 = cursor data follows)
[if cursor_type == 1 and cursor_data_size > 0:]
[cursor_data_size bytes] — raw cursor shape data
Total: 17 bytes minimum, plus optional cursor data.
Type 0x16 — KeepAlive
Echoed back to the client with the same status byte the client sent.
[u8 type = 0x16]
[u8 status] — echoed from client's KeepAlive request
Total: 2 bytes.
Type 0x33 — VideoInfo
Returns current video resolution from the VideoCaptureDevice.
[u8 type = 0x33]
[u16 width] — BE, current capture width in pixels
[u16 height] — BE, current capture height in pixels
Total: 5 bytes. Width and height come from DeviceManager::GetVideoInfo which reads the VCD resolution struct.
Type 0x35 — KbdMouseInfo
Returns keyboard info state (previously set via SetKeyboardInfo).
[u8 type = 0x35]
[u8 info_byte_2] — low byte of KeyboardDevice+0x18
[u8 info_byte_1] — low byte of KeyboardDevice+0x14
Total: 3 bytes. These values are set by type 0x34 (SetEncoding2) via KeyboardDevice::SetKeyboardInfo, but the vtable+0x20 call is actually a stub, so these values remain at their initial state (0).
Client/server discrepancy: The client expects type 0x35 to carry 6 bytes (type + 2 keyboard info bytes + 3 mouse info bytes), calling ProcKeyboardInfo then ProcMouseInfo in sequence. However, the server only sends 3 bytes (type + 2 keyboard info bytes) and does NOT include mouse info. If the client sends a type 0x35 request, it will attempt to read 3 more bytes of mouse info that the server never sent, causing a protocol desync. In practice, the client sends type 0x35 as a 1-byte request and the server responds, but the client's handler expects the combined format from the server-to-client jump table entry at 0x001183d0 which calls both ProcKeyboardInfo and ProcMouseInfo. This appears to be a firmware bug — the server should send mouse info as well.
Type 0x37 — SessionInfo
Returns mouse mode and server power status.
[u8 type = 0x37]
[u8 mouse_extra_param] — MouseDevice+0x18 (set by SetEncoding3)
[u8 mouse_mode] — 1=absolute, 2=relative+scroll, 3=passthrough
[u8 power_status] — from UtilGetNowPowerStatus() (0=off, 1=on)
Total: 4 bytes.
Type 0x39 — PrivilegeCommand
Sent from ProcessCommandQueue when a session receives inter-session messages.
[u8 type = 0x39]
[u32 pending_count] — BE, remaining pending commands in queue
[u32 command_code] — BE, MQ command code (see MQ IPC section)
[256 bytes payload] — raw payload from MQ message
Total: 265 bytes. Command codes 0 and 4 cause session disconnect after sending. Other codes (1=session connected, 3=web logout, 6=port change, 7=keepalive broadcast) are informational.
Type 0x3C — ViewerLang
Returns the viewer language settings for this session's user.
[u8 type = 0x3C]
[u32 lang_param1] — BE, from PrivilegeInfo+0x15c
[u32 lang_param2] — BE, from PrivilegeInfo+0x160
Total: 9 bytes. Values loaded from /nv/IKVMViewerLang during authentication.
Type 0x3E — SessionStatus
Returns keyboard LED status.
[u8 type = 0x3E]
[u8 kb_led_status] — bit 0=NumLock, bit 1=CapsLock, bit 2=ScrollLock
Total: 2 bytes.
libipmicrypt.so — Crypto Library
- Path:
original_binaries/server/180000/lib/libipmicrypt.so - Ghidra name:
/180000/lib/libipmicrypt.so-4d09a1 - Type: ARM 32-bit LE ELF shared object, stripped
- Source: Custom AES implementation (matches XySSL/PolarSSL/mbed TLS)
Exported Functions
| Address | Function | Purpose |
|---|---|---|
| 0x10eb4 | rotr8 |
Byte rotation (8 bytes) |
| 0x10ebc | xtime |
GF(2^8) multiply (28 bytes) |
| 0x10ed8 | aes_gen_tables |
Generate S-box and round tables |
| 0x11484 | aes_set_key |
Key expansion. Param3 = bits (128/192/256) |
| 0x11bcc | aes_encrypt |
Single-block ECB encrypt (16 bytes) |
| 0x122ac | aes_decrypt |
Single-block ECB decrypt (16 bytes) |
| 0x11a24 | aes_found |
Forward round function |
| 0x120fc | aes_rround |
Inverse round function |
| 0x127b4 | Xor |
16-byte XOR helper |
| 0x127e0 | AES_Codec_Fun |
CBC single-block wrapper |
| 0x128ac | HMAC_MD5 |
HMAC-MD5 (428 bytes) |
| 0x12a58 | HMAC_SHA1 |
HMAC-SHA1 (428 bytes) |
| various | MD5Init/Update/Final |
MD5 hash |
| various | MD2Init/Update/Final |
MD2 hash |
| various | sha1_starts/process/update/finish |
SHA-1 hash |
AES Context Layout (0x204 bytes = 0x81 × 4 + 4)
| Offset | Field |
|---|---|
| 0x000-0x0FF | Encryption round keys (forward schedule) |
| 0x100-0x1FF | Decryption round keys (inverse schedule) |
| 0x200 | Number of rounds (10/12/14) |
Kernel Modules
vcddev.ko — Video Capture Device
- Path:
original_binaries/server/180000/bin/module/vcddev.ko - Ghidra name:
/180000/bin/module/vcddev.ko-179d04 - Author: Winbond Limited (Nuvoton)
- Source:
vcddev.c - Device:
/dev/vcd(major 25, char device) - IRQ: 22 (0x16)
- NOT stripped — full symbol table available
Module Init (init_module at 0x10280)
- Hardware detection: Writes 0xFFFFFFFF to
vcd+0x08andvcd+0x0c, reads back. Expected values:0xFE00FE00(-0x1ff0200) and0x07FF07FF. If mismatch, prints "VCD: hardware not present" and returns-ENODEV. - Register char device:
register_chrdev(25, "vcd", &vcddev_fops). - Request IRQ:
request_irq(22, vcddev_interrupt, SA_SHIRQ, "vcd", NULL).
File Operations
The module uses a two-level fops structure:
vcddev_fops (outer, at 0x10804) — registered with the kernel:
open→vcddev_open(0x101e0): validates minor number, redirects tovcd_fopsopen
vcd_fops (inner, at 0x1087c) — the actual operations:
open→vcd_open(0x10000): returns 0 (no-op)release→vcd_release(0x10014): clearsvcd+0x20(interrupt mask register), returns 0read→vcd_read(0x1003c): blocking read for capture completionpoll→vcd_poll(0x1014c): poll for capture done (returns POLLIN|POLLRDNORM)mmap→vcd_mmap(0x101cc): returns 0 (stub — the actual mmap is done in userspace via/dev/mem)
vcd_read (0x1003c)
Blocks until capture is done:
- If
O_NONBLOCKis set andcaptureDone == 0: returns-EAGAIN - Otherwise, enters a wait loop on
vcd_queue(wait queue):- Calls
prepare_to_wait(&vcd_queue, &wait, TASK_INTERRUPTIBLE) - If
captureDone != 0: breaks - If signal pending: returns
-ERESTARTSYS - Calls
schedule_timeout(50)(50 jiffies = ~500ms at HZ=100)
- Calls
- Clears
captureDone = 0 - Writes 0 to the output buffer (1 byte), returns 50 (success)
vcd_poll (0x1014c)
- If
invalidateCache != 0: clears it - If
captureDone == 0: writes 9 tovcd+0x20(enables capture-done interrupt bits 0 and 3) - Calls
poll_wait(filp, &vcd_queue, table) - If
captureDone != 0: returnsPOLLIN | POLLRDNORM(0x41), setsinvalidateCache = 1 - Otherwise: returns 0 (not ready)
Interrupt Handler (vcddev_interrupt at 0x10228)
Checks VCD status register vcd+0x1c. If bits 0 or 3 are set (capture complete or valid capture):
- Clears interrupt mask: writes 0 to
vcd+0x20 - Sets
captureDone = 1 - Calls
__wake_up(&vcd_queue, TASK_NORMAL, 1, NULL)to wake blocked readers
Returns IRQ_HANDLED (1) always.
Global Variables
| Symbol | Address | Purpose |
|---|---|---|
vcd |
0x108f0 | VCD device structure (base: physical 0xF0028000) |
vcd_queue |
0x108f4 | Wait queue head for capture completion |
invalidateCache |
0x108fc | Flag: 1 when cache needs invalidation after poll |
captureDone |
0x10a40 | Flag: 1 when capture is complete, cleared by read |
Hardware Register Access Model
The vcd structure at 0x108f0 holds the VCD hardware base address (0xF0028000 physical). The kernel module does NOT provide an ioctl interface — all VCD register manipulation is done directly from userspace by the ikvmserver process via /dev/mem mmap. The kernel module's role is limited to:
- Interrupt handling — detecting capture completion via IRQ 22
- Signaling completion — via wait queue to the userspace poll/read interface
The ikvmserver VCD functions at 0x11368-0x12a6c directly read/write the memory-mapped VCD register block (pointed to by DAT_0001e4b0 in the ikvmserver address space, mapped via /dev/mem).
Module Cleanup (cleanup_module at 0x10374)
- Clears interrupt mask: writes 0 to
vcd+0x20 - Frees IRQ 22
- Unregisters char device major 25
usb_hid.ko — Virtual USB HID
- Path:
original_binaries/server/180000/bin/module/usb_hid.ko - Ghidra name:
/180000/bin/module/usb_hid.ko-f1446b - Author: "ATEN, Bobby"
- Source:
wpcm450_hid.c - Devices:
/dev/keyboard(minor 0),/dev/mouse(minor 1) - NOT stripped
| Symbol | Purpose |
|---|---|
wpcm450_usb_write |
Write handler — dispatches to keyboard (minor 0) or mouse (minor 1) work queues |
wpcm450_usb_ioctl |
Ioctl handler — dispatches via keyboradmouse_ioctl_rountin table |
keyboardAction |
Keyboard work queue handler — builds HID report, sends on EP1 |
mouseAction |
Mouse work queue handler — builds HID report, sends on EP2 |
mouseSync |
Resets mouse position to origin (sends min-value relative moves or abs (0,1,0,1)) |
km_insert |
Insert keycode into kb_code slots [2..7] |
km_remove |
Remove keycode from kb_code slots [2..7], shift remaining |
Complement2 |
Identity function (no-op byte cast) |
set_protocol |
USB HID protocol mode set |
wpcm450_service_ep1 |
EP1 ISR completion — releases mouse_data_mutex, wakes mswaitq |
wpcm450_service_ep2 |
EP2 ISR completion — releases keyboard_data_mutex, wakes kbwaitq |
wpcm450_hotplug |
ioctl 0x40046201: USB disconnect/reconnect sequence (500ms+1500ms) |
wpcm450_get_mode |
ioctl 0x80046202: Read current mousetype to user space |
wpcm450_set_mode |
ioctl 0x40046203: Write mousetype from user space |
wpcm450_resync |
ioctl 0x40046204: Queue a mouseSync work item |
wpcm450_getkbled |
ioctl 0x80046205: Read kb_led_status (1 byte) to user space |
hid_reset |
Reset a USB HID endpoint |
resolutionX |
Global: current display width (set by mouseAction from write data) |
resolutionY |
Global: current display height (set by mouseAction from write data) |
mousetype |
Global: mouse mode (1=absolute, 2=relative+scroll, 3=passthrough), default=2 |
hid_support_flag |
Global: if 1 and mousetype!=3, use absolute mouse mode |
kb_code |
Global 8-byte buffer: USB HID keyboard report being built |
ms_code |
Global 6-byte buffer: USB HID mouse report being built |
kb_led_status |
Global: current keyboard LED state (NumLock/CapsLock/ScrollLock bits) |
xOld / yOld |
Global: last sent mouse position (for relative mode delta calculation) |
ikvm_vmass.ko — Virtual USB Mass Storage
- Path:
original_binaries/server/180000/bin/module/ikvm_vmass.ko - Ghidra name:
/180000/bin/module/ikvm_vmass.ko-070d10 - Source:
wpcm450_vmass.c - NOT stripped (369 symbols)
- USB Vendor/Product: ATEN 0x0EA0 / 0x2168
- USB Device Class: Miscellaneous (0xEF), Interface Association Descriptor (subclass 0x02, protocol 0x01)
Architecture Overview
The module implements a USB 2.0 device controller driver for the WPCM450's on-chip USB hardware. It presents a USB Mass Storage (Bulk-Only) device to the host, with SCSI command handling delegated to userspace (the virtual media daemon). The module handles:
- USB device enumeration (Chapter 9 control transfers)
- Bulk endpoint data transfer (EP1 OUT for CBW, EP2 IN for data/CSW)
- DMA buffer management
- Character device interface for userspace communication
Module Init (wpcm450_USB20_init_module at 0x12600)
- DMA allocation:
dma_alloc_coherent(NULL, 0x152000, &USB20_PhyAddr, GFP_KERNEL)— allocates 1.3MB coherent DMA buffer for USB transfer descriptors and data - Char device registration: Allocates 4 minor numbers (using
alloc_chrdev_regionorregister_chrdev_region) - Device initialization: Allocates
wpcm450_devices20struct (0x280 bytes), initializes 5 cdev entries - Per-device setup: For each of 5 devices (minors 0-4):
- Allocates a 0x5000-byte (20KB) non-cached SCSI buffer at device+0x178
- Initializes cdev with
wpcm450_usb20_fops - Sets EP max packet size: 2 (initially), later 0x80 (128 bytes)
- Wait queue: Initializes
delaywaitqfor startup delay
Ioctl Interface (wpcm450_usb20_ioctl at 0x120e8)
| Ioctl Code | Direction | Purpose |
|---|---|---|
| 0xC0085501 | Read/Write | Register device / Set descriptors — main setup ioctl |
| 0x40085502 | Write | Disconnect USB — unbinds gadget, shuts down USB controller |
| 0x40085503 | Write | Connect USB — starts USB device controller |
Ioctl magic number: 0x55 ('U'), size field: 8 bytes.
Ioctl 0xC0085501 — Register Device / Set Descriptors:
Copies a 0x20-byte control struct from userspace. The first byte determines behavior:
- byte[0] == 2: Calls
RegisterDevice()to allocate a device slot (max 2 slots, returns 0 or 1). Incrementsat_b_USB_count. - byte[0] < 2: Copies 0x520 bytes of USB descriptor data from a userspace pointer (at control+0x10) into
at_a_vuDevRespData. Allocates a 0xA0000-byte (640KB) SCSI data buffer at device+0x78. If this is the second device (at_b_USB_count == 1), callsikvm_vmass_gadget_register_driverto bind the USB gadget driver.
The 0x520-byte descriptor data (at_a_vuDevRespData at 0x1629c) contains the complete USB device, configuration, interface, and endpoint descriptors that will be presented to the host. The userspace daemon constructs these descriptors to define the virtual CD-ROM device identity.
Ioctl 0x40085502 — USB Disconnect: Unbinds the gadget driver, shuts down the USB controller, resets all global state. Wakes all wait queues with error code 0xF1.
Ioctl 0x40085503 — USB Connect:
Starts the USB device controller via _usb20_device_run. Increments g_init_cnt.
Data Flow: Network → USB
The virtual media daemon (userspace) communicates with the kernel module via read/write on the char device:
Read (wpcm450_usb20_read at 0x11928) — receives SCSI CBW from host:
- If
TEST_ENABLED == 0: returns 0 (USB not active) - Startup delay: waits on
delaywaitqfor up to 5 seconds (50 ticks) on first call - Reads 8-byte header from userspace (device index + flags)
- Calls
_usb20_device_recv_datato initiate a USB OUT transfer on the bulk endpoint - Waits on the device's OUT wait queue until
at_b_usb_out_ackis set by the ISR - If 0xF1 (disconnect): returns 0
- Otherwise: copies the received USB data to the userspace buffer (8-byte header + payload)
Write (wpcm450_usb20_write at 0x11cd4) — sends SCSI response data to host:
- Reads 8-byte header from userspace (device index, endpoint index)
- Copies payload data into the device's DMA buffer (device+0x78)
- If payload is exactly 13 bytes (0x0D): this is a CSW (Command Status Wrapper). The first byte is set to 0x55 ('U' — CSW signature), the device state is set to 2 (ready for new CBW), and a semaphore is released.
- Calls
_usb20_device_send_datato transmit data on the bulk IN endpoint - Waits on the device's IN wait queue until
at_b_usb_in_ackis set - Supports multi-packet transfers: large SCSI IN data is accumulated in the buffer via
at_dw_SCSIInCont/at_dw_SCSIInLencounters, sent in max-packet-size chunks
SCSI Command Parsing (Endpoint Service, 0x11158)
The wpcm450_USB20_service_ep1 function (EP1 completion callback) parses incoming CBW packets:
- If received length == 31 (0x1F): this is a CBW (USB Mass Storage Bulk-Only Transport command wrapper)
- Sets device state to 0 (processing command)
- Reads direction flag from CBW byte 12 (bit 7: 0=OUT, 1=IN)
- Reads SCSI opcode from CBW byte 15
- Dispatches based on opcode:
- 0x28 (READ_10), 0xBE (READ_CD): Sets
at_dw_SCSIInLenfrom CBW transfer length (data IN to host) - 0x20 (proprietary?), 0x2A (WRITE_10): Sets
at_dw_SCSIOutLenfrom CBW transfer length (data OUT from host) - All others: Sets both lengths to 0 (no data phase)
- 0x28 (READ_10), 0xBE (READ_CD): Sets
- If received length != 31 but < max packet size: short packet, sets state to 1 (data received)
- Wakes the OUT wait queue and sets ack flags
The second device (EP4/EP5 pair) mirrors this logic in wpcm450_USB20_service_ep4 and wpcm450_USB20_service_ep5.
USB Gadget Driver Registration
ikvm_vmass_gadget_register_driver (0x10000) validates the driver struct and calls the bind callback. The bind function (wpcm450_USB20_udc_bind at 0x100cc):
- Initializes the USB 2.0 device controller:
_usb20_device_init(2, &handle, 8)— 2 endpoints, 8 config slots - Registers endpoint service callbacks:
- EP0:
wpcm450_USB20_service_ep0(control/enumeration) - Service 0x10:
wpcm450_USB20_reset_ep0(bus reset) - Service 0x15:
wpcm450_udc_mstorage_service_speed(speed change) - EP1 (bulk OUT):
wpcm450_USB20_service_ep1(CBW receive, device 0) - EP2 (bulk IN):
wpcm450_USB20_service_ep2(data/CSW send, device 0) - EP4 (bulk OUT):
wpcm450_USB20_service_ep4(CBW receive, device 1) - EP5 (bulk IN):
wpcm450_USB20_service_ep5(data/CSW send, device 1)
- EP0:
- Flushes ARM926 caches
USB Enumeration (EP0 Service, 0x10fb4)
Standard USB Chapter 9 control transfer handling. Dispatches on bmRequestType type field:
- Standard (0x00): Full Chapter 9 implementation: GetStatus, ClearFeature, SetFeature, SetAddress, GetDescriptor, SetDescriptor, GetConfig, SetConfig, GetInterface, SetInterface, SynchFrame
- Class (0x20):
USB20_ch9Class— handles mass storage class requests (e.g., Bulk-Only Mass Storage Reset = 0xFE). Responds with 0-length data. - Vendor (0x40): Silently ignored
Speed Negotiation (0x117f8)
wpcm450_udc_mstorage_service_speed adjusts endpoint max packet sizes:
- Full-speed (1): 64 bytes per packet (0x40)
- High-speed (2): 512 bytes per packet (0x200)
Updates both the hardware endpoint configuration and the descriptor data in at_a_vuDevRespData.
Dual-LUN Support
The module supports up to 2 LUNs (logical units), each with its own:
- Bulk OUT endpoint (EP1 for LUN 0, EP4 for LUN 1)
- Bulk IN endpoint (EP2 for LUN 0, EP5 for LUN 1)
- 640KB SCSI data buffer (device+0x78)
- Independent SCSI length tracking (
at_dw_SCSIInLen[0..1],at_dw_SCSIOutLen[0..1])
When 2 devices are registered (at_b_USB_count == 2), update_descriptor (0x120ac) patches the configuration descriptor to add the second interface.
Key Global Variables
| Symbol | Address | Purpose |
|---|---|---|
wpcm450_devices20 |
0x16ae0 | Pointer to device struct array (0x80 bytes per device, 5 entries) |
at_a_vuDevRespData |
0x1629c | USB descriptor data (0x520 bytes, set by ioctl from userspace) |
at_dw_SCSIOutLen |
0x16a40 | SCSI OUT (host→device) transfer length per LUN (4 bytes × 4) |
at_dw_SCSIInLen |
0x16a50 | SCSI IN (device→host) transfer length per LUN (4 bytes × 4) |
at_dw_SCSIInCont |
0x16a60 | SCSI IN bytes accumulated so far per LUN |
at_b_usb_out_ack |
0x16a78 | USB OUT transfer completion flags (per LUN) |
at_b_usb_in_ack |
0x16a7c | USB IN transfer completion flags (per LUN) |
at_b_USB_count |
0x16a30 | Number of registered USB devices (0-2) |
at_b_DevicesBitMap |
0x16a31 | Bitmap of allocated device slots |
USB20_LogAddr |
0x16a34 | DMA buffer logical (kernel virtual) address |
USB20_PhyAddr |
0x16a38 | DMA buffer physical address |
at_speed |
0x167bc | USB speed: 1=full, 2=high |
TEST_ENABLED |
0x16a24 | USB active flag (set by connect ioctl) |
g_init_cnt |
0x16a3c | USB controller initialization count |
g_usb_init_done |
0x16c5c | USB initialization complete flag |
delay_flag |
0x16a20 | Startup delay completion flag |
USB_Reset_Flag |
0x16a32 | Set on USB bus reset, cleared on next read/write |
Module Cleanup (wpcm450_USB20_cleanup_module at 0x1246c)
- Sets all ack flags to 0xF1 (error/disconnect sentinel)
- Wakes all wait queues (causes blocked reads/writes to fail)
- If USB was initialized: unbinds gadget, shuts down controller
- Deletes all 5 cdev entries
- Frees device struct
- Unregisters char device region
- Frees DMA buffer:
dma_free_coherent(NULL, 0x152000, OrigLogAddr, OrigPhyAddr)
Shared Libraries (ikvmserver dependencies)
All in original_binaries/server/180000/lib/:
| Library | Size | Purpose |
|---|---|---|
libutility.so |
700KB | Auth, power, iKVM, sensors — the big one |
libipmi.so |
213KB | IPMI protocol |
libldap_client.so |
275KB | LDAP authentication |
libradius_client.so |
35KB | RADIUS authentication |
libipmicrypt.so |
24KB | Custom AES/MD5/SHA1/HMAC |
libnvram.so |
31KB | Persistent storage |
libsys.so |
23KB | Shared memory, mutexes, queues |
libcgi.so |
29KB | CGI/web interface |
libcommon.so |
8KB | Timers, SDR, matching |
libchnl.so |
4KB | IPMI channel management |
libsol.so |
12KB | Serial-over-LAN |
libuart.so |
6KB | UART serial |
libupdate.so |
35KB | Firmware update |
libnet.so |
5KB | Network helpers |
libgpio.so |
5KB | GPIO control |
libwdt.so |
4KB | Watchdog timer |
Important libutility.so Exports (used by ikvmserver)
| Function | Signature | Purpose |
|---|---|---|
UtilAuthUser |
int UtilAuthUser(char *username, char *password) |
Returns user index 0-9 on success, >=10 on failure. Supports LDAP/RADIUS backends via libldap_client.so and libradius_client.so |
UtilGetNowPowerStatus |
int UtilGetNowPowerStatus(void) |
Returns u8 power status (0=off, 1=on), masked to 0xFF. Server wrapper at 0xe710 prints "DBPower Status == %d" |
UtilPowerDown |
void UtilPowerDown(void) |
Hard power off (IPMI chassis control) |
UtilPowerUp |
void UtilPowerUp(void) |
Power on (IPMI chassis control) |
UtilPowerReset |
void UtilPowerReset(void) |
Hard reset (IPMI chassis control) |
UtilSoftPowerDown |
void UtilSoftPowerDown(void) |
ACPI soft power off (IPMI chassis control) |
UtilSearchLanChannelNo |
int UtilSearchLanChannelNo(int start, int *channel) |
Finds LAN channel number for privilege lookup. Used to index into per-user privilege table |
UtilSaveSysEventLog |
void UtilSaveSysEventLog(int event_code) |
Write to System Event Log. ikvmserver uses event 6 (5th session) and event 3 (normal session) |
UtilResetiKVMOption |
void UtilResetiKVMOption(void) |
Reset iKVM configuration |
Configuration Files
| Path | Purpose |
|---|---|
180000/etc/defaults/service.conf |
IKVM_SERVICE=1, VM_SERVICE=1, ports |
180000/etc/defaults/factory.xml |
IKVMInfo: VideoEnable, KMEnable, KickEnable |
180000/etc/init.d/ikvmd |
ikvmserver start/stop script |
180000/etc/init.d/rcS |
Boot sequence (insmod vcddev.ko, usb_hid.ko, etc.) |
Web Interface (Client-Side)
At original_binaries/server/B80000/:
| Path | Purpose |
|---|---|
iKVM__V1.69.21.0x0.jar |
ATEN Java KVM viewer (main class: tw.com.aten.ikvm.KVMMain) |
liblinux_x86_64__V1.0.5.jar |
Native client library (contains libiKVM64.so) |
SOL__V0.5.11.jar |
Serial-over-LAN viewer |
cgi/CapturePreview.cgi |
Video frame capture for web preview |
cgi/url_redirect.cgi |
Generates JNLP files for Java Web Start |
page/man_ikvm.www |
iKVM launch page |
page/test.jnlp |
JNLP template |
Shared Memory Regions
The BMC uses shared memory for inter-process state:
| Accessor | Purpose |
|---|---|
GlobalVar_Shm_AT / GlobalVar_Shm_DT |
Global variables (attach/detach) |
NVRAM_Shm_AT / NVRAM_Shm_DT |
NVRAM persistent storage |
PS_Shm_AT |
Platform-specific configuration (iKVM port at +0x1b42) |
AES Crypto Summary
Both client (libiKVM64.so) and server (ikvmserver via libipmicrypt.so)
use identical hardcoded AES parameters:
| Property | Value |
|---|---|
| Algorithm | AES-128 |
| Mode | CBC (single 16-byte block, fresh static IV each call) |
| Key | 2b7e151628aed2a6abf7158809cf4f3c (NIST FIPS-197 Appendix B) |
| IV | 000102030405060708090a0b0c0d0e0f (sequential bytes 0-15) |
Because the IV is always reset to the same static value and only one block is ever processed, CBC degenerates to deterministic encryption (equivalent to ECB for single blocks). The same plaintext always produces the same ciphertext.
The 96-byte key data block at 0x14cf0 contains all three NIST test vector
keys (AES-128, AES-192, AES-256) but only the first 16 bytes (AES-128) are used.
Tips for Further RE
-
Kernel modules are NOT stripped —
vcddev.ko,usb_hid.ko,ikvm_vmass.koall have full symbol tables. Start here for hardware interface understanding. -
RTTI strings in ikvmserver give class hierarchy: search for strings matching the pattern
\d+[A-Z]to find class names. -
Virtual dispatch is pervasive in ikvmserver. The RFBProtocol vtable at
0x14b00has 5 entries: [dtor, dtor2, InitHandShake, Authenticate, ProtocolHandler]. The Auth vtable at0x14c64uses a thunk at0x10f48that strips thethispointer and forwards toAuthBackendat0xe638. The DeviceManager vtable at0x14518has 18 entries (0x00-0x44) that delegate to three sub-objects. Protocol+0x04 stores a pointer to&DAT_0001e560(the global DeviceManager instance), NOT a direct object pointer — requires double dereference to reach the vtable. -
Message queue IPC: Each session thread creates a POSIX message queue named
/<hex_threadid>. Messages are 0x104 bytes:[u32 cmd][256B payload]. Queue processing happens at the TOP of every ProtocolHandler iteration. Commands 0 and 4 cause immediate disconnect. Other commands are forwarded to the client as type 0x39 messages. -
Session files:
/tmp/sess_<username>format is<auth_status> <username> <privilege> <display_name> <counter>. These are created by the web UI and checked by ikvmserver for SSO. The counter is atomically incremented under flock when a session authenticates. -
Error strings in the binary are useful landmarks:
"ProtocolHandler: Error message type !!"— unknown client message"Empty IKVM Port in PS,use default port 5900"— port config"Kill Session Due to Web Logout"— session termination"video source:INT_VGA"— video capture init (in VideoCaptureDevice ctor)"VCD block not present"— no video capture hardware (in VCD::Init)"[Hermon] vcd_mode = 0x%x, %s mode"— Nuvoton VCD status (in WaitForValidResolution)"[Hermon] digital mode: h=%d, v=%d"— detected resolution"Can't malloc MEM"— frame buffer allocation failure"Wait for done timeout, break"— VCD capture timeout (in ResetCapture)"Wait valid resolution fails"— no valid video signal detected"PreviewHandler: Create pthread fail"— preview thread spawn failure"fopen /tmp/Snapshot.bmp error"— snapshot file write failure