vhost-device-console: Add initial implementation

The device was tested with:
1) Upstream QEMU's vhost-user-device

    qemu-system-x86_64  \
            <normal QEMU options> \
            -machine <machine options>,memory-backend=mem0 \
            -object memory-backend-memfd,id=mem0,size=<Guest RAM size> \ # size == -m size
            -chardev socket,id=con0,path=/tmp/console.sock0 \
            -device vhost-user-device-pci,chardev=con0,virtio-id=3,num_vqs=4,config_size=12 \
            ...

2) A new QEMU vhost-user-console device which can be found in the following repo:
- https://github.com/virtualopensystems/qemu/tree/vhu-console-rfc

For more information, please check the README.md file under
staging/vhost-device-console/.

Co-authored-by: dorindabassey <53014273+dorindabassey@users.noreply.github.com>
Signed-off-by: Timos Ampelikiotis <t.ampelikiotis@virtualopensystems.com>
This commit is contained in:
Timos Ampelikiotis 2024-01-09 13:39:34 +00:00 committed by Stefano Garzarella
parent f3d564fb60
commit 079d9024be
13 changed files with 1964 additions and 1 deletions

View file

@ -48,6 +48,7 @@ Here is the list of device backends in **staging**:
- [Video](https://github.com/rust-vmm/vhost-device/blob/main/staging/vhost-device-video/README.md)
- [Can](https://github.com/rust-vmm/vhost-device/blob/main/staging/vhost-device-can/README.md)
- [Console](https://github.com/rust-vmm/vhost-device/blob/main/staging/vhost-device-console/README.md)
<!--
Template:

View file

@ -3,4 +3,5 @@ resolver = "2"
members = [
"vhost-device-video",
"vhost-device-can",
"vhost-device-console",
]

View file

@ -1,5 +1,5 @@
{
"coverage_score": 63.00,
"coverage_score": 60.50,
"exclude_path": "",
"crate_features": ""
}

View file

@ -0,0 +1,15 @@
# Changelog
## Unreleased
### Added
### Changed
### Fixed
### Deprecated
## v0.1.0
First release

View file

@ -0,0 +1,37 @@
[package]
name = "vhost-device-console"
version = "0.1.0"
authors = ["Timos Ampelikiotis <t.ampelikiotis@virtualopensystems.com>"]
description = "vhost console backend device"
repository = "https://github.com/rust-vmm/vhost-device"
readme = "README.md"
keywords = ["console", "vhost", "virt", "backend"]
license = "Apache-2.0 OR BSD-3-Clause"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[features]
xen = ["vm-memory/xen", "vhost/xen", "vhost-user-backend/xen"]
[dependencies]
console = "0.15.7"
crossterm = "0.27.0"
nix = "0.26.4"
queues = "1.0.2"
clap = { version = "4.4", features = ["derive"] }
env_logger = "0.10"
epoll = "4.3"
log = "0.4"
thiserror = "1.0"
vhost = { version = "0.11", features = ["vhost-user-backend"] }
vhost-user-backend = "0.15"
virtio-bindings = "0.2.2"
virtio-queue = "0.12"
vm-memory = "0.14.1"
vmm-sys-util = "0.12"
[dev-dependencies]
assert_matches = "1.5"
virtio-queue = { version = "0.12", features = ["test-utils"] }
vm-memory = { version = "0.14.1", features = ["backend-mmap", "backend-atomic"] }

View file

@ -0,0 +1 @@
../../LICENSE-APACHE

View file

@ -0,0 +1 @@
../../LICENSE-BSD-3-Clause

View file

@ -0,0 +1,138 @@
# vhost-device-console - Console emulation backend daemon
## Description
This program is a vhost-user backend that emulates a VirtIO Console device.
The device's binary takes as parameters a socket path, a socket number which
is the number of connections, commonly used across all vhost-devices to
communicate with the vhost-user frontend devices, and the backend type
"nested" or "network".
The "nested" backend allows input/output to the guest console through the
current terminal.
The "network" backend creates a local TCP port (specified on vhost-device-console
arguments) and allows input/output to the guest console via that socket.
This program is tested with QEMU's `vhost-user-device-pci` device.
Examples' section below.
## Staging Device
This device will be in `staging` until we complete the following steps:
- [ ] Increase test coverage
- [ ] Support VIRTIO_CONSOLE_F_SIZE feature (optional)
- [ ] Support VIRTIO_CONSOLE_F_EMERG_WRITE feature (optional)
## Synopsis
```text
vhost-device-console --socket-path=<SOCKET_PATH>
```
## Options
.. program:: vhost-device-console
.. option:: -h, --help
Print help.
.. option:: -s, --socket-path=PATH
Location of vhost-user Unix domain sockets, this path will be suffixed with
0,1,2..socket_count-1.
.. option:: -p, --tcp-port=PORT_NUMBER
The localhost's port to be used for each guest, this part will be increased with
0,1,2..socket_count-1.
-- option:: -b, --backend=nested|network
The backend type vhost-device-console to be used. The current implementation
supports two types of backends: "nested", "network" (described above).
Note: The nested backend is selected by default and can be used only when
socket_count equals 1.
## Limitations
This device is still work-in-progress (WIP). The current version has been tested
with VIRTIO_CONSOLE_F_MULTIPORT, but only for one console (`max_nr_ports = 1`).
Also it does not yet support the VIRTIO_CONSOLE_F_EMERG_WRITE and
VIRTIO_CONSOLE_F_SIZE features.
## Features
The current device gives access to multiple QEMU guest by providing a login prompt
either by connecting to a localhost server port (network backend) or by creating an
nested command prompt in the current terminal (nested backend). This prompt appears
as soon as the guest is fully booted and gives the ability to user run command as a
in regular terminal.
## Examples
### Dependencies
For testing the device the required dependencies are:
- Linux:
- Set `CONFIG_VIRTIO_CONSOLE=y`
- QEMU (optional):
- A new vhost-user-console device has been implemented in the following repo:
- https://github.com/virtualopensystems/qemu/tree/vhu-console-rfc
### Test the device
The daemon should be started first:
```shell
host# vhost-device-console --socket-path=/tmp/console.sock --socket-count=1 \
--tcp-port=12345 --backend=network
```
>Note: In case the backend is "nested" there is no need to provide
"--socket-count" and "--tcp-port" parameters.
The QEMU invocation needs to create a chardev socket the device can
use to communicate as well as share the guests memory over a memfd.
There are two option for running QEMU with vhost-device-console:
1) Using `vhost-user-console-pci`:
```text
host# qemu-system \
<normal QEMU options> \
-machine <machine options>,memory-backend=mem0 \
-object memory-backend-memfd,id=mem0,size=<Guest RAM size> \ # size == -m size
-chardev socket,path=/tmp/console.sock0,id=con \
-device vhost-user-console-pci,chardev=con0,id=console \
...
```
> Note: For testing this scenario the reader needs to clone the QEMU version from the following repo
> which implements `vhost-user-console` device.
> - https://github.com/virtualopensystems/qemu/tree/vhu-console-rfc
2) Using `vhost-user-device-pci`:
```text
host# qemu-system \
<normal QEMU options> \
-machine <machine options>,memory-backend=mem0 \
-object memory-backend-memfd,id=mem0,size=<Guest RAM size> \ # size == -m size
-chardev socket,id=con0,path=/tmp/console.sock0 \
-device vhost-user-device-pci,chardev=con0,virtio-id=3,num_vqs=4,config_size=12 \
...
```
Eventually, the user can connect to the console by running:
```test
host# stty -icanon -echo && nc localhost 12345 && stty echo
```
>Note: `stty -icanon -echo` is used to force the tty layer to disable buffering and send / receive each character individually. After closing the connection please run `stty echo` so character are printed back on the local terminal console.
>Note: In case the backend is "nested" a nested terminal will be shown into
vhost-device-console terminal space.
## License
This project is licensed under either of
- [Apache License](http://www.apache.org/licenses/LICENSE-2.0), Version 2.0
- [BSD-3-Clause License](https://opensource.org/licenses/BSD-3-Clause)

View file

@ -0,0 +1,313 @@
// VIRTIO CONSOLE Emulation via vhost-user
//
// Copyright 2023-2024 VIRTUAL OPEN SYSTEMS SAS. All Rights Reserved.
// Timos Ampelikiotis <t.ampelikiotis@virtualopensystems.com>
//
// SPDX-License-Identifier: Apache-2.0 or BSD-3-Clause
use log::{error, info, warn};
use std::any::Any;
use std::collections::HashMap;
use std::path::PathBuf;
use std::sync::{Arc, RwLock};
use std::thread::Builder;
use thiserror::Error as ThisError;
use vhost_user_backend::VhostUserDaemon;
use vm_memory::{GuestMemoryAtomic, GuestMemoryMmap};
use crate::console::{BackendType, ConsoleController};
use crate::vhu_console::VhostUserConsoleBackend;
pub(crate) type Result<T> = std::result::Result<T, Error>;
#[derive(Debug, ThisError)]
/// Errors related to low level Console helpers
pub(crate) enum Error {
#[error("Invalid socket count: {0}")]
SocketCountInvalid(usize),
#[error("Could not create console backend: {0}")]
CouldNotCreateBackend(crate::vhu_console::Error),
#[error("Could not create daemon: {0}")]
CouldNotCreateDaemon(vhost_user_backend::Error),
#[error("Fatal error: {0}")]
ServeFailed(vhost_user_backend::Error),
#[error("Thread `{0}` panicked")]
ThreadPanic(String, Box<dyn Any + Send>),
#[error("Error using multiple sockets with Nested backend")]
WrongBackendSocket,
}
#[derive(PartialEq, Debug)]
pub struct VuConsoleConfig {
pub(crate) socket_path: PathBuf,
pub(crate) backend: BackendType,
pub(crate) tcp_port: String,
pub(crate) socket_count: u32,
}
impl VuConsoleConfig {
pub fn generate_socket_paths(&self) -> Vec<PathBuf> {
let socket_file_name = self
.socket_path
.file_name()
.expect("socket_path has no filename.");
let socket_file_parent = self
.socket_path
.parent()
.expect("socket_path has no parent directory.");
let make_socket_path = |i: u32| -> PathBuf {
let mut file_name = socket_file_name.to_os_string();
file_name.push(std::ffi::OsStr::new(&i.to_string()));
socket_file_parent.join(&file_name)
};
(0..self.socket_count).map(make_socket_path).collect()
}
pub fn generate_tcp_addrs(&self) -> Vec<String> {
let tcp_port_base = self.tcp_port.clone();
let make_tcp_port = |i: u32| -> String {
let port_num: u32 = tcp_port_base.clone().parse().unwrap();
"127.0.0.1:".to_owned() + &(port_num + i).to_string()
};
(0..self.socket_count).map(make_tcp_port).collect()
}
}
// This is the public API through which an external program starts the
/// vhost-device-console backend server.
pub(crate) fn start_backend_server(
socket: PathBuf,
tcp_addr: String,
backend: BackendType,
) -> Result<()> {
loop {
let controller = ConsoleController::new(backend);
let arc_controller = Arc::new(RwLock::new(controller));
let vu_console_backend = Arc::new(RwLock::new(
VhostUserConsoleBackend::new(arc_controller).map_err(Error::CouldNotCreateBackend)?,
));
let mut daemon = VhostUserDaemon::new(
String::from("vhost-device-console-backend"),
vu_console_backend.clone(),
GuestMemoryAtomic::new(GuestMemoryMmap::new()),
)
.map_err(Error::CouldNotCreateDaemon)?;
let vring_workers = daemon.get_epoll_handlers();
vu_console_backend
.read()
.unwrap()
.set_vring_worker(&vring_workers[0]);
// Start the corresponding console thread
let read_handle = if backend == BackendType::Nested {
VhostUserConsoleBackend::start_console_thread(&vu_console_backend)
} else {
VhostUserConsoleBackend::start_tcp_console_thread(&vu_console_backend, tcp_addr.clone())
};
daemon.serve(&socket).map_err(Error::ServeFailed)?;
// Kill console input thread
vu_console_backend.read().unwrap().kill_console_thread();
// Wait for read thread to exit
match read_handle.join() {
Ok(_) => info!("The read thread returned successfully"),
Err(e) => warn!("The read thread returned the error: {:?}", e),
}
}
}
pub fn start_backend(config: VuConsoleConfig) -> Result<()> {
let mut handles = HashMap::new();
let (senders, receiver) = std::sync::mpsc::channel();
let tcp_addrs = config.generate_tcp_addrs();
let backend = config.backend;
for (thread_id, (socket, tcp_addr)) in config
.generate_socket_paths()
.into_iter()
.zip(tcp_addrs.iter())
.enumerate()
{
let tcp_addr = tcp_addr.clone();
info!("thread_id: {}, socket: {:?}", thread_id, socket);
let name = format!("vhu-console-{}", tcp_addr);
let sender = senders.clone();
let handle = Builder::new()
.name(name.clone())
.spawn(move || {
let result = std::panic::catch_unwind(move || {
start_backend_server(socket, tcp_addr.to_string(), backend)
});
// Notify the main thread that we are done.
sender.send(thread_id).unwrap();
result.map_err(|e| Error::ThreadPanic(name, e))?
})
.unwrap();
handles.insert(thread_id, handle);
}
while !handles.is_empty() {
let thread_id = receiver.recv().unwrap();
handles
.remove(&thread_id)
.unwrap()
.join()
.map_err(std::panic::resume_unwind)
.unwrap()?;
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::ConsoleArgs;
use assert_matches::assert_matches;
#[test]
fn test_console_valid_configuration_nested() {
let args = ConsoleArgs {
socket_path: String::from("/tmp/vhost.sock").into(),
backend: BackendType::Nested,
tcp_port: String::from("12345"),
socket_count: 1,
};
assert!(VuConsoleConfig::try_from(args).is_ok());
}
#[test]
fn test_console_invalid_configuration_nested_1() {
let args = ConsoleArgs {
socket_path: String::from("/tmp/vhost.sock").into(),
backend: BackendType::Nested,
tcp_port: String::from("12345"),
socket_count: 0,
};
assert_matches!(
VuConsoleConfig::try_from(args),
Err(Error::SocketCountInvalid(0))
);
}
#[test]
fn test_console_invalid_configuration_nested_2() {
let args = ConsoleArgs {
socket_path: String::from("/tmp/vhost.sock").into(),
backend: BackendType::Nested,
tcp_port: String::from("12345"),
socket_count: 2,
};
assert_matches!(
VuConsoleConfig::try_from(args),
Err(Error::WrongBackendSocket)
);
}
#[test]
fn test_console_valid_configuration_network_1() {
let args = ConsoleArgs {
socket_path: String::from("/tmp/vhost.sock").into(),
backend: BackendType::Network,
tcp_port: String::from("12345"),
socket_count: 1,
};
assert!(VuConsoleConfig::try_from(args).is_ok());
}
#[test]
fn test_console_valid_configuration_network_2() {
let args = ConsoleArgs {
socket_path: String::from("/tmp/vhost.sock").into(),
backend: BackendType::Network,
tcp_port: String::from("12345"),
socket_count: 2,
};
assert!(VuConsoleConfig::try_from(args).is_ok());
}
fn test_backend_start_and_stop(args: ConsoleArgs) {
let config = VuConsoleConfig::try_from(args).expect("Wrong config");
let tcp_addrs = config.generate_tcp_addrs();
let backend = config.backend;
for (_socket, tcp_addr) in config
.generate_socket_paths()
.into_iter()
.zip(tcp_addrs.iter())
{
let controller = ConsoleController::new(backend);
let arc_controller = Arc::new(RwLock::new(controller));
let vu_console_backend = Arc::new(RwLock::new(
VhostUserConsoleBackend::new(arc_controller)
.map_err(Error::CouldNotCreateBackend)
.expect("Fail create vhuconsole backend"),
));
let mut _daemon = VhostUserDaemon::new(
String::from("vhost-device-console-backend"),
vu_console_backend.clone(),
GuestMemoryAtomic::new(GuestMemoryMmap::new()),
)
.map_err(Error::CouldNotCreateDaemon)
.expect("Failed create daemon");
// Start the corresponinding console thread
let read_handle = if backend == BackendType::Nested {
VhostUserConsoleBackend::start_console_thread(&vu_console_backend)
} else {
VhostUserConsoleBackend::start_tcp_console_thread(
&vu_console_backend,
tcp_addr.clone(),
)
};
// Kill console input thread
vu_console_backend.read().unwrap().kill_console_thread();
// Wait for read thread to exit
assert_matches!(read_handle.join(), Ok(_));
}
}
#[test]
fn test_start_net_backend_success() {
let args = ConsoleArgs {
socket_path: String::from("/tmp/vhost.sock").into(),
backend: BackendType::Network,
tcp_port: String::from("12345"),
socket_count: 1,
};
test_backend_start_and_stop(args);
}
#[test]
fn test_start_nested_backend_success() {
let args = ConsoleArgs {
socket_path: String::from("/tmp/vhost.sock").into(),
backend: BackendType::Nested,
tcp_port: String::from("12345"),
socket_count: 1,
};
test_backend_start_and_stop(args);
}
}

View file

@ -0,0 +1,44 @@
// Console backend device
//
// Copyright 2023-2024 VIRTUAL OPEN SYSTEMS SAS. All Rights Reserved.
// Timos Ampelikiotis <t.ampelikiotis@virtualopensystems.com>
//
// SPDX-License-Identifier: Apache-2.0 or BSD-3-Clause
use crate::virtio_console::VirtioConsoleConfig;
use clap::ValueEnum;
use log::trace;
#[derive(ValueEnum, Clone, Copy, Default, Debug, Eq, PartialEq)]
pub enum BackendType {
#[default]
Nested,
Network,
}
#[derive(Debug)]
pub(crate) struct ConsoleController {
config: VirtioConsoleConfig,
pub backend: BackendType,
pub exit: bool,
}
impl ConsoleController {
pub(crate) fn new(backend: BackendType) -> ConsoleController {
ConsoleController {
config: VirtioConsoleConfig {
cols: 20.into(),
rows: 20.into(),
max_nr_ports: 1.into(),
emerg_wr: 64.into(),
},
backend,
exit: false,
}
}
pub(crate) fn config(&self) -> &VirtioConsoleConfig {
trace!("Get config\n");
&self.config
}
}

View file

@ -0,0 +1,69 @@
// VIRTIO CONSOLE Emulation via vhost-user
//
// Copyright 2023-2024 VIRTUAL OPEN SYSTEMS SAS. All Rights Reserved.
// Timos Ampelikiotis <t.ampelikiotis@virtualopensystems.com>
//
// SPDX-License-Identifier: Apache-2.0 or BSD-3-Clause
mod backend;
mod console;
mod vhu_console;
mod virtio_console;
use crate::console::BackendType;
use clap::Parser;
use log::error;
use std::path::PathBuf;
use std::process::exit;
pub(crate) type Result<T> = std::result::Result<T, Error>;
use crate::backend::{start_backend, Error, VuConsoleConfig};
#[derive(Parser, Debug)]
#[clap(author, version, about, long_about = None)]
struct ConsoleArgs {
/// Location of vhost-user Unix domain socket. This is suffixed by 0,1,2..socket_count-1.
#[clap(short = 's', long, value_name = "SOCKET")]
socket_path: PathBuf,
/// Number of guests (sockets) to connect to.
#[clap(short = 'c', long, default_value_t = 1)]
socket_count: u32,
/// Console backend (Network, Nested) to be used.
#[clap(short = 'b', long, value_enum, default_value = "nested")]
backend: BackendType,
/// Initial tcp port to be used with "network" backend. If socket_count is N then
/// the following tcp ports will be created: tcp_port, tcp_port + 1, ..., tcp_port + (N - 1).
#[clap(short = 'p', long, value_name = "PORT", default_value = "12345")]
tcp_port: String,
}
impl TryFrom<ConsoleArgs> for VuConsoleConfig {
type Error = Error;
fn try_from(args: ConsoleArgs) -> Result<Self> {
if args.socket_count == 0 {
return Err(Error::SocketCountInvalid(0));
}
if (args.backend == BackendType::Nested) && (args.socket_count != 1) {
return Err(Error::WrongBackendSocket);
}
Ok(VuConsoleConfig {
socket_path: args.socket_path,
backend: args.backend,
tcp_port: args.tcp_port,
socket_count: args.socket_count,
})
}
}
fn main() {
env_logger::init();
if let Err(e) = VuConsoleConfig::try_from(ConsoleArgs::parse()).and_then(start_backend) {
error!("{e}");
exit(1);
}
}

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,60 @@
// Console virtio bindings
//
// Copyright 2023-2024 VIRTUAL OPEN SYSTEMS SAS. All Rights Reserved.
// Timos Ampelikiotis <t.ampelikiotis@virtualopensystems.com>
//
// SPDX-License-Identifier: Apache-2.0 or BSD-3-Clause
use vm_memory::{ByteValued, Le16, Le32};
/// Feature bit numbers
#[allow(dead_code)]
pub const VIRTIO_CONSOLE_F_SIZE: u16 = 0;
pub const VIRTIO_CONSOLE_F_MULTIPORT: u16 = 1;
#[allow(dead_code)]
pub const VIRTIO_CONSOLE_F_EMERG_WRITE: u16 = 2;
/// Console virtio control messages
pub const VIRTIO_CONSOLE_DEVICE_READY: u16 = 0;
pub const VIRTIO_CONSOLE_PORT_ADD: u16 = 1;
#[allow(dead_code)]
pub const VIRTIO_CONSOLE_PORT_REMOVE: u16 = 2;
pub const VIRTIO_CONSOLE_PORT_READY: u16 = 3;
pub const VIRTIO_CONSOLE_CONSOLE_PORT: u16 = 4;
#[allow(dead_code)]
pub const VIRTIO_CONSOLE_RESIZE: u16 = 5;
pub const VIRTIO_CONSOLE_PORT_OPEN: u16 = 6;
pub const VIRTIO_CONSOLE_PORT_NAME: u16 = 7;
/// Virtio Console Config
#[derive(Copy, Clone, Debug, Default, PartialEq)]
#[repr(C)]
pub(crate) struct VirtioConsoleConfig {
pub cols: Le16,
pub rows: Le16,
pub max_nr_ports: Le32,
pub emerg_wr: Le32,
}
// SAFETY: The layout of the structure is fixed and can be initialized by
// reading its content from byte array.
unsafe impl ByteValued for VirtioConsoleConfig {}
#[derive(Copy, Clone, Debug, Default, PartialEq)]
#[repr(C)]
pub(crate) struct VirtioConsoleControl {
pub id: Le32,
pub event: Le16,
pub value: Le16,
}
impl VirtioConsoleControl {
pub fn to_le_bytes(self) -> Vec<u8> {
let mut buffer = Vec::new();
buffer.extend_from_slice(&self.id.to_native().to_le_bytes());
buffer.extend_from_slice(&self.event.to_native().to_le_bytes());
buffer.extend_from_slice(&self.value.to_native().to_le_bytes());
buffer
}
}