VNC OSD interface

From BATCOM-IT Services Wiki
Jump to: navigation, search

Lightweight VNC OSD interface for MAME

Purpose

Running an emulator on any machine and displaying its screen(s) / accessing its input(s) remotely from one or more VNC clients can be useful in a number of ways. Here are some examples:

  • You want to play with friends :)
  • You need to show a developer some emulator core behavior or bug live
  • You want to support someone else in installing an OS on some exotic machine in MAME
  • The hardware you'd like to use as display device isn't capable of running MAME (due to performance restrictions or because there's no recent port to its platform yet)
  • You want to be able to offer your thin-client users some opportunity of having fun :)

Of course, this project is also of some academic interest for its author(s) ;).

Notes and restrictions

This OSD implementation currently has the following restrictions:

  • Latency may be an issue for a number of very fast paced games when run over the internet, but local LAN connections should always be fine - however, it also depends on the encoding type in use, so play with that in case of trouble.
  • Scaling is inherently supported by the RFB protocol server and the VNC OSD, and many clients support scaling, but the speed at which this works is highly client-dependent.
  • If the menu font looks awful to you then that's because it's rendered in the machine's native display resolution, so scaling that up to modern high-res displays makes it nearly unreadable. You can overcome this partially by specifying an integer scaling factor >= 2 through the -prescale option. The frame-buffer will then be software-scaled by MAME to create a larger display. However, this costs bandwidth so don't overdo! The increase is quadratic, so for example -prescale 2 would generate four times as much transfers as the default (-prescale 1). Some encodings such as copyrect, hextile, zlib or tight will compensate this mostly (for the cost of image quality), so play with them if it's too slow!
  • No support for web-clients - this may change in the future, but its priority is very low.
  • Linux only for now, and a clean pkg-config environment is assumed/required!

However, apart from that it's quite stable at least and works fine for most games/machines :).

Build instructions

Source download

Do this to get the latest source from github.com (a direct fork of the official MAME github.com repository):

rene@marvin:~> mkdir -p ~/src/mame-git-qmc2
rene@marvin:~> cd ~/src
rene@marvin:~/src> git clone https://github.com/qmc2/mame.git mame-git-qmc2

Later on run this to update the source tree:

rene@marvin:~> cd ~/src/mame-git-qmc2
rene@marvin:~/src/mame-git-qmc2> git pull -v

Installing LibVNCServer and libavcodec

Besides the requirements for (SDL)MAME itself you'll need the development packages of LibVNCServer and libavcodec (FFmpeg's version of the library!) for your Linux distribution.

Here are the package installation commands for Linux distributions that we've tried so far:

openSUSE

rene@marvin:~> sudo zypper install LibVNCServer-devel libavcodec-devel

Debian GNU Linux / Ubuntu / Linux Mint

rene@mint18 ~ $ sudo apt-get install libvncserver-dev libavcodec-dev

If you can't find an equivalent LibVNCServer package for your distribution you'd have to build & install it from source. Same basically holds for libavcodec but this library is much more common / available for any relevant distribution.

Building the emulator

Build the emulator binary as usual, using vnc as the value for the OSD make option. Example:

rene@marvin:~/src/mame-git-qmc2> make -j9 OSD=vnc
...
Linking mame64...

Or if you'd like to utilize ccache and/or want to include the MAME tools in your build, you may use something like this command line:

rene@marvin:~/src/mame-git-qmc2> make -j9 OSD=vnc TOOLS=1 OVERRIDE_CC="ccache gcc" OVERRIDE_CXX="ccache g++" PRECOMPILE=0

Cleaning up the build tree

To remove all generated (i.e. binary) files from the build tree, run this command:

rene@marvin:~/src/mame-git-qmc2> make clean
GCC 6 detected
Cleaning genie
Cleaning...
make[1]: Entering directory '/home/rene/src/mame-git-qmc2/src/devices/cpu/m68000'
Cleaning...
make[1]: Leaving directory '/home/rene/src/mame-git-qmc2/src/devices/cpu/m68000'

Command line (or ini) options

Additional to the core CLI (or ini) options you'll find these VNC OSD specific options at the end of the -showusage output:

#
# VNC OPTIONS
#
-vnc_port            TCP port to listen on for incoming VNC connections (default: 5900)
-vnc_adjust_fb       Auto-adjust the frame-buffer width to be a multiple of 4 for best client compatibility (default: 1)
-vnc_autopause       Pause the machine when all clients disconnected, resume it when a client connects (default: 1)
-vnc_mp2write        Writes MP2 encoded audio data to 'mame_audio_stream.mp2' in the current working directory (default: 0)
-vnc_audio_bitrate   Audio encoder bit rate (default: 128000)
-vnc_audio_port      Audio server UDP port (default: 6900)
-vnc_audio_maxconn   Maximum number of client connections at a time (default: 32)

Details:

Option (and <value>)

Type

Default

Description

-vnc_port <port> integer 5900 The RFB protocol server will automatically listen on the TCP port specified by vnc_port on all IPv4 and IPv6 interfaces available on the system. The display number the client will have to use to make a VNC connection is equal to <port> - 5900.
-vnc_adjust_fb boolean 1 (true) When the frame-buffer width is not a multiple of 4, the VNC OSD interface will automatically adjust it (this results in a thin unused rectangle on the right which will stay black) because some VNC clients have problems otherwise. Use -novnc_adjust_fb to disable this (corresponding warnings may be logged on -verbose).
-vnc_autopause boolean 1 (true) Automatically pause the running machine when all clients have disconnected, and resume it when a new client connection is made. This saves CPU resources when they aren't actually required. If the machine is already paused when the last client disconnects the running machine's state won't be touched, and it will stay paused when a new client connects. Use -novnc_autopause to disable auto-pausing/resuming.
-vnc_mp2write boolean 0 (false) When enabled, MP2 encoded audio data will be written to the file 'mame_audio_stream.mp2' in the current working directory.
-vnc_audio_bitrate <bitrate> integer 128000 This option can be used to adjust the audio encoder's bit rate. The default of 128000 bit/s produces good/acceptable quality audio - if bandwidth is an issue try lower values. AFAICT, the allowed bit rates are 32, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320 and 384 kbit/s.
-vnc_audio_port <port> integer 6900 The audio server will bind to the given port (UDP). Clients connected to this port will receive the MP2 audio stream via UDP datagrams. See protocol specification for details.
-vnc_audio_maxconn <clients> integer 32 This option can be used to customize the allowed number of client connections to the audio stream at a time. When this number has been reached, the audio server will reject further connection requests.

Execution example

Client log

rene@marvin:~> vncviewer -NoJPEG :0

TigerVNC Viewer 64-bit v1.7.0
Built on: ??-??-?? ??:??
Copyright (C) 1999-2016 TigerVNC Team and many others (see README.txt)
See http://www.tigervnc.org for information on TigerVNC.
XOpenIM() failed

Thu Nov 17 19:14:57 2016
 DecodeManager: Detected 8 CPU core(s)
 DecodeManager: Creating 4 decoder thread(s)
 CConn:       connected to host localhost port 5900
 CConnection: Server supports RFB protocol version 3.8
 CConnection: Using RFB protocol version 3.8
 CConnection: Choosing security type None(1)
 X11PixelBuffer: Using default colormap and visual, TrueColor, depth 24.
 CConn:       Using pixel format depth 24 (32bpp) little-endian rgb888
 CConn:       Using Tight encoding

Thu Nov 17 19:14:58 2016
 CConn:       End of stream

Client screens

TightVNC connected to MAME: Macintosh IIx with a RasterOps Colorboard 264 as second video card

TightVNC connected to MAME: Fire one

3 TightVNC clients connected to MAME: Klax

TightVNC connected to MAME: C64 PAL (with Commodore 1702 monitor artwork)

Two TightVNC viewers (which cannot scale themselves) connected to MAME: Viper Phase 1, software-scaled by the server using '-prescale 3' on the command line

Server log

rene@marvin:~/src/mame-git-qmc2> ./mame64 -rp /home/games/mame/roms -vnc_mp2write -verbose mmatrix
VNC OSD v0.2
RFB: Listening for VNC connections on TCP port 5900
RFB: Listening for VNC connections on TCP6 port 5900
Screen #0: 512x262
Input: Adding keyboard #0: Keyboard (device id: KBD)
MP2 codec successfully opened
MP2 output file successfully opened
Enter init_monitors
Leave init_monitors
Audio: Start initialization
Audio: Driver is pulseaudio
Audio: frequency: 48000, channels: 2, samples: 256
sdl_create_buffers: creating stream buffer of 25600 bytes
Audio: End initialization
Searching font Liberation Sans in -. path/s
Matching font: /usr/share/fonts/truetype/LiberationSans-Regular.ttf
Region ':maincpu' created
...
qsound.bin ROM NEEDS REDUMP
WARNING: the machine might not run correctly.
Starting Mars Matrix: Hyper Solid Shooting (USA 000412) ':'
Optional memory region ':stars' not found
Optional device ':soundlatch2' not found
Optional device ':soundlatch' not found
Optional device ':msm2' not found
Optional device ':msm1' not found
Optional device ':m48t35' not found
Optional device ':oki' not found
Optional shared pointer ':mainram' not found
  (missing dependencies; rescheduling)
Starting M68000 ':maincpu'
Starting Timer ':scantimer'
Starting Z80 ':audiocpu'
Starting Serial EEPROM 93C46 (64x16) ':eeprom'
Optional memory region ':eeprom' not found
Starting Video Screen ':screen'
  (missing dependencies; rescheduling)
Starting gfxdecode ':gfxdecode'
Starting palette ':palette'
Starting Speaker ':lspeaker'
  (missing dependencies; rescheduling)
Starting Speaker ':rspeaker'
  (missing dependencies; rescheduling)
Starting Q-Sound ':qsound'
Starting DSP16 ':qsound:qsound'
Starting Mars Matrix: Hyper Solid Shooting (USA 000412) ':'
Optional memory region ':stars' not found
Optional device ':soundlatch2' not found
Optional device ':soundlatch' not found
Optional device ':msm2' not found
Optional device ':msm1' not found
Optional device ':m48t35' not found
Optional device ':oki' not found
Optional shared pointer ':mainram' not found
  (missing dependencies; rescheduling)
Starting Video Screen ':screen'
Starting Speaker ':lspeaker'
Starting Speaker ':rspeaker'
Starting Mars Matrix: Hyper Solid Shooting (USA 000412) ':'
Optional memory region ':stars' not found
Optional device ':soundlatch2' not found
Optional device ':soundlatch' not found
Optional device ':msm2' not found
Optional device ':msm1' not found
Optional device ':m48t35' not found
Optional device ':oki' not found
Optional shared pointer ':mainram' not found
RFB:   other clients:
RFB: Normal socket connection
RFB: Client Protocol Version 3.8
RFB: Protocol version sent 3.8, using 3.8
RFB: rfbProcessClientSecurityType: executing handler for type 1
RFB: rfbProcessClientSecurityType: returning securityResult for client rfb version >= 3.8
RFB: Pixel format for client ::1:
RFB:   32 bpp, depth 8, little endian
RFB:   true colour: max r 255 g 255 b 255, shift r 0 g 8 b 16
RFB: Enabling X-style cursor updates for client ::1
RFB: Enabling full-color cursor updates for client ::1
RFB: Enabling cursor position updates for client ::1
RFB: Enabling KeyboardLedState protocol extension for client ::1
RFB: Enabling NewFBSize protocol extension for client ::1
RFB: Enabling SupportedMessages protocol extension for client ::1
RFB: Enabling SupportedEncodings protocol extension for client ::1
RFB: Enabling ServerIdentity protocol extension for client ::1
RFB: Enabling Xvp protocol extension for client ::1
RFB: Using hextile encoding for client ::1
Video RFB updates: 3.7% [1,003.82 KB / 26.76 MB]
Video RFB updates: 0.3% [84.03 KB / 26.76 MB]
Audio codec ratio: 8.3% [15.75 KB / 189.00 KB]
Video RFB updates: 0.0% [0.00 KB / 26.76 MB]
...
Average speed: 100.00% (15 seconds)
sdl_kill: closing audio
RFB: Client ::1 gone
RFB: Statistics             events    Transmit/ RawEquiv ( saved)
RFB:  XvpServerMessage    :      1 |         4/        4 (  0.0%)
RFB:  FramebufferUpdate   :    931 |         0/        0 (  0.0%)
RFB:  hextile             :   3740 |   9634235/218047448 ( 95.6%)
RFB:  ServerIdentify      :      1 |        42/       42 (  0.0%)
RFB:  SupportedEncoding   :      1 |        96/       96 (  0.0%)
RFB:  SupportedMessage    :      1 |        76/       76 (  0.0%)
RFB:  PointerPos          :      1 |        12/       12 (  0.0%)
RFB:  RichCursor          :      1 |       255/      255 (  0.0%)
RFB:  TOTALS              :   4677 |   9634720/218047933 ( 95.6%)
RFB: Statistics             events    Received/ RawEquiv ( saved)
RFB:  KeyEvent            :     40 |       320/      320 (  0.0%)
RFB:  PointerEvent        :    355 |      2130/     2130 (  0.0%)
RFB:  FramebufferUpdate   :    932 |      9320/     9320 (  0.0%)
RFB:  SetEncodings        :      1 |        44/       44 (  0.0%)
RFB:  SetPixelFormat      :      1 |        20/       20 (  0.0%)
RFB:  TOTALS              :   1329 |     11834/    11834 (  0.0%)

Audio protocol specification

This is preliminary information and may change at any time!

As it stands, the protocol we're using for audio transmission is pretty simple: The client connects to the given server's UDP port and sends a request for the connection to the audio stream. If the maximum number of client connections (default: 32) has not been reached, the server will respond with the sample-rate that the client is expected to honor and adjust its audio output device to. From now on each datagram the server sends to the client will contain an MPEG-II encoded audio frame (signed 16-bit L-R stereo format) until the client sends a request to disconnect from the stream.

Here's a concrete protocol example (assuming that the maximum number of connections has not been reached yet):

Server                                    <=>    Client
--------------------------------------    ---    ---------------------------------------
<create-UDP-sockect>                             ...
<bind-UDP-socket-to-audio-port>                  ...
<wait-for-client-connection-on-socket>           ...
...                                              <create-UDP-socket>
...                                              <bind-UDP-socket-to-random-port>
...                                              <connect-socket-to-server's-audio-port>
<honor-client-connect>                    <==    "VNC_OSD_AUDIO_CONNECT_TO_STREAM"
"VNC_OSD_AUDIO_SAMPLE_RATE 48000"         ==>    <adjust-to-sample-rate>
[MPEG2-encoded-audio-frame]               ==>    <process-audio-data>
[MPEG2-encoded-audio-frame]               ==>    <process-audio-data>
[MPEG2-encoded-audio-frame]               ==>    <process-audio-data>
...                                       ...    ...
[MPEG2-encoded-audio-frame]               ==>    <process-audio-data>
[MPEG2-encoded-audio-frame]               ==>    <process-audio-data>
[MPEG2-encoded-audio-frame]               ==>    <process-audio-data>
<honor-client-disconnect>                 <==    "VNC_OSD_AUDIO_DISCONNECT_FROM_STREAM"
...                                              <unbind-and-clean-up-socket>

If the maximum number of connections has been reached, instead of sending the sample-rate followed by the MPEG-II-stream the server would send a single datagram containing the string VNC_OSD_AUDIO_CLIENT_REJECTED to the client and not send any audio data to that client subsequently.

The <process-audio-data> step on the client side involves decoding of the MPEG-II frames and playing back the resulting PCM stream through some audio device. The Qt VNC Viewer (see below) uses FFmpeg's libavcodec for the decoding part and Qt's QAudioOutput for the playback. See audioclient.h / audioclient.cpp in its source code for an example implementation which combines the networking-, decoding- and playback-code in a thread separate from the GUI thread in order to not interfere with frame-buffer updates and user interaction.

Also, the raw PCM data's audio volume is 100% (0 dB attenuation). You have to control the volume client-wise. The MAME volume slider has no effect :).

Qt VNC Viewer

We're also developing a Qt based VNC client called Qt VNC Viewer which is specifically tuned to cooperate nicely with the VNC OSD. It has (or will have) support for

  • Qt 4 and 5 (Qt 4.8 minimum) (done)
  • frame buffer drawing, scaling and filtering through Qt's raster paint engine or directly through OpenGL (done, however the Qt 5 OpenGL renderer isn't implemented yet)
  • keyboard and mouse input (done)
  • a VNC OSD specific network audio protocol (more or less done, quality needs to be improved, Qt 5 is required/recommended due to its advanced QtMultimedia module)
  • Windows and Mac OS X (Linux only right now) (planned)
  • SDL joysticks mapped to mouse and/or keyboard input (planned)
  • a match-making protocol (future)
  • support for common VNC authentication schemes (future)

The overall goal is to offer a fairly similar user experience through VNC - even when truly remote - as the standard (Windows and SDL) OSDs do for local gaming.

Here's a screen shot of it in action:

Qvncviewer-preview-1.png

The code is already quite stable and should work fine for almost everyone!

It hasn't been tested with any other VNC server than the VNC OSD for MAME for which it was made primarily. As it doesn't support any VNC authentication schemes right now it's not ready for use cases like desktop sharing anyway (so bug reports related to this will be ignored).

If you'd like to try out Qt VNC Viewer use this SVN URL to checkout its code: http://svn.code.sf.net/p/qvncviewer/code/trunk

We recommend to use Qt 5 for it, although it builds & works as well with Qt 4 (except for sound support which is based on QtMultimedia - Qt 4's multimedia module requires QtMobility which isn't available everywhere).

Make sure to install all development packages for FFmpeg's libavcodec, Qt 5 (+ QtMultimedia!) and LibVNCClient.

Then just run qmake-qt5 && make to build the client.

Pre-built binary packages are not yet available.

See http://sourceforge.net/projects/qvncviewer/ for details!

FAQs

When I start the emulator it just sits there and does nothing. Huh?

Your assumption is most likely wrong :). Remember that the emulator binary now acts as VNC server, it has no primary display. Also, logging to stdout will only happen when -verbose was specified on the command line (or via an ini file).

To see it doing something you have to connect to it through a VNC client!

On Linux, for example, you might want to use the netstat command in the following way to verify that it's actually listening on the given port:

rene@marvin:~> sudo netstat -anp | grep 5900
tcp        0      0 0.0.0.0:5900            0.0.0.0:*               LISTEN      32264/mame64      <= IPv4
tcp        0      0 :::5900                 :::*                    LISTEN      32264/mame64      <= IPv6

rene@marvin:~> pidof mame64
32264

Bug-tracker

VNC OSD bugs and feature requests are tracked in our bug-tracking system.

License

The VNC OSD, Copyright © 2013 - 2016 René Reucher, is free software distributed under the terms of the MAME license.