VNC OSD interface
- 1 Purpose
- 2 Notes and restrictions
- 3 Build instructions
- 4 Command line (or ini) options
- 5 Execution example
- 6 Audio protocol specification
- 7 Qt VNC Viewer
- 8 FAQs
- 9 Bug-tracker
- 10 License
Lightweight VNC OSD interface for MAME
Running an emulator on any machine and displaying its screen(s) / accessing its input(s) remotely from one or more VNC clients can be useful in a number of ways. Here are some examples:
- You want to play with friends :)
- You need to show a developer some emulator core behavior or bug live
- You want to support someone else in installing an OS on some exotic machine in MAME
- The hardware you'd like to use as display device isn't capable of running MAME (due to performance restrictions or because there's no recent port to its platform yet)
- You want to be able to offer your thin-client users some opportunity of having fun :)
Of course, this project is also of some academic interest for its author(s) ;).
Notes and restrictions
This OSD implementation currently has the following restrictions:
- Latency may be an issue for a number of very fast paced games when run over the internet, but local LAN connections should always be fine - however, it also depends on the encoding type in use, so play with that in case of trouble.
- Scaling is inherently supported by the RFB protocol server and the VNC OSD, and many clients support scaling, but the speed at which this works is highly client-dependent.
- If the menu font looks awful to you then that's because it's rendered in the machine's native display resolution, so scaling that up to modern high-res displays makes it nearly unreadable. You can overcome this partially by specifying an integer scaling factor >= 2 through the
-prescaleoption. The frame-buffer will then be software-scaled by MAME to create a larger display. However, this costs bandwidth so don't overdo! The increase is quadratic, so for example
-prescale 2would generate four times as much transfers as the default (
-prescale 1). Some encodings such as copyrect, hextile, zlib or tight will compensate this mostly (for the cost of image quality), so play with them if it's too slow!
- No support for web-clients - this may change in the future, but its priority is very low.
- Linux only for now, and a clean pkg-config environment is assumed/required!
However, apart from that it's quite stable at least and works fine for most games/machines :).
Do this to get the latest source from github.com (a direct fork of the official MAME github.com repository):
rene@marvin:~> mkdir -p ~/src/mame-git-qmc2 rene@marvin:~> cd ~/src rene@marvin:~/src> git clone https://github.com/qmc2/mame.git mame-git-qmc2
Later on run this to update the source tree:
rene@marvin:~> cd ~/src/mame-git-qmc2 rene@marvin:~/src/mame-git-qmc2> git pull -v
Installing LibVNCServer and libavcodec
Here are the package installation commands for Linux distributions that we've tried so far:
rene@marvin:~> sudo zypper install LibVNCServer-devel libavcodec-devel
Debian GNU Linux / Ubuntu / Linux Mint
rene@mint18 ~ $ sudo apt-get install libvncserver-dev libavcodec-dev
If you can't find an equivalent LibVNCServer package for your distribution you'd have to build & install it from source. Same basically holds for libavcodec but this library is much more common / available for any relevant distribution.
Building the emulator
Build the emulator binary as usual, using
vnc as the value for the
OSD make option. Example:
rene@marvin:~/src/mame-git-qmc2> make -j9 OSD=vnc ... Linking mame64...
Or if you'd like to utilize ccache and/or want to include the MAME tools in your build, you may use something like this command line:
rene@marvin:~/src/mame-git-qmc2> make -j9 OSD=vnc TOOLS=1 OVERRIDE_CC="ccache gcc" OVERRIDE_CXX="ccache g++" PRECOMPILE=0
Cleaning up the build tree
To remove all generated (i.e. binary) files from the build tree, run this command:
rene@marvin:~/src/mame-git-qmc2> make clean GCC 6 detected Cleaning genie Cleaning... make: Entering directory '/home/rene/src/mame-git-qmc2/src/devices/cpu/m68000' Cleaning... make: Leaving directory '/home/rene/src/mame-git-qmc2/src/devices/cpu/m68000'
Command line (or ini) options
Additional to the core CLI (or ini) options you'll find these VNC OSD specific options at the end of the
# # VNC OPTIONS # -vnc_port TCP port to listen on for incoming VNC connections (default: 5900) -vnc_adjust_fb Auto-adjust the frame-buffer width to be a multiple of 4 for best client compatibility (default: 1) -vnc_autopause Pause the machine when all clients disconnected, resume it when a client connects (default: 1) -vnc_mp2write Writes MP2 encoded audio data to 'mame_audio_stream.mp2' in the current working directory (default: 0) -vnc_audio_bitrate Audio encoder bit rate (default: 128000) -vnc_audio_port Audio server UDP port (default: 6900) -vnc_audio_maxconn Maximum number of client connections at a time (default: 32)
| Option (and <value>)
||integer||5900|| The RFB protocol server will automatically listen on the TCP port specified by |
||boolean||1 (true)|| When the frame-buffer width is not a multiple of 4, the VNC OSD interface will automatically adjust it (this results in a thin unused rectangle on the right which will stay black) because some VNC clients have problems otherwise. Use |
||boolean||1 (true)|| Automatically pause the running machine when all clients have disconnected, and resume it when a new client connection is made. This saves CPU resources when they aren't actually required. If the machine is already paused when the last client disconnects the running machine's state won't be touched, and it will stay paused when a new client connects. Use |
||boolean||0 (false)||When enabled, MP2 encoded audio data will be written to the file 'mame_audio_stream.mp2' in the current working directory.|
||integer||128000||This option can be used to adjust the audio encoder's bit rate. The default of 128000 bit/s produces good/acceptable quality audio - if bandwidth is an issue try lower values. AFAICT, the allowed bit rates are 32, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320 and 384 kbit/s.|
||integer||6900||The audio server will bind to the given port (UDP). Clients connected to this port will receive the MP2 audio stream via UDP datagrams. See protocol specification for details.|
||integer||32||This option can be used to customize the allowed number of client connections to the audio stream at a time. When this number has been reached, the audio server will reject further connection requests.|
rene@marvin:~> vncviewer -NoJPEG :0 TigerVNC Viewer 64-bit v1.7.0 Built on: ??-??-?? ??:?? Copyright (C) 1999-2016 TigerVNC Team and many others (see README.txt) See http://www.tigervnc.org for information on TigerVNC. XOpenIM() failed Thu Nov 17 19:14:57 2016 DecodeManager: Detected 8 CPU core(s) DecodeManager: Creating 4 decoder thread(s) CConn: connected to host localhost port 5900 CConnection: Server supports RFB protocol version 3.8 CConnection: Using RFB protocol version 3.8 CConnection: Choosing security type None(1) X11PixelBuffer: Using default colormap and visual, TrueColor, depth 24. CConn: Using pixel format depth 24 (32bpp) little-endian rgb888 CConn: Using Tight encoding Thu Nov 17 19:14:58 2016 CConn: End of stream
rene@marvin:~/src/mame-git-qmc2> ./mame64 -rp /home/games/mame/roms -vnc_mp2write -verbose mmatrix VNC OSD v0.2 RFB: Listening for VNC connections on TCP port 5900 RFB: Listening for VNC connections on TCP6 port 5900 Screen #0: 512x262 Input: Adding keyboard #0: Keyboard (device id: KBD) MP2 codec successfully opened MP2 output file successfully opened Enter init_monitors Leave init_monitors Audio: Start initialization Audio: Driver is pulseaudio Audio: frequency: 48000, channels: 2, samples: 256 sdl_create_buffers: creating stream buffer of 25600 bytes Audio: End initialization Searching font Liberation Sans in -. path/s Matching font: /usr/share/fonts/truetype/LiberationSans-Regular.ttf Region ':maincpu' created ... qsound.bin ROM NEEDS REDUMP WARNING: the machine might not run correctly. Starting Mars Matrix: Hyper Solid Shooting (USA 000412) ':' Optional memory region ':stars' not found Optional device ':soundlatch2' not found Optional device ':soundlatch' not found Optional device ':msm2' not found Optional device ':msm1' not found Optional device ':m48t35' not found Optional device ':oki' not found Optional shared pointer ':mainram' not found (missing dependencies; rescheduling) Starting M68000 ':maincpu' Starting Timer ':scantimer' Starting Z80 ':audiocpu' Starting Serial EEPROM 93C46 (64x16) ':eeprom' Optional memory region ':eeprom' not found Starting Video Screen ':screen' (missing dependencies; rescheduling) Starting gfxdecode ':gfxdecode' Starting palette ':palette' Starting Speaker ':lspeaker' (missing dependencies; rescheduling) Starting Speaker ':rspeaker' (missing dependencies; rescheduling) Starting Q-Sound ':qsound' Starting DSP16 ':qsound:qsound' Starting Mars Matrix: Hyper Solid Shooting (USA 000412) ':' Optional memory region ':stars' not found Optional device ':soundlatch2' not found Optional device ':soundlatch' not found Optional device ':msm2' not found Optional device ':msm1' not found Optional device ':m48t35' not found Optional device ':oki' not found Optional shared pointer ':mainram' not found (missing dependencies; rescheduling) Starting Video Screen ':screen' Starting Speaker ':lspeaker' Starting Speaker ':rspeaker' Starting Mars Matrix: Hyper Solid Shooting (USA 000412) ':' Optional memory region ':stars' not found Optional device ':soundlatch2' not found Optional device ':soundlatch' not found Optional device ':msm2' not found Optional device ':msm1' not found Optional device ':m48t35' not found Optional device ':oki' not found Optional shared pointer ':mainram' not found RFB: other clients: RFB: Normal socket connection RFB: Client Protocol Version 3.8 RFB: Protocol version sent 3.8, using 3.8 RFB: rfbProcessClientSecurityType: executing handler for type 1 RFB: rfbProcessClientSecurityType: returning securityResult for client rfb version >= 3.8 RFB: Pixel format for client ::1: RFB: 32 bpp, depth 8, little endian RFB: true colour: max r 255 g 255 b 255, shift r 0 g 8 b 16 RFB: Enabling X-style cursor updates for client ::1 RFB: Enabling full-color cursor updates for client ::1 RFB: Enabling cursor position updates for client ::1 RFB: Enabling KeyboardLedState protocol extension for client ::1 RFB: Enabling NewFBSize protocol extension for client ::1 RFB: Enabling SupportedMessages protocol extension for client ::1 RFB: Enabling SupportedEncodings protocol extension for client ::1 RFB: Enabling ServerIdentity protocol extension for client ::1 RFB: Enabling Xvp protocol extension for client ::1 RFB: Using hextile encoding for client ::1 Video RFB updates: 3.7% [1,003.82 KB / 26.76 MB] Video RFB updates: 0.3% [84.03 KB / 26.76 MB] Audio codec ratio: 8.3% [15.75 KB / 189.00 KB] Video RFB updates: 0.0% [0.00 KB / 26.76 MB] ... Average speed: 100.00% (15 seconds) sdl_kill: closing audio RFB: Client ::1 gone RFB: Statistics events Transmit/ RawEquiv ( saved) RFB: XvpServerMessage : 1 | 4/ 4 ( 0.0%) RFB: FramebufferUpdate : 931 | 0/ 0 ( 0.0%) RFB: hextile : 3740 | 9634235/218047448 ( 95.6%) RFB: ServerIdentify : 1 | 42/ 42 ( 0.0%) RFB: SupportedEncoding : 1 | 96/ 96 ( 0.0%) RFB: SupportedMessage : 1 | 76/ 76 ( 0.0%) RFB: PointerPos : 1 | 12/ 12 ( 0.0%) RFB: RichCursor : 1 | 255/ 255 ( 0.0%) RFB: TOTALS : 4677 | 9634720/218047933 ( 95.6%) RFB: Statistics events Received/ RawEquiv ( saved) RFB: KeyEvent : 40 | 320/ 320 ( 0.0%) RFB: PointerEvent : 355 | 2130/ 2130 ( 0.0%) RFB: FramebufferUpdate : 932 | 9320/ 9320 ( 0.0%) RFB: SetEncodings : 1 | 44/ 44 ( 0.0%) RFB: SetPixelFormat : 1 | 20/ 20 ( 0.0%) RFB: TOTALS : 1329 | 11834/ 11834 ( 0.0%)
Audio protocol specification
This is preliminary information and may change at any time!
As it stands, the protocol we're using for audio transmission is pretty simple: The client connects to the given server's UDP port and sends a request for the connection to the audio stream. If the maximum number of client connections (default: 32) has not been reached, the server will respond with the sample-rate that the client is expected to honor and adjust its audio output device to. From now on each datagram the server sends to the client will contain an MPEG-II encoded audio frame (signed 16-bit L-R stereo format) until the client sends a request to disconnect from the stream.
Here's a concrete protocol example (assuming that the maximum number of connections has not been reached yet):
Server <=> Client -------------------------------------- --- --------------------------------------- <create-UDP-sockect> ... <bind-UDP-socket-to-audio-port> ... <wait-for-client-connection-on-socket> ... ... <create-UDP-socket> ... <bind-UDP-socket-to-random-port> ... <connect-socket-to-server's-audio-port> <honor-client-connect> <== "VNC_OSD_AUDIO_CONNECT_TO_STREAM" "VNC_OSD_AUDIO_SAMPLE_RATE 48000" ==> <adjust-to-sample-rate> [MPEG2-encoded-audio-frame] ==> <process-audio-data> [MPEG2-encoded-audio-frame] ==> <process-audio-data> [MPEG2-encoded-audio-frame] ==> <process-audio-data> ... ... ... [MPEG2-encoded-audio-frame] ==> <process-audio-data> [MPEG2-encoded-audio-frame] ==> <process-audio-data> [MPEG2-encoded-audio-frame] ==> <process-audio-data> <honor-client-disconnect> <== "VNC_OSD_AUDIO_DISCONNECT_FROM_STREAM" ... <unbind-and-clean-up-socket>
If the maximum number of connections has been reached, instead of sending the sample-rate followed by the MPEG-II-stream the server would send a single datagram containing the string
VNC_OSD_AUDIO_CLIENT_REJECTED to the client and not send any audio data to that client subsequently.
The <process-audio-data> step on the client side involves decoding of the MPEG-II frames and playing back the resulting PCM stream through some audio device. The Qt VNC Viewer (see below) uses FFmpeg's libavcodec for the decoding part and Qt's QAudioOutput for the playback. See
audioclient.cpp in its source code for an example implementation which combines the networking-, decoding- and playback-code in a thread separate from the GUI thread in order to not interfere with frame-buffer updates and user interaction.
Also, the raw PCM data's audio volume is 100% (0 dB attenuation). You have to control the volume client-wise. The MAME volume slider has no effect :).
Qt VNC Viewer
We're also developing a Qt based VNC client called Qt VNC Viewer which is specifically tuned to cooperate nicely with the VNC OSD. It has (or will have) support for
- Qt 4 and 5 (Qt 4.8 minimum) (done)
- frame buffer drawing, scaling and filtering through Qt's raster paint engine or directly through OpenGL (done, however the Qt 5 OpenGL renderer isn't implemented yet)
- keyboard and mouse input (done)
- a VNC OSD specific network audio protocol (more or less done, quality needs to be improved, Qt 5 is required/recommended due to its advanced QtMultimedia module)
- Windows and Mac OS X (Linux only right now) (planned)
- SDL joysticks mapped to mouse and/or keyboard input (planned)
- a match-making protocol (future)
- support for common VNC authentication schemes (future)
The overall goal is to offer a fairly similar user experience through VNC - even when truly remote - as the standard (Windows and SDL) OSDs do for local gaming.
Here's a screen shot of it in action:
The code is already quite stable and should work fine for almost everyone!
It hasn't been tested with any other VNC server than the VNC OSD for MAME for which it was made primarily. As it doesn't support any VNC authentication schemes right now it's not ready for use cases like desktop sharing anyway (so bug reports related to this will be ignored).
If you'd like to try out Qt VNC Viewer use this SVN URL to checkout its code: http://svn.code.sf.net/p/qvncviewer/code/trunk
We recommend to use Qt 5 for it, although it builds & works as well with Qt 4 (except for sound support which is based on QtMultimedia - Qt 4's multimedia module requires QtMobility which isn't available everywhere).
Make sure to install all development packages for FFmpeg's libavcodec, Qt 5 (+ QtMultimedia!) and LibVNCClient.
Then just run
qmake-qt5 && make to build the client.
Pre-built binary packages are not yet available.
See http://sourceforge.net/projects/qvncviewer/ for details!
When I start the emulator it just sits there and does nothing. Huh?
Your assumption is most likely wrong :). Remember that the emulator binary now acts as VNC server, it has no primary display. Also, logging to stdout will only happen when
-verbose was specified on the command line (or via an ini file).
To see it doing something you have to connect to it through a VNC client!
On Linux, for example, you might want to use the
netstat command in the following way to verify that it's actually listening on the given port:
rene@marvin:~> sudo netstat -anp | grep 5900 tcp 0 0 0.0.0.0:5900 0.0.0.0:* LISTEN 32264/mame64 <= IPv4 tcp 0 0 :::5900 :::* LISTEN 32264/mame64 <= IPv6 rene@marvin:~> pidof mame64 32264
VNC OSD bugs and feature requests are tracked in our bug-tracking system.