Provide non-X11 support, using fullscreen EGL video and SDL2 controls

[hissingshark]

As described in mcpelauncher-manifest issues #457 and #474 I am trying to port your project to an ARM based platform with no X11 environment.

This commit is my working contribution. Video is provided via fullscreen framebuffer drivers under EGL. Mouse, keyboard and gamepad controls via SDL2.

Building on my platform is configured with
CFLAGS="-march=armv7 -mfpu=neon" CXXFLAGS="-march=armv7 -mfpu=neon" CC=clang CXX=clang++ cmake -DJNI_USE_JNIVM=ON -DGAMEWINDOW_SYSTEM=SDL2 ..

I have tested on the Vero4K - an OSMC device based on Debian. SOC is the same as the ODROID C2. I will be testing on the RPi further down the line.

TODO:

1. Get the QT-UI working without X11 (some progress there having compiled the QT5 EGLFS plugins for my platform).
2. Get audio via FMOD rather than PulseAudio as that is soon to be unsupported on my platform. libfmod.so.10.20 seems to be available to download for ARMHF, so I assume this should be possible.

I hope this will be useful. Any feedback or changes appreciated.

[ChristopherHX]

Looks good so far,

Yes there is an armhf version of libfmod, it is implemented in my fork using a method like armhfrewrite (https://github.com/ChristopherHX/mcpelauncher-client/blob/master/src/main.cpp#L65-L100). I haven't ported it over to ng, because my method linked against libfmod and there is a chance that two different libfmod are loaded.

Is it possible for your device to output sound via SDL? Doing so could also restore audio for 32bit mac without finding a way to use pulseaudio.

My thoughts about changes to your implementation after I tested and merged it.

• Move your egl specfic code into a subclass of sdlwindow
• Create another subclass which uses sdl as window
• Add sound to game-window and move my pulseaudio impl over to a eglut and glfw base class
• Implement sound for sdl

I will be testing on the RPi further down the line.

Well my arm32/64 ng port is only tested there :) so a good device to share issues.

Well your cmake option might need changes, after having two sdl based game-windows.

[MCMrARM]

This should be renamed to something like EmbeddedSDL2 in order to avoid confusion, since it does not use SDL2 as a windowing system.

[hissingshark]

Is it possible for your device to output sound via SDL?

I think that's generally how it is done for the emulators, so it should be. Working with SDL audio will be a new one for me. Only done video and input in the past. But I'll look into it.

Move your egl specfic code into a subclass of sdlwindow
Create another subclass which uses sdl as window

I think I follow what you mean here. Is that refactoring to make the components reusable in other window types in the future?

This should be renamed to something like EmbeddedSDL2 in order to avoid confusion

Good idea. Particularly if the audio becomes part of it. I'm rubbish at naming things. I just gave my kids numbers (starting from 0 obviously).

[ChristopherHX]

I think I follow what you mean here. Is that refactoring to make the components reusable in other window types in the future?

I want reuse the common sdl2 input code once to add full sdl2 video, your embedded sdl2 + EGL is a special case.
Avoids copying code and maintaining both.

SDL2GameWindow
                                                              -> SDL2GameWindowBase -> GameWindow
EmbeddedSDL2GameWindow (EGL + framebuffer + cursor)

EGL specfic code + framebuffer manipulation + cursor code, into EmbeddedSDL2GameWindow
SDL Input, into SDL2GameWindowBase

I think the sdl2 sound api and pulseaudio simple api are similar enough to get it working.

Some links about audio, not related to this PR.
pulseaudio simple current working version: https://github.com/minecraft-linux/mcpelauncher-client/blob/master/src/jni/pulseaudio.cpp

Use pcm audio without callback ( SDL >= 2.0.4 )
https://wiki.libsdl.org/SDL_AudioSpec

Choose the same format as for pulseaudio https://wiki.libsdl.org/SDL_AudioFormat
'Signed 16 Bit PCM, native endian', AUDIO_S16SYS

Use this to queue input on demand from game instead of pa_simple_write
https://wiki.libsdl.org/SDL_QueueAudio

[ChristopherHX]

Hmm, here is not compiling code for me ubuntu 20.04, clang-10, x86_64.
I need these changes to get it compiled. I have no idea how to run it without errors.

diff --git a/CMakeLists.txt b/CMakeLists.txt
index fc45042..26a4748 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -21,7 +21,7 @@ if (GAMEWINDOW_SYSTEM STREQUAL "EGLUT")
 elseif (GAMEWINDOW_SYSTEM STREQUAL "GLFW")
     target_sources(gamewindow PRIVATE ${GAMEWINDOW_SOURCES_GLFW})
     target_link_libraries(gamewindow PUBLIC glfw3)
-elseif (GAMEWINDOW_SYSTEM STREQUAL "SDL2")
+elseif (GAMEWINDOW_SYSTEM STREQUAL "EMBEDDED_EGL_SDL2")
     find_package(EGL REQUIRED)
     message(STATUS "Found EGL")
     find_package(SDL2 REQUIRED)
diff --git a/src/window_sdl2.cpp b/src/window_sdl2.cpp
index 87d7402..5bb1430 100644
--- a/src/window_sdl2.cpp
+++ b/src/window_sdl2.cpp
@@ -22,6 +22,10 @@ SDL2GameWindow::SDL2GameWindow(const std::string& title, int width, int height,
     initEGL();
     initSDL();
     initCursor();
+    auto proc = GameWindowManager::getManager()->getProcAddrFunc();
+    glClear = (decltype(glClear))proc("glClear");
+    glClearColor = (decltype(glClearColor))proc("glClearColor");
+    glFinish = (decltype(glFinish))proc("glFinish");
 }
 
 SDL2GameWindow::~SDL2GameWindow() {
@@ -145,7 +149,7 @@ void SDL2GameWindow::initSDL() {
 void SDL2GameWindow::initCursor() {
     const char mcw = 16; // raw mouse cursor image width
     // rgba mouse cursor image data
-    const char mci[16][16][4] = {
+    const unsigned char mci[16][16][4] = {
     {{6,61,51,255}, {6,61,51,255}, {6,61,51,255}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}},
     {{2,32,26,255}, {165,253,240,255}, {165,253,240,255}, {6,61,51,255}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}},
     {{2,32,26,255}, {165,253,240,255}, {38,200,174,255}, {165,253,240,255}, {6,61,51,255}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}, {0,0,0,0}},
diff --git a/src/window_sdl2.h b/src/window_sdl2.h
index 8af9d0c..8a2e6fd 100644
--- a/src/window_sdl2.h
+++ b/src/window_sdl2.h
@@ -1,16 +1,18 @@
 #pragma once
 
 #include <game_window.h>
+#define EGL_NO_X11
 #include <EGL/egl.h>
+#include <GLES2/gl2.h>
 #include <SDL2/SDL.h>
 
 
 struct bgra_pixel {
     // channels in framebuffer order
-    char b;
-    char g;
-    char r;
-    char a;
+    unsigned char b;
+    unsigned char g;
+    unsigned char r;
+    unsigned char a;
 };
 
 class SDL2GameWindow : public GameWindow {
@@ -64,6 +66,10 @@ private:
     void handleKeyboardEvent(SDL_KeyboardEvent *event);
     KeyCode getKeyMinecraft(int keyCode);
 
+    void (*glClearColor)(GLfloat red, GLfloat green, GLfloat blue, GLfloat alpha);
+    void (*glClear)(GLbitfield mask);
+    void (*glFinish)();
+
 public:
 
     SDL2GameWindow(const std::string& title, int width, int height, GraphicsApi api);

You said yourself char have no sign, but you are using it as if it is unsigned, but's thats not good for other platforms. Clang errored out.

My pure sdl2 branch (WIP) based on your work https://github.com/minecraft-linux/game-window/tree/christopherhx/sdl2.

Keyboard input is very broken and unusable. (Yes I mean the layout)

This shows libEGL without a extra findEGL.cmake
https://developer.nvidia.com/blog/linking-opengl-server-side-rendering/

[hissingshark]

Ok, I follow the refactoring idea now. Will do.

Thanks for the audio info. I'd started looking into that and I've also proven SDL audio works on my platform. I've now had to stop and deal with my libSDL2 first. I use a custom build and it's requiring some additional changes to leave PulseAudio behind. Getting there.

Good point about the char inconsistency. I'll mark those as unsigned. It's useful to have the feedback from compiling on other platforms. I've been dedicated to get it working on the one box. Although this is SBC/embedded effort, you're on Ubuntu and x86_64. Are you cross-compiling? All mine is native on the board itself.

Not sure why you're having all those problems with the GL functions. Perhaps a difference in the drivers?

How is keyboard broken? As you'll see from the TODO in my code my qwerty and numpad keys don't map right for me, but I just remap those in the game menu (this could even be an issue with the keyboard I'm using - only got the one to test with).

[ChristopherHX]

Are you cross-compiling?

No, I compiled and ran it natively on my intel x86_64 device. I think my problem is libEGL needs x11 or wayland to work on my system. I still have no idea how to use libEGL on my system without a windowmanager.

Not sure why you're having all those problems with the GL functions. Perhaps a difference in the drivers?

Your cmake files only links against libEGL.so, it was found. Then you referenced functions from libGLESv2.so which aren't exported by mine libEGL, so please load them dynamically (see my diff) or link explicitly to libGLESv2.so.

As you'll see from the TODO in my code my qwerty and numpad keys don't map right for me

You are right, the input works with random keycodes. As far as I remember, I was unable to exit the game via ESC.

numpad keys, needs changes to the Keymap enum + remapping, which I haven't published yet.
The Keymap enum contains the virtual keycodes from microsoft windows.