Skip to content

Android Activity

Jump to bottom
tpecholt edited this page Dec 24, 2023 · 10 revisions

Android Support

As ImGui contains an Android backend implementation ImRAD added support for Android Activities as well. I never liked Android UI development because the Java APIs were too verbose and too messy but ImGui seems like a good idea to me. ImRAD support is currently experimental as there are some remaining issues to solve but you can already create functional android apps.

In this tutorial I demonstrate how to create a simple Android app and discuss what more is needed or missing.

Choosing the right IDE

For developing for android we need an IDE which is able to target ARM devices, compile APK, run the code in Android emulator and deploy it to the device.

Visual Studio 2022

On Windows Visual Studio is my IDE of choice and I knew it also contains support for developing on Android so I was naturally inclined to try it out. In VS 2022 installer there is an option for "Mobile development with C++". I installed it and tried to use it but I didn't get very far. There were many issues like error saying Android SDK license agreements were not accepted so open SDK manager but SDK manager is not available from Tools/Android Tools menu and the SDK installation seemed incomplete. Then the wrong version of java was chosen although VS installed newer version so JAVA_HOME had to be specified. Then Native activity which I wanted to use was not available. There are some youtube tutorials from this year but they talk about Native Activity with Ant and all Ant templates were removed in some recent VS update without replacement with gradle alternative. Honestly it looks like VS android support is driven by management who doesn't have clear understanding about developer needs so strange decisions are taken. Fortunately better alternative exist.

Android Studio

Unlike VS Android Studio worked from the beginning without a major issue. You can even set key mapping to VS mode to make it more familiar. The android emulator starts really fast. Even deployment to the device is fast and you can connect your phone for debugging via USB or WiFi (but that didn't always work for me). Things like creating assets directory are easily done from popup menus. It even supports CMake so it's not really necessary to touch gradle at all.

Android App Structure

Although it may be possible to develop Native Activity in 100% C++ I advise against it. Calling complicated java API through JNI is no fun and quickly becomes tedious to write. Therefore it's better to implement Native Activity stub in Java (or Kotlin).

In Android Studio create a new project "Native C++". You will have:

  • Android manifest - certain options needs to be set.

  • Java activity class - here you can implement few important methods to wrap the desired functionality behind simple API which can be called easily through JNI.

  • C++ shared library built by CMake - This library should contain main.cpp with ImGui initialization, ImGui source codes including the android backend and Activity classes generated by ImRAD.

main.cpp code, java activity code and android manifest can be generated automatically by ImRAD by choosing the last option from the New File Menu. C++ Activity class can be created by using the Activity - Android option.

image

Be sure to move these files to their appropriate folders which were generated by Android Studio.

MainActivity.java

Generated code was originally based on the original ImGui android example but it proved to be insufficient so it got revamped and improved in many ways.

Generated MainActivity class contains:

  • Code to load the C++ shared object library

  • showSoftInput() - code for showing or hiding the soft keyboard. The code is able to show different types of keyboards such as standard, number input, decimal input, phone and email input keyboard. It can also set the action button. This is achieved by creating an invisible TextEdit, setting its InputType and activating it before keyboard is shown. Generated ImRAD code reports IME type of the active Input widget so this can be used to request the right keyboard type. More needs to be done to support other features such as text prediction and voice input.

  • getDpi() - returns screen DPI for ImGui UI scaling. Scaling is enabled by using dp units instead of pixel units. dp units are the default for android activities.

    on Android dp units usually refer to the dimensions of a pixel on a 160 DPI device. With higher DPI dp gets greater than 1 pixel so that the dimensions on the screen stay the same. Generated main.cpp calculates the scaling factor against 120 DPI screen because that is the default for desktops where ImRAD will be normally used. The scaling factor is saved under ImGui::GetIO().UserData->dpiScale and you can modify the code to calculate your own.

  • getRotation() - returns current display rotation in degrees. Can be used for drawing area setup.

  • showMessage() - shows a toast message, e.g. informational message which disappears in few seconds. This can be handy. TODO: Currently not functional?

  • OnKeyboardShown() - although in C++ code the keyboard is shown or hidden when desired (based on io.WantTextInput value) it is possible the user dismissed the keyboard by pressing the back button. This method ten calls a callback implemented in C++ to react on that (further enhancements like keeping the keyboard hidden but show it again as soon as different input control gets focused should be implemented). As there is no direct way to know keyboard visibility status in android API the code installs GlobalLayoutListener and reacts on visible display frame changes.

  • OnScreenRotation() - called when screen rotation changes. C++ code will be notified since screen rotation implies drawing clipping rectangle change (e.g. when navigation bar moves to different part of screen)

  • OnInputCharacter() - called when new character was typed on the keyboard

  • OnSpecialKey() - called when keyboard action button was pressed

AndroidManifest.xml

This is a standard android manifest with few important options:

  • icons, label - specify this for your app

  • activity/name - dotted name of the Java activity class

  • activity/theme such as @android:style/Theme.NoTitleBar.Fullscreen. This will let the app to draw to the whole screen minus the title bar on top.

  • meta-data/android.app.lib_name - specifies the name of the C++ shared object (without the lib prefix). TODO: is this needed?

Shared library

1. CMakeLists.txt

this is pretty much standard CMakeLists with one add_library(SHARED) call.

  • add_library should contain ${CMAKE_ANDROID_NDK}/sources/android/native_app_glue/android_native_app_glue.c

  • target_include_directories should contain ${CMAKE_ANDROID_NDK}/sources/android/native_app_glue

  • target_link_libraries should contain android, log, EGL, GLESv3

2. main.cpp

Generated main.cpp is loosely based on the original example but extended with many new functionality.

  • Java_com_example_myapplication_MainActivity_OnKeyboardShown - reacts on keyboard visibility change. Currently only resets g_IMEType when keyboard was dismissed by user.

  • Java_com_example_myapplication_MainActivity_OnScreenRotation - reacts on screen rotation change. It sets clipping rectangle used by generated activity class to avoid drawing into NavBar area. You may implement different logic f.e. when drawing into NavBar is desired but it should be rendered dimmed.

  • Java_com_example_myapplication_MainActivity_OnInputCharacter - reacts on typed character

  • Java_com_example_myapplication_MainActivity_OnSpecialKey - reacts on keyboard action button press. Currently it will send it as ImGuiKey_AppForward

  • GetAssetData - helper to load data from asset directory such as fonts. Fonts will then be loaded using AddFontFromMemoryTTF instead of AddFontFromFileTTF. In Android Studio you can create asset directory by clicking New / Folder / Assets Folder popup menu.

  • GetDisplayInfo - reads android screen DPI, calculates height of the NavBar (currently fixed as 48*dp), and sets initial clipping rectangle for the generated activity (should be improved when the app is already launched rotated)

    ...

3. ImGui

Add ImGui sources including the Android OpenGL3 backend.

There are some known issues:

  • #6627 - buttons are rendered as clicked after being released. As suggested adding additional io.AddMousePosEvent(-FLT_MAX, -FLT_MAX); call fixes the issue. The fix remains to be merged into the master branch.

  • #6854 - content scrolling without scrollbars should be allowed. ImRad implements its own scrolling functionality which works fine.

  • #7031 - input text cursor has a fixed 1px width which is problematic on high DPI phone screens. Also the color can't be changed although in Android the color is always different.

  • #5686 - the current character input reporting in java activity is not reliable and it doesn't work with certain settings or HW keyboards. ImRad implements different character input which works fine.

4. Activity classes

Add files generated by ImRAD. Usage of an activity class is similar to other window classes generated by ImRAD. But there are two differences:

  • UI scaling - the UI should be designed using the dp units. This way it becomes DPI aware so the dimensions of widgets used on different phones will be approximately equal. If you use the generated main.cpp same scaling factor will be used to adjust font sizes during font loading time and for ImGui style dimensions as well.

    For widgets expanding to the right or bottom edge of the activity use size = -1 as usual.

  • Activity is hidden by default so you need to call Open first. Calling Open additionally closes the previously opened activity so you don't need to keep track which activity is currently open.

.

Final Looks

Following activity was designed in ImRAD 0.7. Just a couple of input widgets stretched to fill the screen (size_x = -1, size_y)

screen

Clone this wiki locally