Create AppImage for linux? Maybe containerize just one linux build environment in Docker image, first?

[bdklahn]

Would it work to put a single xpra AppImage in the executable path? I don't know about the Travis CI they mention, but GitHub Actions has Docker builder actions, and can automatically build and store images in GitHub's container registry.

It seems like one could then get away with having just one linux build environment to build something that could work on all the Linux distros. A Dockerfile, maybe "FROM" a CentOS or Ubuntu/Debian LTS distro, could serve as . . .
a. where Xpra devs can do the builds
d. community users can do builds
c. living documentation of how to create a working Linux build environment for Xpra

Personally, I really like Singularity containers, because they don't need to be run as root nor have a daemon running (are possible for running things in multi-user environments, like HPC compute clusters).
So what I do is use them with the docker:// bootstrap. But Docker is much more widely used and supported in automatic build systems. It is also nice that Dockerfile parsing leverages a unionfs to enable layered builds, which can save time when adding software.

For some projects, I have a build Action set up to trigger a build when I push a tag matching a certain pattern.

[bdklahn]

Ah. I see you might actually be using Buildah, in . . .
https://github.com/Xpra-org/repo-build-scripts

[totaam]

@bdklahn It's not clear to me what you're trying to achieve.
Can you please clarify?
AppImage / flatpak / etc are heavy and unlikely to ever provide the level of integration that can be achieved with a native OS package.

[bdklahn]

I guess I was thinking it would make it easier to have packages which would actually work on Linux.

I, personally, run OpenSUSE. The latest package (updated this January), again, doesn't work, testing the simple "seamless xterm" example on your main page in two different shells on the same machine (I filed a bug there, assuming you'd say "Their package is broken."). So I can, like I did just last summer, try to build from source. But then I always encounter like a half dozen dependencies IN ADDITION TO the many listed on the dependencies page. And with the regularly cryptic or useless error messages, it is often very difficult to figure out what needs to be installed.

I figured, OK, I'm on a bit of a weird distribution, which I can't expect folks to support in addition to the most popular ones. So I thought I'd try an installation in a container of a distribution you support. I chose a stable LTS version of probably the most popular: Ubuntu focal. Should pretty easily work, right? It must have worked for you, since you have an example with Ubuntu Focal Fossa.

Except that doesn't work, as written. Whoever created that must have been on a personal machine where they forgot they had installed (directly or indirectly) at least three additional, packages, so far.

I'm currently building up a Singularity def file based on an "Official . . ." Ubuntu Focal Fossa image.

Here's what I have so far.

> cat xpra_ubuntu-focal.def 
BootStrap: docker
From: ubuntu:focal-20220113

%post
    DISTRO=focal
    echo 'APT::Get::Assume-Yes "true";' > /etc/apt/apt.conf.d/90forceyes
    #echo 'APT::Get::force-yes "true";' >> /etc/apt/apt.conf.d/90forceyes
    #install https support for apt (which may be installed already):
    apt-get update
    apt-get install apt-transport-https
    apt install ca-certificates
    apt-get install gnupg2
    apt-get install wget
    # add Xpra GPG key
    wget -q https://xpra.org/gpg.asc -O- | apt-key add -
    # add Xpra repository
    add-apt-repository "deb https://xpra.org/ $DISTRO main"
    # install Xpra package
    apt-get update
    apt-get install xpra

(ultimately solved with export DEBIAN_FRONTEND=noninteractive and apt-get install software-properties-common)

I am currently hitting . . .

. . .
+ wget -q https://xpra.org/gpg.asc -O-
+ apt-key add -
OK
+ add-apt-repository deb https://xpra.org/ focal main
/.post.script: 13: add-apt-repository: not found
FATAL:   While performing build: while running engine: exit status 127
. . .

So I guess my point is that the packages aren't "providing a level of integration . . .", etc. The whole purpose of a distribution packaging system is to have a system where a software and all it's dependencies can be easily installed. The Xpra package does not do that.
AppImage seems to be a way to do that. And I was thinking it could also maybe make it easier to support most Linux, rather than having to keep up with different distribution packages.

I'm not sure what you mean by "heavy". Yes, with more dependencies inside, the file can get pretty big. But this is not the year 2000. Disk space is cheap. There is a lot more network bandwidth, these days. Shared libraries are not as critically useful as they used to be.
But maybe I am missing something in terms of complications a single file might cause for integrating multiple executables in multiple paths, etc.

I thought, maybe, have a single container, where you build a single executable from. At least the container part would be useful. So you, and users, would actually know what the actual dependencies are, and have a clean, reproducible build environment.

I keep forgetting, you are already using containers with buildah, etc.
So maybe this should be about making sure each rpm, deb, etc. is built to install all the dependencies.
If that's not possible or easy with those packager implementations, I'd say to not use them. They fail at their primary purpose. Maybe AppImage can be the solution.

[totaam]

I guess I was thinking it would make it easier to have packages which would actually work on Linux.

As far as we are aware they do on all the platforms supported: https://github.com/Xpra-org/xpra/wiki/Platforms

I, personally, run OpenSUSE. The latest package (updated this January), again, doesn't work, ..
.. assuming you'd say "Their package is broken."

Definitely!

But then I always encounter like a half dozen dependencies IN ADDITION TO the many listed on the dependencies page.

Perhaps you could help us and tell us what those are?
The dependencies list is a generic page, which works reasonably well on platforms as varied as MacOS, Windows and, of course, Linux distros. But there are bound to be package naming differences and boundary differences.
In fact, this was the major paint point when we did have suse support.

And with the regularly cryptic or useless error messages, it is often very difficult to figure out what needs to be installed

Which one? Perhaps we can improve it?

I chose a stable LTS version of probably the most popular: Ubuntu focal. Should pretty easily work, right?

Yes.

add-apt-repository: not found

As per the first hit on google:
sudo apt install software-properties-common

Except that doesn't work, as written.

IIRC, software-properties-common is installed by default, just not in a docker container.

Whoever created that must have been on a personal machine where they forgot they had installed (directly or indirectly) at least three additional, packages, so far.

Which are?
I'm pretty sure it works out of the box, but as per above, you are trying to run in a container, which is a different setup.

The whole purpose of a distribution packaging system is to have a system where a software and all it's dependencies can be easily installed. The Xpra package does not do that.

You haven't included any details of the actual problems you are hitting (apart from the package manager not being fully installed), so it is impossible to ascertain that.

AppImage seems to be a way to do that.

Sort of, with some major caveats.

Disk space is cheap

Disk space is not the primary concern with "heavy" containers, I was thinking of security and maintenance costs.
This is exactly what MacOS and MS Windows binaries are, and the former takes an enormous amount of effort to maintain, the latter is mostly maintained thanks to the work of MSYS2 but we do lose some flexibility because of it.

Shared libraries are not as critically useful as they used to be.

We'll have to agree to disagree on this one!

But maybe I am missing something in terms of complications a single file might cause for integrating multiple executables in multiple paths, etc.

Yes, quite a few features were simply not achievable using flatpak / appimage / snap / whatever. (ie: printer forwarding, etc)

So maybe this should be about making sure each rpm, deb, etc. is built to install all the dependencies.

I'm not sure what you mean there.
The build scripts rely on the packaging files to ensure things are installed as declared.
If things were missing, then the package would not build. (this does happen when we forget something has been added and the packaging has not been updated)

If that's not possible or easy with those packager implementations, I'd say to not use them.

I'm not seeing the problems, please share the details.

[totaam]

FWIW: I needed a clean VM to test something else, so I followed the installation instructions for Ubuntu 20.04 (aka focal) and ran some simple tests without problems. (well, apart from forgetting to start the ssh server to allow remote connections)

So unless you can provide more details, I will close this issue as invalid.

The installation instructions on the wiki now include software-properties-common for those installing in containers (even my minimal install did not need this): https://github.com/Xpra-org/xpra/wiki/Download/_compare/a68f8028cfc2b096e5aed257f3112cfa0b11c2b3

[bdklahn]

Yes. That was my thought: I'd need to run a full VM, to test and use Xpra, in my case.

I suppose . . . there IS a difference between distribution version snapshot, and the full desktop installer version of a specific distribution version snapshot. :-)

And I was going to ask if Xpra is always tested in a reproducible run-time environment (e.g. fresh VM), before releasing a package claimed to work on x distribution.

Anyway, I am definitely also wrongly mixing build time "working" and run time working. Of course end users likely don't care if something builds, but then can't run it.

I've always loved the idea of Xpra. When it works, it's pretty darn cool. It's just that, over the years, when I've tried it, the majority of the time it doesn't "just work". I think the first try was when I was running it on FreeBSD (PC-BSD -> TrueOS). I think it was then, or later, when I also installed on MacOSX. I probably even tried on Windows, but don't recall. Oh yeah, at work we also had it running on CentOS. That might have worked . . . maybe.

So, yes, we can take time and iterate through a bunch of "I'm getting this specific error, etc.", . . . And often each individual error message is an easy Google lookup, but . . .
But the original issue which prompted me to open this had exactly the same error message (I need to look again to remember), as another one (I think on a Debian system? -definitely not SUSE). But I figure that might kinda be a waste of everyone's time, when the answer there was basically "You're missing at least one dependency.", -especially when my OS is not officially supported. So I wondered if there was a way to get my OS supported, while also maybe making making Linux support a little less work for folks here: Build once. Test once, rather than maintaining multiple Linux packages. And I don't like the way snap works (yet another app manager/store, right?). I don't really like Flatpack's design. AppImage seemed fairly well thought out. But I don't really know, as I haven't tried to package anything significant with that. I mean . . . we do develop a Qt app. It sounds like AppImage can package those (complex QT) apps. But if you've looked into it, and it's not feasible or worth it, then I have to trust you. I guess I don't completely understand. I believe those just mount a squashfs image, like a regular directory. Everything on Linux (even devices) are represented as files. Maybe they'd require setting special root permissions on some things. Idk? But it seems like it wouldn't have to package every single thing it needs, to work. But then it might be the same difference, making sure the paths are handled correctly for each special distribution.

It just seems that:
Distribution developers depend heavily on app developers to be able to say "Look. My distribution can actually do something useful".
App developers often depend HEAVILY on other people's software to even work and distributions to gain any real visibility.

Distribution developers would have to test and make sure 1000's of packages work.
App developers maybe "need" to make sure a package works on 1 to, say, 5 "places".

Each benefits from the other. Each often blames the other for apps not working. And I suppose this might be a bit natural, for the opensource stuff, because many are not being (directly) paid. We can't expect support we aren't (somehow) paying for. I've been using open source, as my primary work env for about a decade now, and like it. But sometimes it feels like the tag line could be "Free and open source: Sometimes you get what you pay for.". :-)

Anyway . . . way too much verbiage which probably should have been saved for another place (IRC?)

It seems reasonable to close this issue (which you may have already done, whilst I type in this stale tab :-) ). Regarding the initial issue, proper: I'm not sure it is "invalid" to consider a more unified Linux packaging strategy.
Probably something like "won't do", is more accurate.
. . . and completely reasonable, if the time/resources/will aren't there, or there are truly hard deal-breaker issues.

Maybe you can suggest a reasonable best option if I do still want to run on OpenSUSE. Maybe first try and get a concrete idea of where the OpenSUSE package maintainer is at, on this? -offer to help there? I might guess that they just have it on "CI autopilot", with their Open Build Service, and maybe don't really, then, run test it(?).
Maybe see if we can get me set up with an rpm build environment, here? (I guess, this: https://github.com/Xpra-org/xpra/blob/master/docs/Build/RPM.md, to get an idea).

Almost forgot. I really appreciate your time here, and working on Xpra, for whatever that's worth to you :-) .

[totaam]

And I was going to ask if Xpra is always tested in a reproducible run-time environment (e.g. fresh VM), before releasing a package claimed to work on x distribution.

Not just a fresh VM, but also upgrades, and compatibility with older versions, across dozens of Linux distro versions and MS Windows + MacOS.
It takes a lot of time and not every combination can be tested before every release, which means that sometimes a bug falls through the cracks.

over the years, when I've tried it, the majority of the time it doesn't "just work".

Sadly, downstream packaging is a problem that has been plaguing the project for many years: https://github.com/Xpra-org/xpra/wiki/Distribution-Packages

I think the first try was when I was running it on FreeBSD (PC-BSD -> TrueOS)

FreeBSD is not an officially supported platform, but I do test it occasionally and it is close enough to Linux that it should work with minimal effort.

I think it was then, or later, when I also installed on MacOSX.

MacOS does work but because of the all-in-one binary image format, it's a real drag to maintain and Apple makes you jump through hoops to install packages outside of their app store.

I probably even tried on Windows, but don't recall.

MS Windows definitely works, it has been a supported platform for over 10 years.
It is probably the most popular OS for the client.

Oh yeah, at work we also had it running on CentOS.

Same as MS Windows. Very popular as an xpra server OS.

So, yes, we can take time and iterate through a bunch of "I'm getting this specific error, etc.

I try to record all the common pitfalls in the: https://github.com/Xpra-org/xpra/blob/master/docs/FAQ.md

But I figure that might kinda be a waste of everyone's time, when the answer there was basically "You're missing at least one dependency."

It's an ongoing battle between those that want fewer direct package dependencies (for minimal installs and such) and those that want the package to just work with all the features included (ie: me).

Everything on Linux (even devices) are represented as files. Maybe they'd require setting special root permissions on some things. Idk?

These subsystems are most likely to cause integration problems: printing, service and authentication, nvenc (needs device access), etc

Maybe you can suggest a reasonable best option if I do still want to run on OpenSUSE.
Maybe first try and get a concrete idea of where the OpenSUSE package maintainer is at, on this? -offer to help there?

That's a good idea.

I might guess that they just have it on "CI autopilot", with their Open Build Service, and maybe don't really, then, run test it(?).

Looks like it.

Maybe see if we can get me set up with an rpm build environment, here? (I guess, this: https://github.com/Xpra-org/xpra/blob/master/docs/Build/RPM.md, to get an idea).

Yes, building from source should be quite easy and you don't need to enable all the features to get working setup.
The most difficult part is figuring out the names of the development packages under suse - because of course they're nothing like the RedHat ones..

Almost forgot. I really appreciate your time here, and working on Xpra, for whatever that's worth to you :-) .

Thank you, much appreciated!

[totaam]

This was a duplicate of #1748 which contains more reasons for not going down that path.