metaruss

GNU/Linux Software Projects

Listed below are some of the projects I have worked on or have packaged to make life a little easier.  Debian/Ubuntu users may add the following APT repository to their sources.list if they wish:

deb http://debian.knizefamily.net/ ./
deb-src http://debian.knizefamily.net/ ./

You may also browse the packages here.

Software I Have Written

• notifier-applet – display notfication messages from UDP packets on the GNOME panel.
• site-upload – quickly upload website changes from the command line.
• SimpleWXR – import posts into WordPress using a simplified WXR XML format.
• helper-scripts – a set of tools to help with the maintenance of Debian-based system.
• synctree – a tool to synchronize two directory trees based on a set of rules.
• bup – yet another easy to use backup tool.

Software I Have Packaged

• lirc-modules – updated LIRC modules packages.
• keyspan – kernel module Debian packages for Keyspan USB to serial converters.
• nedit – my preferred graphical text editor.
• xbox – packages for the Microsoft Xbox.
• misterhouse – a home automation program written in Perl.
• sorune – a tool used to manage the database on the Neuros Audio player.
• cats2procmailrc – a tool generate a procmail recipe from a simple, human readable mail categories file.

So I finally bit the bullet and stood-up a MythTV front end on this Raspberry Pi 4 that had been hanging around the office for years.  I had back-ordered one during the post-COVID supply chain crunch and it turned up many months (possibly more than a year) later, but I hadn’t had the motivation to tackle the project.  The Raspberry 5 announcement reminded me, so I ordered one and decided to try to get Myth working on the RP4 to start.  There are some issues with the decoder and graphics software stacks on the RP5 at the moment, but I knew the RP4 would work.

I started with a degree of ignorance and built it in the same way that my other front ends are setup: using Christian Marillat’s “dmo” packages from deb-multimedia.org and running the front end solo in an auto-login X session.  This worked fine, but I was not getting good performance and could not seem to get the playback profiles setup the way I needed them.  This led me down a rabbit hole; it turns out that a lot of the information in the MythTV Wikis is obsolete and does not apply to modern releases.  For example, the recommendation is to use the OpenMAX decoder but no such decoder exists in MythTV 0.33+.  The V4L2 decoder is there, but it is buggy and tends to lockup sometimes.  The standard decoder drops frames and playback is jerky no matter how I set the read-ahead or fiddle with CPUs.  I even tried overclocking it.

While searching for details, I kept seeing references to “mythtv-light“.  There is not a lot of explanation as to what it really is or why it is needed.  What is so light about it?  The wiki talks about the back end, but very little about the front end.  What build flags are used?  Why is it distributed via a random Google Drive?  It all seemed very sketchy, so I wanted to avoid it.  Eventually I found a separate git repository that contains packaging scripts for MythTV for a variety of platforms.  In there were the scripts for mythtv-light, which seems to focus mainly on making installation as simple as possible from a single package without too many outside dependencies.  It’s not really there to support RPi, necessarily.

That said, in terms of the front end (which is all I care about), the RPi MythTV Light packages seem to enable three main things:

1. QPA EGLFS rendering support.
2. Specific decoder and OpenGL support for the Raspberry Pi.
3. Disabled support for all unnecessary libraries and features.

In terms of overall speed and efficiency, #1 is really important on the Raspberry Pi.  By directly supporting Qt’s EGL platform abstraction layer, there is no need to run X at all.  Christian’s dmo packages do not seem to support this, as they are build against Qt for X.  #2 used to be very important, but that no longer seems to be the case.  I believe this is because the version of ffmpeg that ships with Raspian is already setup to use whatever hardware decoders are currently supported on the RP4.  The “Standard” decoder in Myth uses ffmpeg, so it “just works” without any special support for RPi-specific OpenMAX libraries.  #3 is a nice-to-have, but with 8GB of RAM and X not running I don’t know that is matters so much for the front end on the newer RPi’s.  I am getting very good performance on the RP4 and it can *just about* manage to decode 1080p HEVC with only an occasional skip when the bit rate gets too high.  The CPUs are pretty busy, though.

Proper cooling is a must or the SoC will hit thermal throttling.  The above RP4 that I used as a test bed just has a base plate and a small heat sink on the SoC.  It is not enough: the package temperature quickly rises to 60C under heavy load.  There are a few “brands” that list a passively-cooled case on Amazon like the one pictured on the right (there are versions available for RP4 and RP5).  It comes with thermal pads for all of the hot packages as well as the bottom of the PCB.  The RP4 still gets toasty in mine, but it works well enough if left in the open air (i.e. strapped to the back of the TV).  An RP5 is perfectly content in one of these.

I can’t get access to edit the MythTV wikis, so I’ll document my findings here:

• The advice on how to auto-start the front end using cron is…not ideal.  You can absolutely start it from the mythtv user account’s .profile.  You just need launch it only on a tty session and not for an ssh session.  I have mythtv setup to auto-login to tty1, so that is easy.  I have a few other configurable options in there, so I’ll share those files here (you can rip out what you don’t need).
  

  mythtv-autologin.tar.gz

• For the issue of the keyboard not working (important if you use a remote that masquerades as a keyboard like I do), it is just a permissions issue.  Add the mythtv user to the “input” group in /etc/groups and reboot.  Do not start MythTV from /etc/rc.local as the Wiki suggests; that will run the front end as root, which is a bad idea.
• The table showing how to setup the custom playback profile is out of date.  There is no MMAL decoder.  Use the “Standard” decoder instead.  The V4L2 decoder is still listed, but it is not very stable in my experience.  The other fields seem to work OK (4 CPUs, etc).  Use the “Medium quality” deinterlacer, not the low quality one as I have read in a few places.  I got horrible flicker with that one.
• Increase the read-ahead buffer in the advanced settings to quell some of the random stuttering you might see, depending on your network’s performance.  I set mine to 400ms, but you can go higher.
• There is no need to set the gpu_mem value in the firmware config.  The defaults for RP4 and RP5 are fine.  I am overclocking my RP4 as follows:
  

  arm_freq=2147
  over_voltage=6
  gpu_freq=750

With that feather in my cap, I decided to try a Raspberry Pi 5.  I’ve read a lot of conflicting opinions about its viability as a media player, but on paper it looks like a slam-dunk.  As with everything on Linux, the issues seem to stem more from the software support and not so much from the hardware.  The hardware is very capable.  All of the above applies to the RP5 just fine, it just needs a few workarounds at the moment.  These will probably not be needed in the future as support for RP5 matures.

• The front end will fail to start EGLFS.  This is because the RP5 has two devices for the two GPUs and EGL needs to use the second one for 2D OpenGL.  This can be addressed by passing the QPA layer a configuration file that specifies what device and display port to use.  Mine is included in the above archive.

Once it’s up and running, the RP5 works great as a front end.  Watching the same 1080p HEVC video that made the RP4 get out of breath caused the RP5 to hardly break a sweat.  I have not tried 4k yet, but it looks promising.  Things should only get better as support for hardware acceleration improves.

I’ve been using MythTV for more than two decades now.  I have separate front and back ends, as I’ve always had some sort of server running in the basement that is on all the time.  Our first front end was an original Xbox with a Cromwell BIOS running Xebian.  It was just powerful enough to do the job at standard definition and we already had the DVD remote accessory, so it was the perfect choice.  The Xebian project was eventually abandoned and despite my efforts to keep Debian on the Xbox going, software bloat made the experience rather sluggish.  A VIA EPIA M10000 Mini-ITX system took its place in our living room, while the Xbox moved into our bedroom.  We used the EPIA for many years until the capacitors started to fail.

At this point, we had our first HDTV in our basement: a Sony Wega KD-34XBR960 (what an epic boat-anchor of a CRT that was).  I was using an ASUS A8N-VM based PC for the front end so that we could watch HD videos and I wanted something to replace the EPIA system that could at least decode HD on our living room SD TV without issue.  nVidia, with their ION 2 chipset, was the only show in town with efficient, hardware-accelerated H.264 decoding on Linux that MythTV also natively supported.  I picked up an ASUS AT3IONT-I Deluxe and built a new front end around it.  The CPU is quite modest (Intel Atom 330), but this board is all about the integrated GPU and nvdec support.  It worked fantastically and as a bonus the “Deluxe” version came with a remote that sort-of worked (remote controls and MythTV are a whole other thing).

Eventually it came time to retire the old Xbox and turn it back into a game console: it was getting unusably sluggish and there was no hope of watching any HD programming on it (even though the TV was SD).  I wanted to get another identical ION based board, so I picked up an ASUS AT5IONT-I Deluxe.  It had a noticeably faster CPU (Intel Atom D550), but was otherwise pretty similar.  We used these as our main front ends for many years.

Then came the troubles: nVidia started obsoleting their older drivers and the GPUs on these ION boards were not supported by any of the newer drivers.  I managed to limp things along for a few more Debian OS upgrades until it was no longer possible to shoe-horn the required, ancient binary drivers into modern X servers.  The open-source nouveau drivers were and still are pretty terrible and did not seem to support the decoder blocks at all.  The only way to keep using these old systems was to freeze the OS versions.  This only worked to a point, as it became difficult to support newer versions of MythTV on older version of Debian without doing custom builds…which I grew tired of.  I ran into the same problem on some of my older laptops and other machines with nVidia GPUs as well.  I will not be buying any new nVidia-based hardware for the foreseeable future, as I like to reuse old hardware for other purposes.  nVidia has decided to make that impossible by keeping even their most obsolete hardware closed.

Then there was the issue of HEVC.  The hardware decoder blocks on these boards were several generations too old to have any support for H.265 decoding and the CPUs were far too modest to handle software decode.  My only choice is to avoid HEVC files entirely, but that is getting more and more difficult.  H.265 is a far superior codec for dealing with HD and especially 4k, so it is kind of silly to try to keep dancing around the problem.

Finally there is the issue of power.  Running all these machines is neither cheap nor wise.  I should be using something more efficient that preferably shuts-down when the TV is off.

The obvious choice is the Raspberry Pi.  I have been exploring this on and off for at least a decade.  I bought a first-generation one when it first came out in 2012.  It is a fun little toy that I used on a few little projects, but it didn’t occur to me to try to use it as a MythTV front end.  It seemed far too modest.  I felt the same about the second-generation as well, though it turned out that with the right build options it could be made to work.  While the Broadcom SoC is very capable, the issue has always been software support of the hardware acceleration blocks in Linux.  Various licensing issues get in the way of a clean implementation.  I did give it a shot with a Raspberry Pi 3, but I was not terribly impressed with it’s performance even after jumping through the necessary hoops to get everything to work.  My ION-based machines seemed to work better overall.  I was never able to get enough performance to support H.265 decoding on the CPU, so I abandoned it.

Then came the Raspberry Pi 4.  It seemed like it was going to be the hot ticket, but I was too late to the party.  The post-COVID supply chain crunch made buying any Raspberry Pi a total nightmare.  I tried for a long time to get one through legitimate channels, but eventually had to give up.  I wasn’t going to spend flipper dollars on eBay for one.  Eventually, one of my back-orders got fulfilled and an RP4 turned up in the mail.  I got busy with other things and forgot about it, so it sat on a shelf for a long time until….

I recently switched back to Cinnamon from MATE on one of my machines and found I needed to reapply this patch again.  I’ve updated it a bit to support all three modes of operation (workspace desktop preview, workspace labels, and the original useless numbers), selectable from the configuration dialog.  You can also specify the button width in label mode, which can be nice if you want control over the aesthetics.

This patch works for version 6.x of cinnamon:

• workspace-switcher@cinnamon.org.v6.patch

Download it and apply it thusly on a Debian-based system:

$ cd /usr/share/cinnamon/applets/workspace-switcher@cinnamon.org
$ sudo patch -p1 < ~/Downloads/workspace-switcher@cinnamon.org.v6.patch
patching file applet.js
patching file settings-schema.json
$

Then restart cinnamon with Alt+F2 and then “r” and enter.

This is a brain-dump of what I ended up doing to setup an “Eaglercraft Server”.  I’ve been running small Minecraft servers since around 2011.  It was just for myself and a few of my work colleagues.  I enjoyed experiencing MC coming out of beta and eventually growing up.  We ended up with two servers, the original that started in Beta1.3 became a creative server and a “newer” 1.4 server running in survival mode.  Over time, I eventually lost interest in the game but left the servers running.  Like so many kids over the past decade or so, mine inevitably got into MC in a big way and still spend time on the old servers with their friends from time to time.  I also stood up a third server for them to have a fresh start, since the old worlds are massive and had been generated using such old versions (there are some pretty weird chunk transitions in places).  One day, my youngest asked me if I could help him setup an “Eaglercraft Server”.  Boy, what a rabbit hole that turned out to be…

I am by no means an expert here, but I wanted to document what I’ve learned.  At its core, Eaglercraft is a Javascript port of the Minecraft client that can run in a web browser (I’ll leave the reason why it exists and the origin of the name as an exercise to the reader).  Since it was an unofficial port, the project came under fire by Mojang’s current owners and had to go underground to some extent.  The client is only currently available in specific versions: 1.5.2 and 1.8.8.  There is no such thing as an “Eaglercraft server”, per se.  What is actually needed is a Java Edition server configured such that:

1. It’s preferably running in offline mode so that unauthorized users can connect…ah-hem.
2. It’s reachable by Eaglercraft clients from a web browser, preferably via port 80 if possible.
3. It supports version 1.8 clients.

All of these present their own sets of problems, but all are solvable thanks to the amazingly active development community around Minecraft servers.  I am assuming that the reader knows how to administrate a Linux server and has background knowledge of operating a Minecraft server.  If not, there are lots of resources out there.  My goal here was to fill in the missing big picture around what is required.

Authentication

One can, technically, login to an “online” (officially-authenticating) server from an Eaglercraft client by going through a number of convoluted steps to acquire the necessary credentials.  Each player would need to do this on their own, which can be a bit of a pain for the less-technically-inclined.

All of that can be avoided by running in offline mode and setting up an authentication server that users can bounce-through when they first connect.  There are plugins available for forked servers such as Forge, Spigot, and Paper that implement this: nLogin can provide authentication which will then use BungeeCord to tie the servers together.  The authentication server needs to support the same client version(s) as the main server and don’t forget to turn on the whitelist for both if they are going to run offline.  The first time a player connects, they will be asked to set a password, which they then have to remember.  nLogin offers tools for administrators to reset passwords, etc.

One annoying problem is the UUID generation that nLogin does will conflict with any online Java clients coming in, which then causes conflicts in the whitelist.  nLogin has some settings for this, but each has caveats.  One workaround is after adding an online player to the whitelist, look for the actual UUID they are using in the logs and manually edit the whitelist.json file to override the generated one for that player.  You’ll have to issue a “whitelist reload” command afterwards, but they should be granted access then.  The same process needs to be repeated on the main server as well, if it is running a whitelist.

Proxy

To avoid exposing old server versions to the Internet, it is wise to put everything behind a proxy.  The PaperMC project has a lightweight proxy server called Velocity that is a good candidate for this, as it supports BungeeCord and many of the required plugins including nLogin and Eaglercraft.  The main and authentication servers can then be setup to bind only to localhost, preventing direct external connections.  BungeeGuard can also be used as a further protective step.

Note that when using nLogin, the plugin is actually resident on the proxy and not the authentication server.  This can be a little bit confusing when trying to manage the plugin from the local console.  One has to connect to the proxy console, not the authentication server console.

Server

The primary server that players will play on needs to support 1.8 clients.  There are two ways to approach this:

1. Run an actual 1.8 server.
2. Run a more modern server with ViaVersion plugins.

The first approach is the simplest, but it doesn’t scale well thanks to the many bugs present in Minecraft 1.8.  It’s fine if playing among friends that behave themselves or that all agree to use the same exploits.  It can also be more fun and nostalgic to play this way, but it can also go horribly wrong.  At the very least, use a forked server project for this.

Otherwise, it is probably better to run a newer server and support older clients via ViaVersion.  A nice compromise might be to run a 1.12 server, which predates Update Aquatic but fixes a lot of the flaws of earlier servers.  This requires the installation of the following plugins:

• ViaVersion – allows newer clients on older servers for those connecting from a modern Java client.
• ViaBackwards – allows older clients on newer servers.
• ViaRewind – expands ViaBackwards support to really old clients, including 1.8 that EaglerCraft needs.
• ViaRewind-Legacy-Support – fixes a number of glitches and bugs that 1.8 clients would normally experience.

Once everything is setup, players will connect to the proxy which will connect them to the authentication server.  nLogin can be setup a few different ways, but again be sure to turn on the whitelist.  Once authenticated, players are connected to the main server.

I’ve been using Linux Mint for years now. Like a lot of people, I switched around the time that GNOME and Ubuntu set out to gaslight the world about how everything about their workflows have been wrong for all this time. I tried GNOME Shell, but with each passing release they stripped more and more useful functionality from it. Linux Mint kept the GNOME 2 torch lit and eventually offered both MATE (a GNOME 2 fork) and their own Cinnamon (GNOME 3 based) desktop variants. I liked what Cinnamon had to offer, but it was a bit buggy and I so stuck with MATE instead.

Cinnamon continued to improve and at some point I decided to switch over to it. I was able to recreate most of my workflows, but one thorn in my side was that at some point they replaced the workspace names in the switcher with these useless numbers. It was either that or a tiny thumbnail of the desktop. I use workspaces in a very specific way that helps me keep my trains of thought in order and the naming convention is a big part of that. Thinking this was just temporary, I modified the JavaScript to bring back the names instead of numbers and kept reapplying the patch every time Cinnamon got upgraded.

Here we are years later and the silly thing is still broken! I sent the patch to dev that made the original change, but they didn’t take it because it has a one major flaw: the width of the buttons is hard-coded. I suspect this is the real reason behind why the simple view only shows the number. It’s tricky to know how to set the width, as it depends on the font being used, DPI, etc. I assumed there was some “proper” way to do this, but I guess not?

Anyway, I’m posting the patch here in case anyone wants it. If you want wider/narrower buttons, just change the constant at the top.

• workspace-switcher@cinnamon.org.patch

I had upgraded my MythTV back and front ends recently from version 0.24 and ran into some annoyances:

• The MediaMonitor has a new dependency on the udisks daemon, even if you don’t use the Media Monitor feature.  Without it, Myth had very poor behavior when trying to eject optical media.  The eject option in the menu would just display a dialog saying “No devices to eject.”  Also, the physical eject button would get disabled after playing the disk even though I could eject it from a terminal with “eject” as the mythtv user.  The fallback in MediaMonitor::ejectOpticalDisk() seems to be useless.
• My carefully-crafted channel lineup for Comcast became non-functional.  This is such a tedious process that i still haven’t gotten around to fixing it.  Right now, my digital tuners are recording programs that can’t actually be tuned.

While the merits of Comcast’s Digital Transport Adapters are debatable, there is one useful side effect to their existance. Because the are so cheap, they lack a POD tuner. This means that the channel tables are broadcast in-band, which in turn means that a normal QAM tuner can find them. There is a tool called scte65scan that can find these tables using a regular DVB tuner (it also supports the HDHomerun).

At some point we will become another victim of Comcast’s digital cable “upgrade” plans. We’ve been an analog cable customer of theirs for about 8 years. The only reason we’ve stuck with them is because they are the only provider that can give us good ol’ NTSC on our coax. That is the only way to have all of our cable programming on more than one TV without paying extra each month for each. In addition to the 3 TVs we also use MythTV for our DVR system, which has 3 NTSC and 2 ATSC/QAM tuners.

We are still getting the analog feed, but last week I finally installed the new digital equipment. Their marketing B.S. is confusing and intentionally misleading, but here is what I have learned from the experience:

“Extended Cable” customers are being migrated to the “Digital Starter” package. You get one standard digital cable box, which comes with their “On Demand” feature, and up to two “Digital Transport Adapters”. They give you the impression that the DTAs are you help make up for the loss of analog service on your other TVs, but this isn’t really true (more on this later). After the migration, you will supposedly be left with your primary local stations and a few other useless cable stations like QVC on analog channels 2-17. The rest are only available as digital QAM channels.

This is where it gets ugly. A subset of our old cable lineup is available as unencrypted QAM256 channels, but the others are all encrypted. Some are flagged as encrypted but are actually clear while the rest are fully encrypted. They also sit on random QAM channels/programs that don’t correspond to how they appear in the lineup on the cable box. Comcast is free to move the channels around and play with the encryption whenever they please because they can just reprogram the cable box and DTAs remotely. This makes using your own QAM tuner frustrating, especially since you can’t receive arbitrary basic cable stations due to encryption (like Discovery Channel in my case).

The DTAs are a joke. Comcast makes sure that the channel lineup is only a tiny subset of the lineup you are paying for. They want you to rent real cable boxes, which makes them no better than DirecTV or U-Verse. The DTA will mainly just tune in your “extra” local digital stations in standard definition, but they do NOT tune in your main local stations. For example, if you have a local broadcaster on channel 10, they transmit high-def over the air on channel 10.1 and also probably broadcast a few extra programs on channel 10.2 and 10.3 or more. The DTA will let you view channel 10.2 and 10.3 on your old NTSC TV, but it won’t tune the main channel 10! Comcast provides those as standard def analog stations, but the DTA can’t tune those and the DTA won’t act as a pass-though when it is off like a VCR would (in fact, you can’t turn DTAs off at all). So to watch channel 10, you need an A/B switch on your antenna to switch between the main cable feed and the output of the DTA. The other stations besides the locals that the DTA will receive are fairly useless (more QVC-like stations, Lifetime, and one or two others I’ve never heard of).

So unsurprisingly, Comcast continues to offer less for more. If they provided basic cable stations on unencrypted channels that would be one thing. The only good news in my eyes is that they offer nothing over what you can get from other communication carriers now. All I have to do is put an antenna on the roof.

Release a new CVS snapshot for nedit, mainly to ease installation on top of the snapshot that comes with Ubuntu 9.10.

Version 0.3-1 of notifier-applet has been released.

Changes

• Add length field to message format 1.
• Update notifier.pl example with message format 1 support.