The One with the Thoughts of Frans

Archive for Software

KOReader Beta 20180314

As one of the maintainers of KOReader, a versatile a document and image viewer for E Ink devices, I’m proud to announce the latest release.


This release adds support for the Kobo H2Ov2 thanks to @cairnsh. You need to use at least KSM 9 with update 1.

@dengste added a widget to configure the natural light on the Kobo H2Ov2 and the Kobo Aura One in #3744.

nl_widget

The Android x86 build is now a regular part of the nightly builds (see #3384 but it wasn’t finalized on the build server until koreader/koreader-misc#14).

We’d like to thank all contributors for their efforts. Some highlights include:

Full changelog

CommentsTags:

MariaDB Fix “Can’t init tc log” Error

I messed up my automation of backups, meaning that after two years my entire VPS had secretly filled up. This lead to MariaDB being unable to initialize. After taking care of the root cause MariaDB still refused to start.

$ sudo tail -3 /var/log/mysql/error.log
2018-02-20 12:07:45 140649776292416 [Note] Recovering after a crash using tc.log
2018-02-20 12:07:45 140649776292416 [ERROR] Can't init tc log
2018-02-20 12:07:45 140649776292416 [ERROR] Aborting

Stupidly, just removing the zero byte `/var/lib/mysql/tc.log` file took care of the problem.

CommentsTags:

Improving Gource Videos with Background and Audio

Gource videos have been popping up on my radar for several years now, but for some reason I decided to play around with it yesterday. My tweaks were very minor: much increased speed, avatars, a logo, a background image, and some music to go along with it.

I started with the avatars. To grab them, I used this script with a quick replacement for the error handling:

    if($rc != 200) {
        unlink($author_image_file);

        my $gh_url = "https://avatars.githubusercontent.com/".$author."?size=".$size; 

        warn "fetching gh image for '$author' $email ($gh_url)...\n";

        my $rc = getstore($gh_url, $author_image_file);

        next;
    }
    sleep(1);

That way users without Gravatars but a GitHub username in their commit message still get the one you’re used to seeing. I manually deleted all of the default GitHub no avatar images. That sounds like a lot of work, but you can just sort by size.

I did a quick inversion of the KOReader logo in GIMP. Nothing special there.

The background image I grabbed from Pixabay, a useful site for free photographs. The music I got from dig.ccmixter.org, something similar for music.

Now for the pièce de résistance, thanks to the Gource wiki:

gource -1920x1080 -a 0.1 -s 0.05 -i 0 --logo koreader-logo-inverted.png --user-image-dir .git/avatar/ --background-image fairyland-canyon-1632749_1920.jpg -o - | ffmpeg -y -r 60 -f image2pipe -vcodec ppm -i - -vcodec libx264 -preset ultrafast -pix_fmt yuv420p -crf 1 -threads 0 -bf 0 gource.mp4

The most important aspect is the resolution and how fast or slow you want the video to be. Then once you have the video, you can mix in the audio:

ffmpeg -i gource.mp4 -i mindmapthat_-_Transmutation.mp3 -c:v copy -c:a aac -strict experimental -filter:a "volume=0.6" gource-muxed.mp4

Note that I lowered the volume of the music, more on that here. You could also play around with fade-outs, whether in Audacity or straight from ffmpeg. Also see my post on lossless audio cutting for when everything is good but you just want to trim the audio a little. That’s all!

CommentsTags: ,

Six Years of KOReader

This is a little experiment with Gource. Direct link to video here. KOReader is a document viewer for E Ink devices. It supports PDF, DjVu, XPS, CBT, CBZ, FB2, PDB, TXT, HTML, RTF, CHM, EPUB, DOC, MOBI, and ZIP files. It currently runs on Kindle, Kobo, PocketBook, Ubuntu Touch and Android devices. More information here.

Credits

Background: https://pixabay.com/nl/sprookjesland-canyon-utah-park-1632749/

Music: Transmutation by Kara Square (c) copyright 2017 Licensed under a Creative Commons Attribution (3.0) license. http://dig.ccmixter.org/files/mindmapthat/56527 Ft: Spinningmerkaba

Also see…

Back in 2015 Landon Wilkins uploaded a visualization here.

Comments (2)Tags:

KOReader Nightly 20180212

As one of the maintainers of KOReader, a versatile a document and image viewer for E Ink devices, I’m proud to announce the latest release.


Special thanks to @dengste this month for adding natural light support on the Kobo Aura One (#3679).

You can control the natural light feature on the Kobo Aura One through the regular frontlight widget and the frontlight gesture controller. The latter allows for controlling warmth by swiping on the right of the screen, and is accessible via ToolsMore pluginsFrontlight gesture controller.

Those who read two-column papers might like the new two-column navigation (@FranMarelli This does not work in conjunction with scroll mode.

Those willing to experiment can try the new Open with… option (#3651).

Finally, a little clarification on last month’s HTML dictionary support. We use MuPDF to render the HTML dictionary results. Unfortunately, MuPDF expects its input to be well-formed XHTML, meaning XML. When the HTML is tag soup instead, KOReader will fall back to a stripped version of the HTML dictionary entry. Should this be the case, you can manually fix up the output by adding a .lua file in the dictionary directory. Further details and a few examples are provided by @poire-z in #3611.

We’d like to thank all contributors for their efforts. Some highlights include:

  • Fix speedup of dictionaries init (@poire-z
  • Swipe down or tap on top left/right to access left/right reader menu tab directly (@nagyation
  • Menu: configure number of items per page, wrap entries (@robert00s
  • OPDS – add confirm when file exist in local destination (@robert00s
  • HTML dictionary link support (@TnS-hun
  • Update History and Last open document when move (cut and paste) file (@poire-z
  • Do not execute background runner if device is suspended (@Hzj-jie
  • [feat] Add NetworkManager:isConnected() (@Frenzie
  • Filemanager – sort by percent of book (@robert00s
  • [UX] Move “Go back to previous location” up one menu level (@Frenzie
  • Wikipedia: show images and allow interrupting queries (@poire-z
  • luasettings: backup previous file and auto-recover (@poire-z
  • epub metadata: support multiple authors (@poire-z
  • HtmlBoxWidget: fix selection when starting from end (@poire-z
  • Bump base: fix gray background instead of white on Kindle (@poire-z
  • Add MovableContainer: allow moving some widgets (@poire-z
  • Add menu item for toggling auto_book_status (@poire-z
  • [UX] Dict: scroll back with tap: go to bottom of prev definition (@poire-z
  • NewsDownloader: better exception handling and basic redirects (@mwoz123
  • [feat] Open with: choose which engine to use for file (@Frenzie
  • Menu: speed up rendering of long texts (@robert00s
  • cre full text search: fix inconsistencies (@poire-z
  • kobolight.plugin: Support for setting natural light (@dengste
  • Show vertical marker at original position when back from link (@poire-z
  • File manager: added ‘Go to letter’ (@robert00s
  • cre cache: allow for disabling compression (@poire-z
  • [feat] Two-column navigation (@FranMarelli
  • [fix] larger max font size for crengine (@Frenzie

Full changelog

CommentsTags:

Why Windows Is Better Than PulseAudio

Wait, what? You heard me. On Windows you can set sample rates per sound card. It looks like this.

Setting one soundcard to 24 bit, 96 kHz in Windows. Windows refers to this as “Studio Quality.”

In PulseAudio you’re limited to a primary and secondary sample rate. I’ve actually been using pasuspender more, also because its AC3 passthrough never seemed to work.

CommentsTags: ,

Is My Firefox Out of Date?

When I logged in, WordPress tried to inform me that the particular browser I was using was out of date.

WordPress’ “Your browser is out of date” greeting.

So, is it?

Well, we can take a look at the Firefox ESR download page to find out.

The current version of Firefox ESR is 52.4.1, releasenotes here.

Nope. Still Firefox 52, which will remain supported until Firefox 60.

The Firefox support window. Source: https://www.mozilla.org/en-US/firefox/organizations/faq/

Always fun, these automated checks. 🙂

Comments

Steam Error: “invalid depot configuration”

I just had to share this mysterious error message.

A screenshot of a Steam error dialog that says invalid depot configuration.

Simply retrying made it go away.

Comments

Image Optimization Guide

On the forum I administer, I am forced to run a tight attachment policy. Disk space doesn’t grow on trees. Occasionally this leads to questions about the small attachment size limit of 50 KiB. This guide is intended to clarify that this is not nearly as tiny as you might think. Note that although I’ll mention commands without much explanation for the sake of brevity, you’re always recommended to further explore the possibilities offered by those commands with the --help flag as well as by running man the-command-here.

First you need to ask yourself what kind of file type is appropriate, if you have the choice. On screenshots, the main purpose of attachments on my forum, you’ll often encounter large areas of uniform background colors. PNG is therefore almost invariably the right choice. Crop out everything but what’s relevant. JPEG is appropriate for more dynamic pictures such as photographs. If you want to do a lot with photographs, you might want to consider an external hosting service. My wife likes SmugMug. Still, for thumbnails you might be able to do a fair bit more within a few hundred KiB than you might think. Finally, the vector graphics in SVG result in pictures that always look sharp. You’ll typically have drawn these in a program like Inkscape or Adobe Illustrator.

Table of Contents

  1. 1. Optimizing JPEG
  2. 2. Optimizing PNG
  3. 3. Optimizing SVG
  4. Addendum A: Scanned Documents
  5. Addendum B: Video

1. Optimizing JPEG

Often you’ll want to crop your file. Do not edit your JPEG followed by resaving it because this will result in reduced quality! You can crop losslessly with cropgui. On Windows you can use IrfanView.

If you don’t want to crop, and also potentially for some post-cropgui optimization, use jpegtran -copy none -progressive -optimize file.jpg > file-opt.jpg. Note that this will get rid of all metadata, which may be undesirable. If so, use jpegtran -copy all -progressive -optimize file.jpg > file-opt.jpg.

Of course if you want to scale down your JPEG there’s no point in mucking about with lossless cropping first. After scaling down, check how low your quality can go (also see a little helper script I wrote). In any case, you should avoid introducing any unnecessary compression steps with associated quality loss. Here are some results:

  • The original 11.jpg at 2.19 MB.
  • Losslessly cropped 11-crop.jpg at 1.11 MB.
  • Optimized with -copy all -progressive -optimize 11-crop-opt.jpg at 1.04 MB. -copy none would’ve saved an extra whopping 40-some KiB, which on this kind of filesize has little benefit, and besides, I quite like the metadata. For thumbnail-sized files the balance is likely to be different. For example, the 52.2 KiB SmugMug auto-generated thumbnail below can be insignificantly reduced to 51.1 KiB with --copy all, but to 48.2 KiB with --copy none. I think an 8% reduction is not too shabby, plus it brings the file size down to under the arbitrary 50 KiB limit on my forum.

2. Optimizing PNG

As I wrote in the introduction, for screenshots PNG is typically the right choice. If you want to use lossless PNG, use optipng -o7. In my experience it’s ever so slightly smaller than other solutions like pngcrush. But as long as you use a PNG optimizer it shouldn’t much matter which one you fancy. Also see this comparison.

If you don’t care about potentially losing some color accuracy, use pngquant instead. To top it off, if you really want to squeeze out your PNG, you can pass quality settings with --quality min-max, meaning you can pass --quality 30-50 or just --quality 10. Here are some quick results for the screenshot in the SVG section below, but be sure to check out the pngquant website for some impressive examples.


$ du -h --apparent-size inkscape-plain-svg.png
27K	inkscape-plain-svg.png

$ du -h --apparent-size inkscape-plain-svg-fs8\ default.png 
7.6K	inkscape-plain-svg-fs8 default.png

$ du -h --apparent-size inkscape-plain-svg-fs8\ quality\ 10.png 
4.3K	inkscape-plain-svg-fs8 quality 10.png

In this case there is no visual distinction between the original PNG and the default pngquant settings. The quality 10 result is almost imperceptibly worse unless you look closely, so I didn’t bother to include a sample.

3. Optimizing SVG

For using SVG on the web, I imagine I don’t have to tell you that in Inkscape, you should save your file as Plain SVG.

Save as Plain SVG in Inkscape.

What you may not know is that just like there are lossy PNGs, you can also create what amounts to lossy SVGs. There are some command-line tools to optimize SVGs, including (partially thanks to this SO answer):

  • Scour is probably the best command line tool for some quick optimization. You can just use the defaults like scour < in.svg > out.svg or scour -i in.svg -o out.svg. But I recommend you go further.
  • SVGO (SVG Optimizer)
  • SVG-optimiser (by Peter Collingridge)
  • SVG-editor (by Peter Collingridge)

My personal preference for squeezing out every last byte goes toward the web-based version of the SVG-editor by Peter Collingridge. By running it in a browser with inferior SVG support such as Firefox, you’ll be sure that your optimized SVG still works properly afterward. The command line tools can only safely be used for basic optimizations, whereas the effects of going lossy (such as lowering precision) can only be fully appreciated graphically.

Addendum A: Scanned Documents

Scanned documents are a different item altogether. The best format for private use is DjVu, but for public sharing PDF is probably preferable. To achieve the best results, you should scan your documents in TIFF or PNG, followed by processing with unpaper or ScanTailor. If you’ve already got a PDF you’d like to improve, you can use pdfsandwich or my own readablepdf.

Addendum B: Video

I’m not aware of any lossless optimization for video compression such as offered by jpegtran, but you can often losslessly cut video. In the general purpose editor Avidemux, simply make sure both video and audio are set to copy. There is also a dedicated cross-platform app for lossless trimming of videos called, unsurprisingly, LosslessCut. If you do want to introduce loss for a smaller file size you can use the very same Avidemux with a different setting, ffmpeg, mpv, VLC, and so forth. You can get reasonable quality that’ll play many places with something like:

ffmpeg -i input-file.ext -c:v libx264 -crf 19 -preset slow -c:a libfaac -b:a 192k -ac 2 output-file.mp4

For the open WebM format, you can use something along these lines:

ffmpeg -i input.mp4 -c:v libvpx -b:v 1M -c:a libvorbis output.webm

More examples on the ffmpeg wiki. Note that in many cases you should just copy the audio using -acodec copy, but of course that’s not always an option. Extra compression artifacts in audio detract significantly more from the experience than low-quality video.

Comments (1)Tags:

UNetbootin Custom Drive Selection

UNetbootin has been broken for many, many years, but just today (a few years after the fact) I discovered that the previous GUI option to show all drives was readded as a command line option. So if the program doesn’t want to detect your drive, just use the targetdrive argument:

unetbootin targetdrive=/dev/sdf1

And voila, it’s working. I have no idea why it should have to be so difficult. The program categorically refuses to detect any of my USB flashdrives or harddrives, so since the removal of show all drives it’s been utterly useless.

PS This is basically only for Windows ISOs. For everything else you can just use, e.g., dd. Much easier.

Comments

« Newer EntriesOlder Entries »