The Blog of Tom Webster

Chronic Ranter, Reviewer, and Developer. I speak only for myself, opinions are my own.

The Most Evil Thing Imaginable: Mimic

  2015-10-30 03:50:03 PDT

This is just truly evil, and amazing to witness. Greg Toombs' (reinderien) new project Mimic. It randomly replaces characters in someone's source code with unicode-lookalikes. Just the other day I had to deal with a dataset that had some unicode spaces in it and I thought I might pull my hair out.

Anyway, awesome project and it does include a "Reverse Mimic" function, so you can fix your files if you suspect someone has mimicked them. Go check it out.

Writing about Jekyll Tags in Jekyll

  2015-10-28 03:08:45 PDT

Just ran into an annoying situation. How do you write about Jekyll tags inside of Jekyll? How do you get it to avoid parsing?

The best answer I've found is this Stack Overflow answer.

Updated 2019-01-29: This post keeps causing errors in recent versions of Jekyll. Just check the Stack Overflow link.

New Theme: Amber

  2015-10-28 02:30:00 PDT

Thanks to Stuart Webster for the awesome background! - CC-BY-2.0

As you may have noticed (unless you're reading this through the RSS feed or a screen reader), this site has undergone a major theme update. I'm calling it "Amber", because of the color choice. I took inspiration from various places on the web to come up with this design. This still uses the awesome Twitter Bootstrap under the hood. It just makes things too easy to build to get rid of.

My favorite feature, aside from the cleaned up navigation and better-looking site, is the per-post-backgrounds. This is accomplished with the code below. Pretty simple, if a bg element isn't present (or set to false), the default background is shown, otherwise show the image specified in the bg YAML Front Matter item.

{% if page.bg %}
  <header style="background-image: url(/images/{{page.bg}});">
{% else %}
 <header style="background-image: url(/images/bg.jpg);">
{% endif %}

I've gotten rid of glyphicons in favor of FontAwesome. It's way better to have just one icon-font library instead of two, faster page loads.

One of my favorite features (that was made before this theme was launched) is the compressed cache-busting asset-cruncher. The plugin is stupid, it takes all of your CSS, JS, and fonts from _assets, compresses what it can, then copies the compressed assets into assets that Jekyll uses to build the site. The result is a single CSS and a single JS file are used, brining the number of total requests down and site speed up. This is hugely helpful until HTTP/2 takes over everywhere. The plugin isn't perfect by any stretch of the imagination, it really needs to be tuned up to run only when it's supposed to. Right now, it doesn't re-fire when Jekyll runs in --watch mode, meaning you have to CTRL+C and re-launch jekyll serve for CSS/JS updates to take effect on your local system. Merge requests are appreciated if you have the time.

As always, you can get the code to my site on GitLab and if you'd like to base your Jekyll site off of mine, you can do so easily with the deployable version.

Gamestation

  2015-10-04 20:02:32 PDT

Thanks for the awesome background libretro!

About

Project Gamestation took a major backseat when I discovered RetroPie. petRockBlog did a fantastic job putting everything together and making it "Just Work". I seriously recommend checking it out. When it comes to gaming on the Raspberry Pi, RetroPie did everything I could have wanted and more, but it's really designed for the Pi, not for x86/x64 systems.

That's where Gamestation comes in. The project is really nothing special. It's a couple shell scripts that automate fetching the repos, building other people's projects, creating a Debian package, and installing it. It's not a major piece of work, but it makes my life easier and has resulted in a great double-click-easy system that I can hook to my TV that has a bit more power than a Pi 2.

The best part of this is combining it with Syncthing. I always hated how my roms and saves would differ between machines, requiring a central folder somewhere on ownCloud or manually syncing. Syncthing is like Bittorrent Sync, but open source and well respected. By using this and a centralized folder structure, you can keep all roms/saves/themes/metadata syncronized between Gamestation machines. It works really well and is useful for bootstrapping new machines.

Installation

Right now Gamestation is only supported by Debian and Debian-family Linux distributions. I'm 98% sure you can get it running on just about any *nix OS out there, but I can't claim to have tested it. If you'd like to add other OS support, check out the GitLab Repo.

Grab the .deb here and install with sudo dpkg -i. If your system complains about dependencies, try sudo apt-get -f install to find and install the missing libraries.

Gamestation expects the following folders to be in ~/roms/:

amiga
amstradcpc
apple2
atari2600
atari5200
atari7800
atari800
atarilynx
atarist
c64
coco
dragon32
fba
fds
gamegear
gb
gba
gbc
intellivision
macintosh
mame-advmame
mame-mame4all
mastersystem
megadrive
msx
n64
nds
neogeo
nes
ngp
pc
pcengine
ports
psx
quake3
scummvm
sega32x
segacd
sg-1000
snes
vectrex
videopac
wonderswan
zmachine
zxspectrum

And a symlink from ~/roms/megadrive to ~/roms/genesis: ln -s ~/roms/megadrive ~/roms/genesis

To actually run the system, use the command emulationstation.

Meta

As usual, the project is open source, grab the code here.

This project is licensed under the GNU GPL v3.

If you're building a nightly-release, check out the source code's README.md. It'll walk you through how to build one from source and end up with a Debian package of your own.

TODO

There's still a lot of work to be done. If you'd like to make this project better, send me a merge request.

  1. Windows support - This will make it easy for non-techies to get their emulators up and running easily.
  2. RetroPie currently makes controller bindings between EmulationStation and Retroarch seamless (for the most part). I'd love to port that over.
  3. Create a build-bot system that puts out weekly beta builds of GameStation. Ideally, this would be an ansible playbook or something equally as automated.
  4. Automated Syncthing setup. Currently, Syncthing is pretty geared towards techies. If mass adoption is going to be possible, this part will need to be easier to set up. Maybe a web service that links IDs together through the Syncthing API? Lots to consider.
  5. PIPE DREAM - I'd LOVE to get this working on a PiGRRL with wifi. Take your roms and saves ANYWHERE! Sync when you hit wifi.

Syncthing Setup

  1. Follow the instructions on Syncthing's Debian page.
  2. Create a service file for syncthing:
    • sudo cp /usr/lib/systemd/system/syncthing@.service /etc/systemd/system/syncthing@`whoami`.service
    • If you aren't on a system with systemd yet, use this init file.
  3. Go to your local syncthing management page at http://localhost:8384/.
  4. Click Add Folder and add ~/roms/, feel free to set up file versioning if you'd like.
  5. Click Add Folder and add ~/.emulationstation, feel free to set up file versioning if you'd like.
  6. Use Actions -> Show ID to show your device's code.
  7. On a new device, install Syncthing and click Add Device, paste in your first device's code and give it a name.
    • Adding your first device as an "Introducer" is pretty helpful, it can act like a central point for all things Syncthing in your setup.
  8. On the original device, Syncthing will eventually ask you if you'd like to trust the new device you've added. Go ahead and trust it, then share the .emulationstation and ~/roms/ folders with it.
  9. Synchronization should happen automatically.
  10. Install Gamestation on the new system, your saves, roms, and themes should all be synced between all Gamestation devices.

Credits

This project wouldn't be possible without the amazing work by the following projects:

For full details and credits, check out the README.md file.

Easy Encryption with SSDs and HDDs in the same machine

  2015-09-20 13:12:35 PDT

If you have a Linux machine with and SSD for one part of the filesystem, but need HDDs for the large storage capacity, encryption can become a pretty huge pain.

If you encrypt multiple filesystems across multiple disks, LVM is the proper choice, but you have a solid state disk you want to keep for it's intended purpose: Booting your system quickly and making applications launch as fast as possible. If you keep all drives in LVM, some data will end up on physical volumes and slow your rig down.

So how do we boot our system without needing two or more LUKS passphrases on boot? How do we make it so one password rules them all?

Enter crypttab.

/etc/crypttab is like fstab for your encrypted filesystem components, and it's really easy to get the hang of. Just a disclaimer: Using a single LUKS passphrase to unlock all drives is technically less safe than using a different passphrase for each drive, but it is way more convenient. That's the ever-long battle: Convenience vs Security.

If you have cryptsetup installed, you should have /etc/crypttab in place already, just with everything commented out. The provided examples make this pretty easy to figure out:

# <name>       <device>                                     <password>              <options>
# home         UUID=b8ad5c18-f445-495d-9095-c9ec4f9d2f37    /etc/mypassword1
# data1        /dev/sda3                                    /etc/mypassword2
# data2        /dev/sda5                                    /etc/cryptfs.key
# swap         /dev/sdx4                                    /dev/urandom            swap,cipher=aes-cbc-essiv:sha256,size=256
# vol          /dev/sdb7                                    none

From this file, we can see that the LUKS volume named home has a specific UUID and a keyfile located at /etc/mypasswd1. The swap LUKS volume is encrypted randomly on each boot by /dev/urandom. The vol LUKS volume has none in the password field, meaning it will ask you for a password at mount time.

With crypttab, we can use the combination of a single passphrase for the root drive (your SSD), then keyfiles for the rest of the encrypted hard drives.

  1. Install your Linux system normally, on an encrypted LVM on the SSD.
  2. Create a new key file for your new drive. We're going to use /dev/urandom and make a 5MB base64-encoded keyfile. While it would be more secure to use /dev/random, this will take a very very long time. Use it if you feel it is neccessary, but keep in mind, this is a single-passphrase boot, if your passphrase is poor, no amount of /dev/random will save you.
    • dd if=/dev/urandom bs=1M count=5 | base64 > ~/.HDDkey
  3. Now we need to format your hard drive with the key you just created:
    • cryptsetup luksFormat -d ~/.HDDkey /dev/sde
  4. Now map it, format it, then unmap it:
    • cryptsetup luksOpen -d ~/.HDDkey /dev/sdd BigStorage
    • mkfs.ext4 -L BigStorage /dev/mapper/BigStorage
    • cryptsetup luksClose BigStorage
  5. Now find the UUID of your drive:
    • blkid
    • Find the device identifier of your new encrypted drive, in my case, mine is /dev/sde.
    • Copy out the UUID, mine is c7792c2a-78fb-425a-8971-6df1c5d5b79c.
  6. Now, let's add this line to /etc/crypttab so it will automatically unlock /dev/sde when the /etc filesystem is available:
    • BigStorage UUID=c7792c2a-78fb-425a-8971-6df1c5d5b79c /home/samurailink3/.HDDkey
    • BigStorage is going to be the name of the LUKS device exposed in /dev/mapper.
  7. Now the device will be mapped on boot and ready to mount. Here's what a line in your /etc/fstab should look like if you want to mount it somewhere specific, with user and exec access.
    • /dev/mapper/BigStorage /run/media/samurailink3/BigStorage ext4 user,exec 0 0
  8. Now all that's left is to use the device. If you're treating it like an external drive, there's nothing you need to do now, the device will be available at /run/media/samurailink3/BigStorage/. What I like to do is symlink out folders from my existing home directory to the larger drive for big files that don't need fast access, like video files or music. Here's an ls -l ~ for an example:
lrwxrwxrwx  1 samurailink3 samurailink3      44 May 24 09:33 SteamLibrary -> /run/media/samurailink3/BigData/SteamLibrary
lrwxrwxrwx  1 samurailink3 samurailink3      35 May 24 09:33 tmp -> /run/media/samurailink3/BigData/tmp
lrwxrwxrwx  1 samurailink3 samurailink3      38 May 24 09:33 Videos -> /run/media/samurailink3/BigData/Videos

Now you have all of your system drives encrypted, with one passphrase. Pretty convenient and way more secure than running with just one drive encrypted.

Page: 6 of 31