The Blog of Tom Webster

Chronic Ranter, Reviewer, and Developer

watchtwitch 4.0.0 - Command Line Twitch.TV Browser

  2016-10-14 21:44:32 EDT

video_game video_game video_game video_game video_game video_game video_game video_game video_game video_game

Yet another updated to watchtwitch, this time removing the dependency on streamlink (or livestreamer). watchtwitch is now able to load up streams all on its own. This is still a new feature, so it may not work 100% of the time, so I've included a -fallback flag that uses streamlink to grab the streams.

Other than that, I've switch the default media player from VLC to MPV.


As always, you can grab the code here on GitLab. If you'd like a build for a different platform, let me know in the comments.

video_game video_game video_game video_game video_game video_game video_game video_game video_game video_game

Let's Encrypt Automation

  2016-09-16 09:46:43 EDT

Originally I had drafted out this post to show and explain a shell script that I created to automatically handle Let's Encrypt certificate issuance and renewal, but honestly, it was a messy hack. A far better solution is Hugo Landau's (hlandau) acmetool.

It is trivial to install, easy to use, and it keeps things updated automatically (even without root if you choose). The official documentation is awesome and you should check it out.

Scripting FreeBSD pkg Installs

  2016-05-18 17:34:41 EDT

I needed to bootstrap pkg and install some packages in FreeBSD via a script, but the pkg was getting hung up on the interactive element. Thanks to this forum post (which references this blog post, I now can.

Use env ASSUME_ALWAYS_YES=YES pkg bootstrap to get pkg up and running, you can use that same env line to install packages without any prompts, like this:

env ASSUME_ALWAYS_YES=YES /usr/sbin/pkg install git zsh curl

pkg won't prompt you and will install the packages. Great for scripting or ezjail flavours.

An Introduction to Tor Hidden Services

  2016-05-14 11:02:49 EDT

Sometimes you'd like to stand up a site on the internet, but you don't want that IP address exposed to the world. There are several reasons for this, many of them not nefarious in nature. What if you'd like to post about your govenment while living in a country with an authoritarian regime? What if you're part of a group of freedom fighters in your country, soliciting bitcoin donations? There are many reasons to want a hidden service, in this post I'll introduce you to them and some common pitfalls associated with their creation.

Tor hidden services are internet services (like websites, chat servers, game servers, etc) that can only be accessed via the Tor network. The easiest, and safest way to access Tor websites (called onion sites because of the .onion domain name) is with the Tor Browser Bundle.

We'll be using a Debian Stable (Jessie) system to create our Tor hidden service. This can be a real box, a hosted system on one of the cloud providers like Digital Ocean or AWS, or a virtual machine living on your local system. Because the traffic will be traveling through the Tor network, you can even host your hidden server behind a NAT router. In this guide, I assume that you'll be running as a normal user with sudo access, because running as root all the time is bad karma.

Let's start with updating our package list so we can get the latest and greatest software:

sudo apt-get update

Next, let's install Tor, an ssh server (if one hasn't been installed already), a web server, and a persistent firewall:

sudo apt-get install tor openssh-server nginx iptables-persistent

While you're installing iptables-persistent, it'll ask if you would like to save the current firewall rules. Answer "Yes" to both IPv4 and IPv6 rules. This will create the firewall rules files we'll be working with later.

First, let's configure Tor and set up our hidden services.

sudo nano /etc/tor/torrc

We'll want to add two hidden service configurations to this file, one for our website to live at and a different one for ssh to live. While Tor does allow multiple ports to exist within the same hidden service, we want to make discovering our ssh port a bit more difficult. If someone hates your site and wants to brute force into it, they'll have to look around for your ssh onion address, instead of using the same address your site uses.

Add these lines to your torrc file:

HiddenServiceDir /var/lib/tor/website/
HiddenServicePort 80
HiddenServiceDir /var/lib/tor/management/
HiddenServicePort 22

What these lines do is set up two different hidden services, they'll both have different .onion addresses and different private keys. You can run a ton of different services on one machine if you'd like. One hidden service will expose port 80 and listen for traffic on port 80 of the loopback address, the other will do the same, but with port 22 instead.

Next, let's configure SSH for public-key only authentication and to disallow root access:

sudo nano /etc/ssh/sshd_config

You'll want to change four items to the config shown below:

ListenAddress ::1


PermitRootLogin no

PasswordAuthentication no

At this point, do not reboot, especially if you're only able to access your server via SSH, you will be locked out. Just follow the rest of the guide.

This will disallow root login via ssh, you should be running as a normal user with sudo privileges, and you should be using ssh keys and not passwords. These two items are very important security-wise, but are out of the scope of this guide.

The ListenAddress is very important when setting up your Tor hidden service. SSH creates several unique keypairs for your ssh server, this is to help prevent man-in-the-middle attacks, if the key changes from what you've already established, you know something shady is going on. There are people/groups/agencies on the net that scan and collect these public keys. If you are exposing ssh to the public internet and running a tor hidden service with ssh enabled, it is trivial to compare scans of the tor network and scans of the internet to find your true IP address. We don't want to expose ANYTHING to the public internet. Your hidden server should expose tor hidden services only, no cleartext connections allowed.

Now that SSH is set up, let's set up our web server:

sudo nano /etc/nginx/site-enabled/default

This one is easy, we're going to remove this line:

listen [::]:80 default_server;

And change this line:

listen 80 default_server;

To this:

listen default_server;

The listen changes have the same effect as the SSH ones, we don't want to publicly blast out our website content to the net, otherwise, why would we need a hidden service?

Now, let's configure the firewall, just for extra insurance:

sudo nano /etc/iptables/rules.v6

The IPv6 rules are easy, we're just going to drop all traffic. Replace the contents of the file with this:


The IPv4 rules are less easy, but not crazy. Replace the contents of the file with this:

# Allow basic browsing and session continuation
# Allow all loopback traffic
-A INPUT -i lo -j ACCEPT

These rules will allow loopback traffic, which is required for our hidden service. We're also allowing outbound traffic, then the established and related traffic to our outbound connections. This allows us to get out to the internet, without people reaching in.

Now for the moment of truth, let's restart our services one by one so the changes will take effect:

sudo systemctl restart tor

Now, let's make a note of our two onion addresses:

sudo cat /var/lib/tor/website/hostname

sudo cat /var/lib/tor/management/hostname

Make a note of those two addresses, you'll need them to access your hidden website and hidden ssh server.

Next, restart your ssh server, your webserver, and lastly, your firewall:

sudo systemctl restart ssh

sudo systemctl restart nginx

sudo systemctl restart netfilter-persistent

Your already-established ssh connection should stay up at this point. Now, let's test to make sure you can ssh to your management hidden service. To do this, though, you'll need Tor installed on your local system and a rule in your local ssh configuration.

On your local system, install socat, it's a multipurpose socket relay, it'll allow us to ssh through the tor network to hit your hidden service. We'll also install Tor.

sudo apt-get install socat tor

Now, let's make a rule to allow all *.onion addresses to use socat and tor to get where we want it to go:

nano ~/.ssh/config

Add this block to your ~/.ssh/config:

# Proxy .onion connections through tor
Host *.onion
    ProxyCommand socat - SOCKS4A:localhost:%h:%p,socksport=9050

Now, start Tor on your local system:

sudo systemctl start tor

By default, Tor will listen on port 9050 for any socks connections.

Now, ssh to your management onion address, just like a standard ssh connection. It will be laggy, very laggy, but you'll be connected to your server on your tor hidden service. You can now manage your server over a tor connection.

Now, on your local system grab the Tor Browser Bundle and install it. In the Tor Browser, open up your website onion address, you should see a default Debian nginx page.

You're now up and running! Restart your server to get rid of any hanging connections, the services should start automatically.

Here's your official disclaimer: Tor isn't perfect, it's good, it's great, the Tor Project does amazing work and it's the best answer to being anonymous on the internet that we have today, but Tor isn't a panacea, neither is this guide. I've tried to steer you away from any major pitfalls, but I'm not perfect. If you're building Silk Road number 72, you probably need more than this guide to protect yourself.

If you find any issues with this guide, leave a comment, a pull request, or hit me up on Twitter. Together we can make this better.

Here's some more great info from the Tor Project on hidden services:

GoLang: Slice contains and dedup

  2016-05-11 18:19:17 EDT

The GoLang Gopher was created by Renee French and is licensed under the Creative Commons 3.0 Attributions license. Check out this blog post for details.

Here's a simple GoLang demonstration showing how to see whether or not a slice contains a value, then how to deduplicate the items in that slice. You can easily run this on the Go Playground, check it out!

package main

import "fmt"

func contains(intSlice []int, searchInt int) bool {
    for _, value := range intSlice {
        if value == searchInt {
            return true
    return false

func dedup(intSlice []int) []int {
    var returnSlice []int
    for _, value := range intSlice {
        if !contains(returnSlice, value) {
            returnSlice = append(returnSlice, value)
    return returnSlice

func main() {
    numbers := []int{1, 2, 2, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6}
    fmt.Println("Numbers:", numbers)
    fmt.Println("Numbers contain 5:", contains(numbers, 5))
    fmt.Println("Numbers contain 8:", contains(numbers, 8))
    fmt.Println("Deduped numbers:", dedup(numbers))
Page: 1 of 23