The Blog of Tom Webster

Chronic Ranter, Reviewer, and Developer

Scripting FreeBSD pkg Installs

  2016-05-18 17:34:41 EDT

I needed to bootstrap pkg and install some packages in FreeBSD via a script, but the pkg was getting hung up on the interactive element. Thanks to this forum post (which references this blog post, I now can.

Use env ASSUME_ALWAYS_YES=YES pkg bootstrap to get pkg up and running, you can use that same env line to install packages without any prompts, like this:

env ASSUME_ALWAYS_YES=YES /usr/sbin/pkg install git zsh curl

pkg won't prompt you and will install the packages. Great for scripting or ezjail flavours.

An Introduction to Tor Hidden Services

  2016-05-14 11:02:49 EDT

Sometimes you'd like to stand up a site on the internet, but you don't want that IP address exposed to the world. There are several reasons for this, many of them not nefarious in nature. What if you'd like to post about your govenment while living in a country with an authoritarian regime? What if you're part of a group of freedom fighters in your country, soliciting bitcoin donations? There are many reasons to want a hidden service, in this post I'll introduce you to them and some common pitfalls associated with their creation.

Tor hidden services are internet services (like websites, chat servers, game servers, etc) that can only be accessed via the Tor network. The easiest, and safest way to access Tor websites (called onion sites because of the .onion domain name) is with the Tor Browser Bundle.

We'll be using a Debian Stable (Jessie) system to create our Tor hidden service. This can be a real box, a hosted system on one of the cloud providers like Digital Ocean or AWS, or a virtual machine living on your local system. Because the traffic will be traveling through the Tor network, you can even host your hidden server behind a NAT router. In this guide, I assume that you'll be running as a normal user with sudo access, because running as root all the time is bad karma.

Let's start with updating our package list so we can get the latest and greatest software:

sudo apt-get update

Next, let's install Tor, an ssh server (if one hasn't been installed already), a web server, and a persistent firewall:

sudo apt-get install tor openssh-server nginx iptables-persistent

While you're installing iptables-persistent, it'll ask if you would like to save the current firewall rules. Answer "Yes" to both IPv4 and IPv6 rules. This will create the firewall rules files we'll be working with later.

First, let's configure Tor and set up our hidden services.

sudo nano /etc/tor/torrc

We'll want to add two hidden service configurations to this file, one for our website to live at and a different one for ssh to live. While Tor does allow multiple ports to exist within the same hidden service, we want to make discovering our ssh port a bit more difficult. If someone hates your site and wants to brute force into it, they'll have to look around for your ssh onion address, instead of using the same address your site uses.

Add these lines to your torrc file:

HiddenServiceDir /var/lib/tor/website/
HiddenServicePort 80 127.0.0.1:80
HiddenServiceDir /var/lib/tor/management/
HiddenServicePort 22 127.0.0.1:22

What these lines do is set up two different hidden services, they'll both have different .onion addresses and different private keys. You can run a ton of different services on one machine if you'd like. One hidden service will expose port 80 and listen for traffic on port 80 of the loopback address, the other will do the same, but with port 22 instead.

Next, let's configure SSH for public-key only authentication and to disallow root access:

sudo nano /etc/ssh/sshd_config

You'll want to change four items to the config shown below:

ListenAddress ::1

ListenAddress 127.0.0.1

PermitRootLogin no

PasswordAuthentication no

At this point, do not reboot, especially if you're only able to access your server via SSH, you will be locked out. Just follow the rest of the guide.

This will disallow root login via ssh, you should be running as a normal user with sudo privileges, and you should be using ssh keys and not passwords. These two items are very important security-wise, but are out of the scope of this guide.

The ListenAddress is very important when setting up your Tor hidden service. SSH creates several unique keypairs for your ssh server, this is to help prevent man-in-the-middle attacks, if the key changes from what you've already established, you know something shady is going on. There are people/groups/agencies on the net that scan and collect these public keys. If you are exposing ssh to the public internet and running a tor hidden service with ssh enabled, it is trivial to compare scans of the tor network and scans of the internet to find your true IP address. We don't want to expose ANYTHING to the public internet. Your hidden server should expose tor hidden services only, no cleartext connections allowed.

Now that SSH is set up, let's set up our web server:

sudo nano /etc/nginx/site-enabled/default

This one is easy, we're going to remove this line:

listen [::]:80 default_server;

And change this line:

listen 80 default_server;

To this:

listen 127.0.0.1:80 default_server;

The listen changes have the same effect as the SSH ones, we don't want to publicly blast out our website content to the net, otherwise, why would we need a hidden service?

Now, let's configure the firewall, just for extra insurance:

sudo nano /etc/iptables/rules.v6

The IPv6 rules are easy, we're just going to drop all traffic. Replace the contents of the file with this:

*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT DROP [0:0]
COMMIT

The IPv4 rules are less easy, but not crazy. Replace the contents of the file with this:

*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
# Allow basic browsing and session continuation
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
# Allow all loopback traffic
-A INPUT -i lo -j ACCEPT
-A OUTPUT -o lo -j ACCEPT
COMMIT

These rules will allow loopback traffic, which is required for our hidden service. We're also allowing outbound traffic, then the established and related traffic to our outbound connections. This allows us to get out to the internet, without people reaching in.

Now for the moment of truth, let's restart our services one by one so the changes will take effect:

sudo systemctl restart tor

Now, let's make a note of our two onion addresses:

sudo cat /var/lib/tor/website/hostname

sudo cat /var/lib/tor/management/hostname

Make a note of those two addresses, you'll need them to access your hidden website and hidden ssh server.

Next, restart your ssh server, your webserver, and lastly, your firewall:

sudo systemctl restart ssh

sudo systemctl restart nginx

sudo systemctl restart netfilter-persistent

Your already-established ssh connection should stay up at this point. Now, let's test to make sure you can ssh to your management hidden service. To do this, though, you'll need Tor installed on your local system and a rule in your local ssh configuration.

On your local system, install socat, it's a multipurpose socket relay, it'll allow us to ssh through the tor network to hit your hidden service. We'll also install Tor.

sudo apt-get install socat tor

Now, let's make a rule to allow all *.onion addresses to use socat and tor to get where we want it to go:

nano ~/.ssh/config

Add this block to your ~/.ssh/config:

# Proxy .onion connections through tor
Host *.onion
    ProxyCommand socat - SOCKS4A:localhost:%h:%p,socksport=9050

Now, start Tor on your local system:

sudo systemctl start tor

By default, Tor will listen on port 9050 for any socks connections.

Now, ssh to your management onion address, just like a standard ssh connection. It will be laggy, very laggy, but you'll be connected to your server on your tor hidden service. You can now manage your server over a tor connection.

Now, on your local system grab the Tor Browser Bundle and install it. In the Tor Browser, open up your website onion address, you should see a default Debian nginx page.

You're now up and running! Restart your server to get rid of any hanging connections, the services should start automatically.

Here's your official disclaimer: Tor isn't perfect, it's good, it's great, the Tor Project does amazing work and it's the best answer to being anonymous on the internet that we have today, but Tor isn't a panacea, neither is this guide. I've tried to steer you away from any major pitfalls, but I'm not perfect. If you're building Silk Road number 72, you probably need more than this guide to protect yourself.

If you find any issues with this guide, leave a comment, a pull request, or hit me up on Twitter. Together we can make this better.

Here's some more great info from the Tor Project on hidden services:

GoLang: Slice contains and dedup

  2016-05-11 18:19:17 EDT

The GoLang Gopher was created by Renee French and is licensed under the Creative Commons 3.0 Attributions license. Check out this blog post for details.

Here's a simple GoLang demonstration showing how to see whether or not a slice contains a value, then how to deduplicate the items in that slice. You can easily run this on the Go Playground, check it out!

package main

import "fmt"

func contains(intSlice []int, searchInt int) bool {
    for _, value := range intSlice {
        if value == searchInt {
            return true
        }
    }
    return false
}

func dedup(intSlice []int) []int {
    var returnSlice []int
    for _, value := range intSlice {
        if !contains(returnSlice, value) {
            returnSlice = append(returnSlice, value)
        }
    }
    return returnSlice
}

func main() {
    numbers := []int{1, 2, 2, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6}
    fmt.Println("Numbers:", numbers)
    fmt.Println("Numbers contain 5:", contains(numbers, 5))
    fmt.Println("Numbers contain 8:", contains(numbers, 8))
    fmt.Println("Deduped numbers:", dedup(numbers))
}

watchtwitch 2.2.1 - Browse Twitch.TV

  2016-05-07 15:49:32 EDT

video_game video_game video_game video_game video_game video_game video_game video_game video_game video_game

Here's a new update to watchtwitch. You can now use the program to really browse Twitch.tv. I've built in functionality to search for games by name and see the top streaming games on Twitch.

Examples

watchtwitch -search zelda -quality medium

1) The Legend of Zelda: Ocarina of Time
2) The Legend of Zelda: A Link to the Past
3) The Legend of Zelda: Majora's Mask
4) The Legend of Zelda: Skyward Sword
5) The Legend of Zelda: The Wind Waker
6) The Legend of Zelda: Ocarina of Time 3D
7) The Legend of Zelda
8) Zelda II: The Adventure of Link
9) The Legend of Zelda: Twilight Princess
10) The Legend Of Zelda: Four Swords Anniversary Edition
11) The Legend of Zelda: Oracle of Seasons
12) The Legend of Zelda: Oracle of Ages
13) Hyrule Warriors
14) The Legend of Zelda: Phantom Hourglass
15) The Legend of Zelda: A Link to the Past & Four Swords
16) The Legend of Zelda: Ocarina of Time / Master Quest
17) Zelda: The Wand of Gamelon
18) The Legend of Zelda: Link's Awakening DX
19) The Legend of Zelda: Link's Awakening
20) The Legend of Zelda: Four Swords Adventures
Make a selection: 2
1) SpeedGaming (383)
2) ChristosOwen (19)
3) wqqqqwrt (8)
4) caznode (4)
5) Cransoon (3)
6) DevolitionDerby (3)
7) iReference (0)
8) Napptasm (0)
Make a selection: 1

watchtwitch -topgames -q medium

1) League of Legends (131815)
2) Counter-Strike: Global Offensive (103283)
3) Hearthstone: Heroes of Warcraft (99037)
4) Dota 2 (49565)
5) Call of Duty: Black Ops III (43002)
6) Overwatch (34032)
7) DayZ (33725)
8) FIFA 16 (29582)
9) StarCraft II (25943)
10) Clash Royale (25615)
Make a selection: 3
1) dreamhackhs (43118)
2) mira_hs (1495)
3) Shadybunny (820)
4) DeerNadia (788)
5) Hotform (602)
6) Alliestrasza (565)
7) RageGamingVideos (313)
8) TZAR1337 (242)
9) Lotsko (134)
10) Himiwako (93)
Make a selection: 1

Downloads

As always, you can grab the code here on GitLab. If you'd like a build for a different platform, let me know in the comments.

video_game video_game video_game video_game video_game video_game video_game video_game video_game video_game

watchtwitch 2.0.0 - GoLang Edition

  2016-05-03 10:35:41 EDT

video_game video_game video_game video_game video_game video_game video_game video_game video_game video_game

If you haven't seen my command line twitch.tv browser, check that out first. While the ruby script works perfectly, it was still pretty heavy, needing ruby and gems installed everywhere I wanted to have at-my-fingertips Dota 2 streams. I've been getting into GoLang recently and this seemed like the perfect project to re-implement.

Notable changes include command flags for choosing the game you'd like to watch, the quality of the stream, and the language of the stream. Check out the readme for more information. All this goodness rolled up into a single binary.

Like with the ruby script, it does rely on livestreamer and vlc to do all of the heavy lifting. This program is merely a menu of the 10 most popular Twitch streams for a given game and language.

As usual, the code is MIT licensed and available on GitLab. If building source code isn't your thing, grab the downloads below.

Downloads

video_game video_game video_game video_game video_game video_game video_game video_game video_game video_game

Page: 1 of 22