HOW TO: Enable Socket SO_REUSEPORT for Shadowsocks-libev on Ubuntu 16.04 Server

Introduction and Summary:

     Shadowsocks is an encrypted proxy for circumventing firewalls and geo-location restrictions.  There is an option to enable SO_REUSEPORT based on the adjustments to the Linux kernel in version 3.9 in 2013.  This option seems to be most useful on a Shadowsocks server when handling many users ("millions of packets per second") in order to deal with congestion on a multi-core server.  Some applications see a speed increase of about 20% of traffic throughput on a congested server.  A Cloudflare engineer summarized it with these statements:

When this flag is set on a socket descriptor, Linux will allow many processes to bind to the same port. In fact, any number of processes will be allowed to bind and the load will be spread across them. With SO_REUSEPORT each of the processes will have a separate socket descriptor. Therefore each will own a dedicated UDP receive buffer."

In the author's summary (Marek Majkowski), he concluding by recommending the SO_REUSEPORT option be enabled by saying:

Ensure traffic is distributed evenly across many RX queues and SO_REUSEPORT processes. In practice, the load usually is well distributed as long as there are a large number of connections (or flows).

It seems that for SO_REUSEPORT to have a significant effect on network performance, the following would need to be true of a Shadowsocks server:

1. There are many users connecting to the server
2. The server is equipped with multiple CPU cores for handling traffic
3. The OS is Linux running kernel 3.9 or higher (a technical requirement)

Matt Haigh reinforces my understanding when he said the following in a Stack Overflow thread:

[SO_REUSEPORT] was added during the 3.9 cycle in a series of patches by Tom Herbert, as you can see here, in order to better support multithreaded web servers.
...
And yes, you can use SO_REUSEPORT to bind a socket to the same address and port as another connection, as long as the initial connection also uses SO_REUSEPORT (and any other connections sharing the source address and port). This is done to prevent port hijacking by rogue applications.

Note: That being said, there seems to be no disadvantage to enabling it if you are connecting from a Linux client, like OpenWRT.  I've enabled it with no noticeable adverse effects.  Nevertheless it's hard for me to comment yet if it has a significant positive effect since very few people are using my server.

Prerequisites:

• Linux Ubuntu 16.04 LTS Server
• Linux Kernel must be > 3.9.0
• Console Access
• Root level permission

Instructions:

1. Simply add the following line to your config.json file (formatted to be in the middle of the config, not the end

  "reuse_port":true,

or add this option to the command line string if you are not using the config file:

  --reuse-port

As far as I can tell, there's nothing to enable in Linux.  If the process declare to the SO_REUSEPORT to the kernel, it will automatically load balance connections.

Blog Formatting Guide:

• Commands written in Consolas in BLUE are LOCAL Client Shell Commands (I'm running an Ubuntu local client)
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in BOLD are CUSTOM user variables that your should change for your configuration.

Helpful References, for Shadowsocks:

Shadowsocks-libev on Github

TCP Page on Ubuntu.com
SO_REUSEPORT Tech Briefing at lwn.net
Gitlabs report from a School in Illinois
Cloudflare blog post on SO_REUSEPORT
CTOLIB.COM's schematic drawing
Ubuntu MAN page on getsockopt
Ubuntu MAN page on Sockets
Github conversation on failback mentions REUSEPORT as "distributes datagrams evenly across all of the receiving threads."

HOW TO: Install Shadowsock-libev using APT on Ubuntu 16.04 LTS

Introduction and Summary:

Shadowsocks is an encrypted proxy for circumventing firewalls and geo-location restrictions.  Some Linux server repositories do not have packages or if they do, the packages might be old since the server and client are under constant development.  I've written another post on how to build the packages from source, but this should only be done if you are willing to troubleshoot issues and want to maintain updating the builds by yourself.  A much easier way to install shadowsocks-libev on an Ubuntu server is to add the PPA to your server and install with APT.

Note: These packages are considered "unsupported" and "untrusted" by Ubuntu.

Prerequisites:

• Linux Ubuntu 16.04.3 LTS Server
• Console Access
• Root level permission

Instructions:

1. When logged into the server, run the following commands:

sudo add-apt-repository ppa:max-c-lv/shadowsocks-libev

2. Update your local APT library with the new packages

sudo apt-get update

3. Install the packages:

sudo apt install libsodium mbedtls shadowsocks-libev

Note: You can run all three of these commands in one line by entering the following string:

sudo add-apt-repository ppa:max-c-lv/shadowsocks-libev && sudo apt-get update && sudo apt install libsodium mbedtls shadowsocks-libev

Blog Formatting Guide:

• Commands written in Consolas in BLUE are LOCAL Client Shell Commands (I'm running an Ubuntu local client)
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in BOLD are CUSTOM user variables that your should change for your configuration.

Helpful References, for Shadowsocks:

Shadowsocks-libev on Github

Simple-obfs on Github

Ubuntu PPA Help Page for Shadowsocks-libev

HOW TO: Enable rng-tools on Ubuntu VPS Server 16.04 for more "random" data

Introduction and Summary:

Headless VPS servers can suffer from a lack of random data inputs since there is little if any hardware to pull truly random data from (mouse, keyboard, webcams, entropy keys, etc.).  A hack for semi-random data to be generated in software is possible with rng-tools.  These instructions will help you install rng-tools on your VPS and hack the config to pull from the urandom file.

Note: This random data is not truly random and it is generally not considered secure for highly sensitive applications.  I'll be using it for generating obfuscation data, so true randomness is not important to me. Also, my paranoia level is in the mid to low range - if you're requiring higher security solutions, please skip this article.

Prerequisites:

• Linux Ubuntu 16.04 LTS Server
• Console Access
• Root level permissions

Instructions:

1. Connect to the server console (local or SSH)
2. Install the rng-tools package:

  sudo apt update && apt install rng-tools

Note: If you start the service, you will see that it fails:

systemctl status rng-tools
● rng-tools.service
   Loaded: loaded (/etc/init.d/rng-tools; bad; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2017-10-26 04:11:02 EDT; 36min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 299 ExecStart=/etc/init.d/rng-tools start (code=exited, status=1/FAILURE)
Oct 26 04:11:02 ubuntuvps systemd[1]: Starting rng-tools.service...
Oct 26 04:11:02 ubuntuvps rng-tools[299]: Starting Hardware RNG entropy gatherer daemon: (Hardware RNG device inode not found)
Oct 26 04:11:02 ubuntuvps rng-tools[299]: /etc/init.d/rng-tools: Cannot find a hardware RNG device to use.
Oct 26 04:11:02 ubuntuvps systemd[1]: rng-tools.service: Control process exited, code=exited status=1
Oct 26 04:11:02 ubuntuvps systemd[1]: Failed to start rng-tools.service.
Oct 26 04:11:02 ubuntuvps systemd[1]: rng-tools.service: Unit entered failed state.
Oct 26 04:11:02 ubuntuvps systemd[1]: rng-tools.service: Failed with result 'exit-code'.

3. Add the /dev/urandom path to the rng config file:

  sudo echo "HRNGDEVICE=/dev/urandom" >> /etc/default/rng-tools

4. Start the service again

  systemctl start rng-tools

5. Check the status of rng-tools

  systemctl status rng-tools

● rng-tools.service
   Loaded: loaded (/etc/init.d/rng-tools; bad; vendor preset: enabled)
   Active: active (exited) since Thu 2017-10-26 04:47:45 EDT; 2s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 562 ExecStart=/etc/init.d/rng-tools start (code=exited, status=0/SUCCESS)
Oct 26 04:47:45 ubuntuvps systemd[1]: Starting rng-tools.service...
Oct 26 04:47:45 ubuntuvps rng-tools[562]: Starting Hardware RNG entropy gatherer daemon: rngd.
Oct 26 04:47:45 ubuntuvps systemd[1]: Started rng-tools.service.
Oct 26 04:47:45 ubuntuvps rngd[564]: RNDADDENTROPY failed: Operation not permitted

Blog Formatting Guide:

• Commands written in Consolas in BLUE are LOCAL Client Shell Commands (I'm running an Ubuntu local client)
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in Courier and BOLD are CUSTOM user variables that your should change for your configuration.
• All text written in Courier and PURPLE are file contents as shown in a text editor.

Helpful References:

Jon Stephen's post. Thank's Jon!

HOW TO: Use BBR TCP Congestion for Improved Shadowsocks Performance on Ubuntu Server 16.04 LTS

Introduction and Summary:

Shadowsocks uses the TCP protocol for transmitting information over the Internet.  A new "congestion algorithm" has been released by Google that shows good potential for increasing maximum speeds for Shadowsocks, TCP BBR. Newer Linux Kernels support BBR natively, so the basic instructions below update your server to a recent kernel and then switch the TCP congestion controls to BBR.  This only needs to be done on the server side; the client will receive TCP traffic control information from the server automatically.

Note: This only works on servers that you have full control of, like a dedicated server or a KVM VPS. It will not work on OpenVZ VPS's or similar types of virtualization environments since you cannot modify most networking settings or the core OS.

Prerequisites:

• Linux Ubuntu 16.04 LTS Server
• Console Access
• Root level permissions
• The following Ubuntu packages (some might be obsolete or unnecessary, send me a message if you discover an improvement): wget

Instructions:

1. Update the Ubuntu Linux Kernel to the most recent generic release (Note: The minimum kernel is v4.9 - feel free to use newer or older kernels if you want. You can browse the current kernel builds here: http://kernel.ubuntu.com/~kernel-ppa/mainline/)

sudo apt install linux-image-generic-hwe-16.04-edge

2. The server will reboot. Sign-in to the server again (console, ssh, etc.)
3. Enable BBR congestion algorithm

echo "net.core.default_qdisc=fq" | sudo tee -a /etc/sysctl.conf
echo "net.ipv4.tcp_congestion_control=bbr" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

4. Now check the results; you should see bbr as the current congestion control setting.

sysctl net.ipv4.tcp_available_congestion_control
sysctl net.ipv4.tcp_congestion_control
lsmod | grep bbr

5. Reboot the server to finalize the new settings

sudo reboot

Blog Formatting Guide:

• Commands written in Consolas in BLUE are LOCAL Client Shell Commands (I'm running an Ubuntu local client)
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in Courier and BOLD are CUSTOM user variables that your should change for your configuration.
• All text written in Courier and PURPLE are file contents as shown in a text editor.

Helpful References, for Shadowsocks:

Shadowsocks-libev on Github
Ubuntu Server documentation on Mainline Kernel implementations
Google's BBR code on Github
Google blog post on BBR
ACM.org blog post on BBR

HOW TO: Create a System Service for Shadowsocks-libev on Ubuntu 16.04 LTS

Introduction and Summary:

After making and installing Shadowsocks from source code, you may want to create a service that starts and stops with the server and behaves like a Daemon.  The following instructions are one way that this can be accomplished.  (Note: The installation via APT will create a service during the installation process and there is no need to create this service manually)

UPDATED 2018 March to use echo commands to manually enter the service text and also the service commands were edited to more closely follow the defaults for debian on shadowsocks on github.

Prerequisites:

• Linux Ubuntu 16.04.3 LTS Server
• Console Access
• Root level permissions
• The following Ubuntu packages (some might be obsolete or unnecessary, send me a message if you discover an improvement): vim

Instructions:

1. Create a file called ss-server.service under /usr/lib/systemd/system/ and edit it with vim.

sudo mkdir /usr/lib/systemd/system && sudo vim /usr/lib/systemd/system/shadowsocks.service

2. Enter the following text into the new file using a series of echo commands.

echo “[Unit]” > /usr/lib/systemd/system/shadowsocks.service
echo “Description=Shadowsocks Server Daemon /usr/local/bin/ss-server” >> /usr/lib/systemd/system/shadowsocks.service
echo “After=network.target network-online.target” >> /usr/lib/systemd/system/shadowsocks.service
echo “” >> /usr/lib/systemd/system/shadowsocks.service
echo “[Service]” >> /usr/lib/systemd/system/shadowsocks.service
echo “Type=simple” >> /usr/lib/systemd/system/shadowsocks.service
echo “ExecStart=/usr/local/bin/ss-server” >> /usr/lib/systemd/system/shadowsocks.service
echo “Group=nogroup” >> /usr/lib/systemd/system/shadowsocks.service
echo “LimitNOFILE=32768” >> /usr/lib/systemd/system/shadowsocks.service
echo “” >> /usr/lib/systemd/system/shadowsocks.service
echo “[Install]” >> /usr/lib/systemd/system/shadowsocks.service
echo “WantedBy=multi-user.target” >> /usr/lib/systemd/system/shadowsocks.service
echo “” >> /usr/lib/systemd/system/shadowsocks.service


3. Enable the service and check its status

sudo systemctl enable shadowsocks && sudo systemctl start shadowsocks && sudo systemctl status shadowsocks

Note: The command to stop the service is: sudo systemctl stop shadowsocks
This can be useful when troubleshooting. You can stop the service and run "ss-server" manually to watch for error messages.
Note: The command to start the service is: sudo systemctl start shadowsocks

Blog Formatting Guide:

• Commands written in Consolas in BLUE are LOCAL Client Shell Commands (I'm running an Ubuntu local client)
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in Courier and BOLD are CUSTOM user variables that your should change for your configuration.
• All text written in Courier and PURPLE are file contents as shown in a text editor.

Helpful References, for Shadowsocks:

Shadowsocks-libev on Github
Simple-obfs on Github
Digital Ocean Forum, starting services for Ubuntu Server

HOW TO: Build Libsodium from source on Ubuntu 16.04 LTS

Introduction and Summary:

Shadowsocks supports various ciphers that require the LibSodium plug-in, like CHACHA20-IETF-POLY1305.  Some Linux server repositories do not have the package or if they do, the packages might be old since the server and client are under constant development.  These commands will help to download and build the latest Libsodium plug-in from source.

WARNING: The build from source version of libsodium CANNOT exist on the same server with the packages installed from APT sources.  Shadowsocks will often fail to load the ciphers properly and you will get errors.  If you have installed both the APT and the source-code versions of libsodium, remove the APT version with a command like sudo apt remove libsodium

Prerequisites:

• Linux Ubuntu 16.04 LTS Server
• Console Access
• Root level permissions
• The following Ubuntu packages (some might be obsolete or unnecessary, send me a message if you discover an improvement): build-essential autoconf automake libtool git 

Instructions:

When logged into the server, run the following commands:

cd /tmp
git clone https://github.com/jedisct1/libsodium.git
cd libsodium
git submodule update --init --recursive && ./autogen.sh
./configure
make
sudo make install
cd ..
rm libsodium -r -f

NOTE: The entire set of commands can be chained together in one long command:
cd /tmp && git clone https://github.com/jedisct1/libsodium.git && cd libsodium && git submodule update --init --recursive && ./autogen.sh && ./configure && make && sudo make install && cd .. && rm libsodium -r -f

Blog Formatting Guide:

• Commands written in Consolas in BLUE are LOCAL Client Shell Commands (I'm running an Ubuntu local client)
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in BOLD are CUSTOM user variables that your should change for your configuration.

Helpful References, for Shadowsocks:

Shadowsocks-libev on Github
Simple-obfs on Github
Libsodium on Github

HOW TO: Build Simple-obfs from source on Ubuntu 16.04 LTS

Introduction and Summary:

Simple-obfs was written primarily for use with Shadowsocks to obfuscate encrypted traffic.  Some Linux server repositories do not have packages or if they do, the packages might be old since the server and client are under constant development.  These commands will help to download and build the latest Simple-obfs plug-in from source.

Prerequisites:

• Linux Ubuntu 16.04.3 LTS Server
• Console Access
• Root level permissions
• The following Ubuntu packages (some might be obsolete or unnecessary, send me a message if you discover an improvement):

build-essential autoconf libtool libssl-dev libpcre3-dev zlib1g-dev libev-dev libsodium-dev git automake libmbedtls-dev libc-ares-dev

Instructions:

When logged into the server, run the following commands:

cd /tmp
git clone https://github.com/shadowsocks/simple-obfs.git
cd simple-obfs
git submodule update --init --recursive
./autogen.sh
./configure
make
sudo make install
cd ..
rm simple-obfs -r -f

NOTE: The entire set of commands can be chained together in one long command:
cd /tmp && git clone https://github.com/shadowsocks/simple-obfs.git && cd simple-obfs && git submodule update --init --recursive && ./autogen.sh && ./configure && make && sudo make install && cd .. && rm simple-obfs -r -f

Blog Formatting Guide:

• Commands written in Consolas in BLUE are LOCAL Client Shell Commands (I'm running an Ubuntu local client)
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in BOLD are CUSTOM user variables that your should change for your configuration.

Helpful References, for Shadowsocks:

Shadowsocks-libev on Github

Simple-obfs on Github

HOW TO: Build Shadowsock-libev from source on Ubuntu 16.04 LTS | Latest Nightly Version

Introduction and Summary:

Shadowsocks is an encrypted proxy for circumventing firewalls and geo-location restrictions.  Some Linux server repositories do not have packages or if they do, the packages might be old since the server and client are under constant development.  These commands will help to download and build the latest Shadowsocks-libev server from source.

UPDATE: Recently the latest editions of shadowsocks (versions 3.2 and higher) have not worked properly on my servers (service does not start, udp does not redirect properly, etc.).  If you would like to build a specific version of shadowsocks-libev, please see this other post for those instructions [LINK]

Prerequisites:

• Linux Ubuntu 16.04 LTS Server
• Console Access
• Root level permissions
• The following Ubuntu packages (some might be obsolete or unnecessary, send me a message if you discover an improvement):

build-essential autoconf libtool libssl-dev libpcre3-dev zlib1g-dev libev-dev git automake init-system-helpers pkg-config asciidoc xmlto libmbedtls-dev libc-ares-dev

Instructions:

When logged into the server, run the following commands:

cd /tmp
git clone https://github.com/shadowsocks/shadowsocks-libev.git
cd shadowsocks-libev
git submodule update --init --recursive
./autogen.sh
./configure
make
sudo make install
cd ..
rm shadowsocks-libev -r -f

NOTE: The entire set of commands can be chained together in one long command:
cd /tmp && git clone https://github.com/shadowsocks/shadowsocks-libev.git && cd shadowsocks-libev && git submodule update --init --recursive && ./autogen.sh && ./configure && make && sudo make install && cd .. && rm shadowsocks-libev -r -f

Blog Formatting Guide:

• Commands written in Consolas in BLUE are LOCAL Client Shell Commands (I'm running an Ubuntu local client)
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in BOLD are CUSTOM user variables that your should change for your configuration.

Helpful References, for Shadowsocks:

Shadowsocks-libev on Github
Simple-obfs on Github

HOW TO: Setup Shadowsocks on LEDE Router with Snapshot ROM and Snapshot Packages (PLAINTEXT EDITION)

## LEDE SNAPSHOT (2017 SEPT OR HIGHER) ROUTER INSTALLATION GUIDE WITH SHADOWSOCKS
## HARDWARE: Xiaomi Mini Wifi Router
## ROM SNAPSHOT DOWNLOAD LINK: https://downloads.lede-project.org/snapshots/targets/ramips/mt7620/
## 

## LEDE SNAPSHOT BASE INSTALLATION ##

## 1. ROUTER CONFIGURATION - GENERIC PACKAGES FOR VPN
## Flash a clean ROM on the router, connect via SSH, then set a password for router...the following command is just to set the password for the router since flashing the ROM is covered in other articles and connecting via SSH should already be in your tool bag of knowledge.

passwd

## Install Packages for Shadowsocks, OpenVPN, and Wireguard.

opkg update && opkg install luci luci-ssl-openssl luci-app-shadowsocks-libev shadowsocks-libev-config shadowsocks-libev-ss-local shadowsocks-libev-ss-redir shadowsocks-libev-ss-rules shadowsocks-libev-ss-server shadowsocks-libev-ss-tunnel iptables-mod-conntrack-extra kmod-ipt-tproxy iptables-mod-tproxy dnscrypt-proxy luci-app-dnscrypt-proxy rng-tools ca-certificates wget luci-app-uhttpd luci-app-wifischedule

## OPTIONAL: Manually install Shadowsocks Simple-OBFS onto the router
## Note: Must be behind a VPN because China blocks sourceforge.net

LEDE=http://openwrt-dist.sourceforge.net/packages/LEDE/base/mipsel_24kc && SIMPLEOBFS=simple-obfs_0.0.3-2_mipsel_24kc.ipk && wget $LEDE/$SIMPLEOBFS && opkg install $SIMPLEOBFS && rm $SIMPLEOBFS

## Backup Default Configs 

mkdir /usr/share/default-configs && cp /etc/config/dnscrypt-proxy /usr/share/default-configs/default.dncrypt-proxy && cp /etc/config/shadowsocks-libev /usr/share/default-configs/default.shadowsocks-libev && cp /etc/config/dhcp /usr/share/default-configs/default.dhcp && cp /etc/config/network /usr/share/default-configs/default.network

## Set Router HOSTNAME, TIMEZONE, NTP, and CUSTOMIZE LED (Indicates Modification) 

uci set system.@system[0].hostname='LEDE' && uci set system.@system[0].timezone='HKT-8' && uci set system.@system[0].zonename='Asia/Hong Kong' && uci set system.ntp.enable_server='1' && uci delete system.ntp.server && uci add_list system.ntp.server='stdtime.gov.hk' && uci add_list system.ntp.server='time.nist.gov' && uci add_list system.ntp.server='us.pool.ntp.org' && uci add_list system.ntp.server='time.google.com'

uci set system.led_power=led && uci set system.led_power.name='power' && uci set system.led_power.sysfs='miwifi-mini:blue:status' && uci set system.led_power.default='1' && uci commit

service rngd enable

## Configure Wireless Network
## NOTE: Update SSID and Password to your preferred name

uci set wireless.radio0.hwmode='11a' && uci set wireless.radio0.channel='40' && uci set wireless.radio0.country='00' && uci set wireless.default_radio0.ssid='LEDE-AC' && uci set wireless.default_radio0.encryption='psk2' && uci set wireless.default_radio0.key='PASSWORD'

uci set wireless.radio1.hwmode='11g' && uci set wireless.radio1.channel='11' && uci set wireless.radio1.country='00' && uci set wireless.default_radio1.ssid='LEDE' && uci set wireless.default_radio1.encryption='psk2' && uci set wireless.default_radio1.key='*PASSWORD'

uci delete wireless.radio0.disabled && uci delete wireless.radio1.disabled && uci commit && service network restart

## Configure LAN IP, Reboot (192.168.99.1 used here)

uci set network.lan.ipaddr='192.168.99.1' && uci commit && reboot && exit

## ENABLE HTTPS FOR LUCI WEB CONSOLE 
## Create the myconfig.conf 

cat > /etc/ssl/myconfig.conf

## Paste the following contents: 

[req]
distinguished_name  = req_distinguished_name
x509_extensions     = v3_req
prompt              = no
[req_distinguished_name]
C           = US
ST          = CA
L           = LEDE
O           = Home
OU          = Router
CN          = 192.168.99.1
[v3_req] 
keyUsage           = keyEncipherment, dataEncipherment
extendedKeyUsage   = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = 192.168.99.1
IP.1 = 192.168.99.1

## Ctrl+d to exit and commit the text.

## Generate the keys 

openssl req -x509 -nodes -days 730 -newkey rsa:2048 -keyout /etc/ssl/mycert.key -out /etc/ssl/mycert.crt -config /etc/ssl/myconfig.conf && uci set uhttpd.main.redirect_https='1' && uci set uhttpd.main.cert='/etc/ssl/mycert.crt' && uci set uhttpd.main.key='/etc/ssl/mycert.key' && uci commit && service uhttpd restart

## Setup DNSCRYPT-PROXY for DNS Resolution
## Updates Resolver List (/usr/share/dnscrypt-proxy/dnscrypt-resolvers.csv) 

mv /usr/share/dnscrypt-proxy/dnscrypt-resolvers.csv /usr/share/dnscrypt-proxy/default.dnscrypt-resolvers ; wget -O- 'https://raw.githubusercontent.com/jedisct1/dnscrypt-proxy/master/dnscrypt-resolvers.csv' > /usr/share/dnscrypt-proxy/dnscrypt-resolvers.csv && cp /etc/config/dnscrypt-proxy /etc/config/default.dncrypt-proxy

## Add Server #1 

uci set dnscrypt-proxy.ns1=dnscrypt-proxy && uci set dnscrypt-proxy.ns1.address='127.0.0.1' && uci set dnscrypt-proxy.ns1.port='5353' && uci set dnscrypt-proxy.ns1.resolver='cisco' && uci set dnscrypt-proxy.ns1.resolvers_list='/usr/share/dnscrypt-proxy/dnscrypt-resolvers.csv' && uci set dnscrypt-proxy.ns1.syslog='1' && uci commit dnscrypt-proxy

## Add Server #2 

uci set dnscrypt-proxy.ns2=dnscrypt-proxy && uci set dnscrypt-proxy.ns2.address='127.0.0.1' && uci set dnscrypt-proxy.ns2.port='5454' && uci set dnscrypt-proxy.ns2.resolver='fvz-anyone' && uci set dnscrypt-proxy.ns2.resolvers_list='/usr/share/dnscrypt-proxy/dnscrypt-resolvers.csv' && uci set dnscrypt-proxy.ns2.syslog='1' && uci commit dnscrypt-proxy

## Enable and Start the Service

service dnscrypt-proxy enable && service dnscrypt-proxy start

## Configure DNSMASQ 
## Server list 

uci add_list dhcp.@dnsmasq[0].server='127.0.0.1#5353' && uci add_list dhcp.@dnsmasq[0].server='127.0.0.1#5454' && uci add_list dhcp.@dnsmasq[0].server='127.0.0.1#1100' && uci set dhcp.@dnsmasq[0].noresolv='1' && uci set dhcp.@dnsmasq[0].nohosts='1' && uci commit dhcp && reboot && exit

## SHADOWSOCKS-LIBEV CONFIGURATION ##

## REMOVE UN-NEEDED CONFIGS

uci delete shadowsocks-libev.@ss_local[0]
uci delete shadowsocks-libev.@ss_tunnel[0]
uci delete shadowsocks-libev.hj
uci delete shadowsocks-libev.ss_rules
uci delete shadowsocks-libev.@ss_server[0]
uci commit shadowsocks-libev

## UPDATE CONFIG SS-HI (CUSTOM, CHECK THE FAST OPEN AND MODE SETTINGS ESPECIALLY)

uci set shadowsocks-libev.hi.server='sss0'
uci set shadowsocks-libev.hi.local_address='0.0.0.0'
uci set shadowsocks-libev.hi.local_port='1100'
uci set shadowsocks-libev.hi.mode='tcp_and_udp'
uci set shadowsocks-libev.hi.timeout='480'
uci set shadowsocks-libev.hi.fast_open='1'
uci set shadowsocks-libev.hi.verbose='1'
uci set shadowsocks-libev.hi.reuse_port='1'
uci delete shadowsocks-libev.hi.disabled
uci commit shadowsocks-libev

## CONFIG SS REMOTE SERVER (CUSTOM, YOUR REMOTE SERVER)

uci set shadowsocks-libev.sss0.server='###.###.###.###'
uci set shadowsocks-libev.sss0.method='rc4-md5'
uci set shadowsocks-libev.sss0.password='**************************'
uci set shadowsocks-libev.sss0.server_port='443'
uci delete shadowsocks-libev.sss0.disabled
uci commit shadowsocks-libev

## Optional - Settings for OBFS

uci set shadowsocks-libev.sss0.plugin='obfs-local'
uci set shadowsocks-libev.sss0.plugin_opts='obfs=http;obfs-host=cloudflare.net'
uci commit shadowsocks-libev

## CONFIG SS FORWARDING RULES (UNIVERSAL)

uci set shadowsocks-libev.ss_rules=ss_rules
uci set shadowsocks-libev.ss_rules.src_ips_forward='192.168.99.0/24'
uci set shadowsocks-libev.ss_rules.redir_tcp='hi'
uci set shadowsocks-libev.ss_rules.redir_udp='hi'
uci set shadowsocks-libev.ss_rules.local_default='forward'
uci set shadowsocks-libev.ss_rules.ifnames='br-lan'
uci set shadowsocks-libev.ss_rules.src_default='forward'
uci set shadowsocks-libev.ss_rules.dst_default='forward'
uci delete shadowsocks-libev.ss_rules.disabled
uci commit shadowsocks-libev
service shadowsocks-libev reload

## Download China bypass list (For Future Enhancement)

wget -O- 'https://ftp.apnic.net/apnic/stats/apnic/delegated-apnic-latest' | awk -F\| '/CN\|ipv4/ { printf("%s/%d\n", $4, 32-log($5)/log(2)) }' > /etc/chinadns_chnroute.txt

## Add some Scheduled Tasks to CRONTABS
## Add these in the LUCI webapp…until I figure out the command line

0 5 * * * sleep 70 && touch /etc/banner && reboot
0 */2 * * * /etc/init.d/shadowsocks-libev reload
5 4 * * 0 wget -O- 'https://ftp.apnic.net/apnic/stats/apnic/delegated-apnic-latest' | awk -F\| '/CN\|ipv4/ { printf("%s/%d\n", $4, 32-log($5)/log(2)) }' > /etc/chinadns_chnroute.txt
10 5 1 * * wget -O- 'https://raw.githubusercontent.com/jedisct1/dnscrypt-proxy/master/dnscrypt-resolvers.csv' > /usr/share/dnscrypt-proxy/dnscrypt-resolvers.csv

Step-by-Step: OpenWRT/LEDE Base Installation for Xiaomi MiWiFi 3GR

The Xiaomi 3G router has a low price and spacious ROM (128MB) and RAM (256MB) for easily supporting most of the packages and loads a person would put on a OpenWRT/LEDE installation.  Thanks to some recent hard work by the LEDE community it’s fairly straight-forward to replace the Stock OEM firmware with LEDE.

NOTE: Development for the Xiaomi 3G is currently considered in BETA and should only be done by people who are content to support themselves and deal with bugs.  Having said that, I've found the most recent ROMs (March 2018) to be good enough for day-to-day use.

PREREQUISITES

A. A computer with a web browser for downloading the files, registering a Xiaomi account, and capable of running SSH.
B. The OpenWRT Installation Files from | downloads.openwrt.org

• mir3g-squashfs-kernel1.bin
• mir3g-squashfs-rootfs0.bin

C. A Xiaomi Account for registering the router | https://account.xiaomi.com/pass/register
D.  The Developer edition of the wifi ROM from Xiaomi | http://www.miwifi.com/miwifi_download.html

• Click the ROM graphic, scroll down to find “ROM for R3G 开发版”

E. A mobile device with Xiaomi’s router app installed | Google Play Link | Xiaomi APK
F. The customized SSH BIN file from Xiaomi.  This file can only be downloaded after registering the router with the Router app on your phone or tablet and it’s unique to each router since the ROOT password is apparently baked into it | Xiaomi SSH Site
G. A physical Ethernet connection to the router’s LAN ports (white colored).  Note: You can do most of the process on Wifi, but not all of it, so I recommend starting with Ethernet to simplify the process.
H. A USB drive formatted to FAT32 for installing the SSH BIN and LEDE BIN files to the router.

INSTRUCTIONS

1. Unbox the router (if it’s new), connect the WAN uplink (blue port) to the Internet, power it on, connect your computer to one of it’s ethernet ports with a computer (white port), open a browser and go to http://192.168.31.1, follow the onscreen router setup wizard (all in Chinese, but just use the same password for Admin and Wifi and you’ll be fine).
2. Download the "Developer" ROM from Xiaomi, if you haven’t already (see pre-reqs).
3. Manually flash the Developer ROM through the web app console.  Select the 3rd tab option, select the last menu item, and click the first command button to upload the file, click the right-hand button to confirm the installation of the new ROM. (It’s all in Chinese, so I’m trying to be helpful here, not condecending :-)
4. Install the Mi Wifi app on a mobile device
5. Connect your mobile device to the Wifi router's network
6. Sign into the app with your "Mi" account (or create one and then sign in, no personal information is needed other than an email account).
7. Sign into the Wifi Router through the app using the admin password you set during setup.  The router should now be registered to your Xiaomi account, our “Bound” as Google Translate says.
8. Download the the SSH BIN patch file at https://d.miwifi.com/rom/ssh. It's supposed to be custom for your router, so I'm imagining it's because the BIN files needs your router's root password baked in.
<insert screenshot>
9. While you're at this site, also copy down the "root" password from the page; you’ll need it for SSH later.
<insert screenshot>
10. Copy the miwifi_ssh.bin file from your computer to a USB drive formatted with FAT32.
11. Insert this USB drive into the USB port on the back of the Xiaomi router
12. Unplug the router's power cable.
13. Use a tool (like a straightened paperclip) to press down the reset button on the back of the router and hold it in.  While keeping the button pressed down, reconnect the power cable...not a task for the manually challenged. The yellow status light will come on the front of the router. After about 10 seconds, the yellow status light will begin to blink indicating that the router is now installing the SSH BIN file.  Now that the yellow light is flashing, you can finally release the reset button.

14. After the SSH file is installed, the router will reboot and the yellow light will become steady again during the boot process.  After the router is done booting, the status LED will turn a steady "blue" color.
15. Sign into the router using a ssh command from your computer and enter the enter the root password from the Xiaomi website
e.g. ssh root@192.168.31.1
<insert screenshot>
You're in! Enjoy your new super powers wisely
16. Download the following LEDE BIN files from the SNAPSHOT tree | Download Page
lede-ramips-mt7621-mir3g-squashfs-kernel1.bin
lede-ramips-mt7621-mir3g-squashfs-rootfs0.bin
17. Copy these two files to your USB drive and insert the USB drive back into the router.
18.  Within the SSH console, use these commands to flash the files:

cd /extdisks/sda1 (can be different if you remove and reinsert the usb stick)
mtd write lede-ramips-mt7621-mir3g-squashfs-kernel1.bin kernel1
mtd write lede-ramips-mt7621-mir3g-squashfs-rootfs0.bin rootfs0
nvram set flag_last_success=1
nvram commit
reboot

     The router will reboot, and congratulations! You're now running LEDE SNAPSHOT. You'll need to SSH back in and install uHTTPd and Luci to get into the web app again (No web console out of the box with SNAPSHOT) and you'll also need to reconfigure Wifi since it's disabled by default in the Snapshot images.  Here's some tips to install those core apps if you're not familiar:

ssh root@192.168.1.1
opkg update
opkg install uhttpd luci luci-app-uhttpd

UPDATING THE ROUTER FIRMWARE

Flash the TAR file from the Snapshot Repository (instead of a BIN file)

• mir3g-squashfs-sysupgrade.tar

REFERENCES
Xiaomi SSH Website After Registering Your Device in the App

小米ID：0000000000
已绑定1台小米路由器
Xiaomi_AAAA(小米路由器3增强版)root密码 xxxxxxxx下载工具包
工具包使用方法：小米路由器需升级到开发版0.5.28及以上，小米路由器mini需升级到开发版0.3.84及以上，小米路由器3即将支持。注意：稳定版不支持。

请将下载的工具包bin文件复制到U盘（FAT/FAT32格式）的根目录下，保证文件名为miwifi_ssh.bin；
断开小米路由器的电源，将U盘插入USB接口；
按住reset按钮之后重新接入电源，指示灯变为黄色闪烁状态即可松开reset键；
等待3-5秒后安装完成之后，小米路由器会自动重启，之后您就可以尽情折腾啦 ：）

Google Translate of Website

Has been bundled with a millet router
Xiaomi_AAAA (millet router 3 enhanced version) root password xxxxxx download kit
Toolkit to use: millet router to be upgraded to the development version of 0.5.28 and above, millet router mini to be upgraded to the development version of 0.3.84 and above, millet router 3 is about to support. Note: Steady version does not support.

Please download the toolkit bin file to U disk (FAT / FAT32 format) root directory, to ensure that the file name miwifi_ssh.bin;
Disconnect the power supply millet router, the U disk into the USB interface;
Hold down the reset button and then re-access the power, the indicator light to yellow flashing state can release the reset button;
Wait 3-5 seconds after the installation is complete, millet router will automatically restart, then you can enjoy tossing :)

LEDE FORUM COMMENT WITH BEST INSTRUCTIONS
https://forum.lede-project.org/t/xiaomi-wifi-router-3g/5377/57

This is how I got LEDE installed. Some of the steps can be done differently. I used this for inspiration https://www.youtube.com/watch?v=CSHNyo5QxaQ40
Unbox router
Connect to the router using WiFi
Goto http://192.168.31.11
Go through the wizard to set passwords for the router + wifi
Reconnect to the router using WiFi
Goto http://192.168.31.11
Logon and find the page where you can upgrade the firmware look for a big yellow dot with an “i” inside. You will see the version number of the router and there is a button below where you can browse for a file. Flash miwifi_r3g_firmware_c2175_2.25.122.bin (developer firmware) and wait a few minutes.
Download https://play.google.com/store/apps/details?id=com.xiaomi.router13 to your phone/tablet (there is also an iOS app)
Open “Mi Wi-Fi” app (and sign-up) and sign-in to your account. Router will be detected and added to your account (assuming you are connected to the WiFi on the router and the routers WAN port is connected to Internet).
On a PC, visit http://d.miwifi.com/rom/ssh18 and sign-in to you account. You will get to a page that should display your router, the root password and a download button. Hit the button to get miwifi_ssh.bin
Format USB drive using FAT32 and copy miwifi_ssh.bin, lede-ramips-mt7621-mir3g-squashfs-kernel1.bin and lede-ramips-mt7621-mir3g-squashfs-rootfs0.bin to the USB drive
Cut the power the router, put the USB drive in the router, press and hold “reset” button (with a paper-clip), power on the router (while holding reset). When the router starts flashing yellow release the reset button. Wait until router has rebooted and you should (finally…) have SSH access.
Login to the router using SSH using the “root” as username and the (root) “password” from http://d.miwifi.com/rom/ssh18
In SSH console
cd /extdisks/sda1 (can be different if you remove and reinsert the usb stick)
mtd write lede-ramips-mt7621-mir3g-squashfs-kernel1.bin kernel1
mtd write lede-ramips-mt7621-mir3g-squashfs-rootfs0.bin rootfs0
nvram set flag_last_success=1
nvram commit
reboot
LEDE should be installed and available at 192.168.1.1 (with WiFi disabled I assume)
Upgrading to a newer snapshot can be done using the regular methods (from the command-line using sysupgrade or through LuCI) using lede-ramips-mt7621-mir3g-squashfs-sysupgrade.tar
Thanks @dissent1 for adding the router to LEDE. Both WiFi radios seems to work (2.4Ghz on 20Mhz, 5Ghz on 80MHz) and I have also tested USB3 port with success. I have not tested 40MHz for 2.4Ghz as it does not make sense for me (very crowded WiFi). 5Ghz radio does not seem to like DFS channels (I get “DFS start_dfs_cac() failed, -1” in the log). The channels seems to work nicely when disabling DFS (by removing the requirement from regdb.txt for the selected country before building firmware) so this is likely “just” a DFS issue.
I can’t say anything about stability yet. I’ll probably try to replace my TP-Link Archer C5 with the Xiaomi router within the next week and then I will have to see how well it performs and how stable it is.

UPDATED on March 3, 2018 to include links to the OpenWRT repositories since the two projects have merged.

Step-by-Step: Base Install - VPN Client Router using LEDE (or OpenWRT)

PUBLISHED 2017 August 14
INTRODUCTION
This step-by-step is designed to install LEDE with the most common packages and protocols needed for it to act as a VPN Client (it connects to a VPN endpoint, it is not a VPN server accepting incoming connections).  The current VPN clients that are supported in this build are:

• OpenVPN (SSL) [LEDE]
• Shadowsocks-libev [github]

I only use the packages in the LEDE repository since they support UCI commands.  This unfortunately means that they are not the latest binaries available, but for support I personally think it's better to use these editions.

Note: LEDE also has packages for the "OpenConnect", "softethervpn", "openvpn-easy-rsa", and more - however I do not have providers to test these protocols against. If I ever do, I'll update this document.

To run these commands you'll need an SSH client on your computer.  PuTTY works fine on Windows, but I'm using Linux and all of my commands will assume that you are too.

I am using the Xiaomi Mini Wifi router since I live in China and can get them easily; they've also got lots of RAM and Storage. I highly recommend them if you can find one.  The downside is that there's a bug in the wifi driver support and the Wifi signal is not very strong; but it's still usable.  For instructions on the initial setup of the Xiaomi Mini, please see my other article on the subject HERE.

NOTES ON SYNTAX (My standards)

For all commands in the Linux shell, I color code them BLUE.

For all commands in the SSH shell inside the router, I color code them RED.

INSTALL LEDE 17.01.2

If you are running an older version of LEDE or you are running OpenWRT, please first upgrade to LEDE (it is a fork of OpenWRT that has essentially replaced it - it seems like they will re-merge in the future though). [LEDE Instructions]

Note: Even if you are already running on LEDE 17.01.2, I recommend a clean wipe of the device from within the LUCI webapp or with the following command (run via SSH): firstboot && reboot

ssh root@192.168.1.1

SNAPSHOT1="https://downloads.lede-project.org/snapshots/targets/ramips/mt7620/openwrt-ramips-mt7620-miwifi-mini-squashfs-sysupgrade.bin" ; SNAPSHOTSHA256SUMS="https://downloads.lede-project.org/snapshots/targets/ramips/mt7620/sha256sums" ; cd /tmp ; wget $SNAPSHOT1 ; wget $SNAPSHOTSHA256SUMS ; sha256sum -c sha256sums 2> /dev/null | grep OK

sysupgrade -v -n /tmp/*.bin

The router will install the new operating system and reboot.

INITIAL ROUTER CONFIGURATION

Sign into the router with SSH and set the administrator password

ssh root@192.168.1.1

passwd

Set the router name, timezone, and custom NTP servers for your region (or preference)

uci set system.@system[0].hostname='KABENEKO' && uci set system.@system[0].timezone='HKT-8' && uci set system.@system[0].zonename='Asia/Hong Kong' && uci set system.ntp.enable_server='1' && uci delete system.ntp.server && uci add_list system.ntp.server='stdtime.gov.hk' && uci add_list system.ntp.server='time.nist.gov' && uci add_list system.ntp.server='us.pool.ntp.org' && uci add_list system.ntp.server='time.google.com'

uci set system.led_power=led && uci set system.led_power.name='power' && uci set system.led_power.sysfs='miwifi-mini:blue:status' && uci set system.led_power.default='1' && uci commit

Configure the Wireless Network and Enable it

uci set wireless.radio0.hwmode='11a' && uci set wireless.radio0.channel='48' && uci set wireless.radio0.country='00' && uci set wireless.default_radio0.ssid='MYWIFINETWORK-AC' && uci set wireless.default_radio0.encryption='psk2' && uci set wireless.default_radio0.key='MYWIFINETWORKPASSWD'

uci set wireless.radio1.hwmode='11g' && uci set wireless.radio1.channel='8' && uci set wireless.radio1.country='00' && uci set wireless.default_radio1.ssid='MYWIFINETWORK' && uci set wireless.default_radio1.encryption='psk2' && uci set wireless.default_radio1.key='MYWIFINETWORKPASSWD'

uci delete wireless.radio0.disabled && uci delete wireless.radio1.disabled && uci commit && service network restart

Now... check your settings.

uci show wireless

SET LAN IP TO NEW RANGE (AVOID ISP CONFLICTS AT 192.168.1.1) and Reboot

uci set network.lan.ipaddr='192.168.55.1' && uci commit && reboot

The router's IP address will change, so you will exit from the SSH shell and you will need a new IP address.  Here's the command on my Linux box:

sudo dhclient -r && sudo dhclient

ssh root@192.168.55.1

Install all of the base packages for running ShadowSock or OpenVPN plus a few other goodies.

opkg update ; opkg install luci-theme-material luci-app-openvpn luci-app-shadowsocks-libev luci-app-uhttpd luci-app-adblock luci-app-wifischedule luci-ssl-openssl openvpn-openssl shadowsocks-libev ca-certificates wifischedule mtr rng-tools dnscrypt-proxy

Almost done!  Let's just turn off some things that shouldn't be running until we have time to configure them:

uci set shadowsocks-libev.@shadowsocks-libev[0].enable='0' && uci set adblock.global.adb_enabled='0' && uci commit && service shadowsocks-libev enable && service shadowsocks-libev stop

And we'll create the OpenVPN Interface for any future needs

uci set network.openvpn=interface ; uci set network.openvpn.proto='none' ; uci set network.openvpn.ifname='tun0' ; uci set firewall firewall.@zone[1].network='wan openvpn' && uci commit

Your router is now ready for a VPN client configuration to be added.

LINK Step-by-Step: Setup a ShadowSocks-libev Client on OpenWRT
LINK Step-by-Step: Setup a OpenVPN Client on OpenWRT

Step-by-Step: Shadowsocks Client on LEDE (OpenWRT) Router - Config with LUCI

UPDATE: I am writing a SSH version of this document since it's much easier to run commands that way.  Please see that document HERE.

PREREQUISITES
A. A computer with a web browser.
B. Router running LEDE or OpenWRT (Xiaomi Mini w/ LEDE 17.01.2, r3435)
     Note: For instructions on installing OpenWRT on a the Xiaomi, please see this article HERE.
C. Shadowsocks account from provider (I'm testing Catswall.net)
D. Router should be connected to an unrestricted Internet connection to reach LEDE package repositories, aka NOT filtered internet

SETUP ROUTER
Reset the router to defaults if it has been used for something else before.  Also consider updating it to the current "stable" release of LEDE if you're running an older OpenWRT or LEDE build.  (I've tested the snapshot versions of LEDE and the necessary packages are not available yet; you'll have to know how to compile your own software if you don't use the stable release...if you know how to do that, you're probably don't need this post.)

• A. To upgrade your router to the LEDE 17.01.2 release: Sign into the router web console, navigate to System | Backup/Flash Firmware | Uncheck the "Keep settings" option, click Choose File, find the .bin file on your computer that you've already downloaded (see point C for a download link), click FLASH IMAGE..., and then when the verification window appears, click PROCEED.
• B. To reset your router to all defaults: Sign into the router web console, navigate to System | Backup/Flash Firmware | click PERFORM RESET, and click OK at the pop-up.

NOTES:

• For more instructions on setting up a new Xiaomi Mini router with OpenWRT or LEDE, please see my article on that subject HERE.
• For more instructions on setting all base software on a LEDE/OpenWRT router, please see my article on the Base Installation for a VPN Client Router HERE.

INSTALL SOFTWARE AND ADD SERVER
After installing all necessary software packages as outlined in the Base Install Guide for Routers, we need to setup the Shadowsocks client options.

1. Under the Services menu option, select ShadowSocks-libev.

2. Enter the correct Shadowsocks server information, port, password, Encrypt Method, etc. from your provider, then save and restart the service by clicking SAVE & APPLY.

3. To force DNS resolution through the Shadowsocks tunnel (SS-TUNNEL) by selecting the "Enable" option under UDP Forward.  The UDP Local Port should be 5300, and the Forward Tunnel can be any valid DNS server, but we'll use Google's in this example and enter 8.8.8.8:53.
4. Add this UDP port address to the Network | DHCP and DNS | General Settings option for "DNS Forwardings" in this format:
127.0.0.1#5300

5. Select the Non-wildcard option and click SAVE & APPLY.
6. Select the Resolve and Hosts Files menu option.  Select the Ignore resolve file and the Ignore /etc/hosts options.  Click SAVE & APPLY.

If the VPN server information is correct you should now be connected to your server and DNS lookups should be going through the tunnel as well.  For best results... reboot the router.  I just unplug the little monster and plug it back in to make sure it's a nice "hard" reboot.

To check if you are browsing the Internet thought your SS connection go to a website like ipinfo.io or ipleak.net.  The IP adress shown on these websites should be the server address you entered in the ShadowSocks menu.

OPTION: ADD A SECONDARY DNS SERVICE FOR BACKUP; DNSCRYPT-PROXY
I'm writing another guide to setup a backup DNS server in-case the Shadowsocks lookups fail or are delayed. The link will be here later!