HOW TO: Install Shadowsocks on Windows 10 with a batch script

Introduction and Summary:

Shadowsocks is an encrypted proxy for circumventing firewalls and geo-location restrictions.  It is similar to a VPN but simpler and has less features.  The client for Android is very simple to use and is the "gold" standard (in my opinion), but the Windows client also works well if you are able to configure it correctly.  The main issue I've run into with getting it working properly are related to folder permission in the Program Files directory.  Below are some instructions on how to download the client from github, then to copy it into the Program Files directory, and to set the permissions so that non-admin users can run the software correctly.

Prerequisites:

• A Windows computer
• Administrator access (to run the script)
• Download the following files from the Internet:

1. The latest Shadowsocks-windows client in a .ZIP file from github.com [LINK] (Shadowsocks-4.0.9.zip as of this writing)
2. The latest Simple-obfs-Cygwin client in a .ZIP file from github (OPTIONAL) [LINK] (I use the msys2-x86-64.zip file)

Instructions:

Unzip all of the files into a single directory.  You should end up with 4 files:

mysys-2.0.dll
obfs-local.exe
obfs-server.exe
Shadowsocks.exe

Create a new batch file in the same directory.  For the sake of this example, we'll call it InstallSSWin.bat

Copy the following text into your InstallSSWin.bat file:

@echo off

::Shadowsocks Windows Client Installation Batch File

::Step 1 - Create folders in "Program Files"
md "%ProgramFiles%\Shadowsocks"
md "%ProgramFiles%\Shadowsocks\temp"
md "%ProgramFiles%\Shadowsocks\ss_win_temp"
md "%ProgramData%\Microsoft\Windows\Start Menu\Programs\Shadowsocks"

::Steup 2 - Set permissions so that all users can edit the contents of the folders
cacls "%ProgramFiles%\Shadowsocks" /e /p Users:C
cacls "%ProgramFiles%\Shadowsocks\temp" /e /p Users:C
cacls "%ProgramFiles%\Shadowsocks\ss_win_temp" /e /p Users:C


::Step 3 - Add shortcut to default programs files list
xcopy *.dll "%ProgramFiles%\Shadowsocks" /q /y
xcopy *.exe "%ProgramFiles%\Shadowsocks" /q /y
::xcopy *.lnk "%ProgramData%\Microsoft\Windows\Start Menu\Programs\Shadowsocks" /q /y

::Step 4 - Set firewall rules
netsh advfirewall firewall add rule name="Shadowsocks Allow - Shadowsocks (TCP)" dir=out action=allow program="C:\Program Files\Shadowsocks\Shadowsocks.exe" enable=yes protocol=TCP
netsh advfirewall firewall add rule name="Shadowsocks Allow - Shadowsocks (UDP)" dir=out action=allow program="C:\Program Files\Shadowsocks\Shadowsocks.exe" enable=yes protocol=UDP
netsh advfirewall firewall add rule name="Shadowsocks Allow - Privoxy (TCP)" dir=out action=allow program="C:\Program Files\Shadowsocks\ss_win_temp\ss_privoxy.exe" enable=yes protocol=TCP
netsh advfirewall firewall add rule name="Shadowsocks Allow - Privoxy (UDP)" dir=out action=allow program="C:\Program Files\Shadowsocks\ss_win_temp\ss_privoxy.exe" enable=yes protocol=UDP

Run the batch file "as administrator" and the files should be copied into the directories and other settings will be made for you.

Note: The script will also copy a shortcut to the Window Start Menu if you create one in the installation directory beforehand.  The trouble is that Windows will try to prevent you from creating a shortcut to a non-existent target, so it's only useful if you are trying to install Shadowsocks on multiple computers and take the time to create a shortcut and copy it over into your installation directory on the next computer....at least that's what I've done :)

You should be able to find the Shadowsocks.exe file in C:\Program Files\Shadowsocks and run it successfully now.  Make shortcuts to your desktop and add your server configs.


Blog Formatting Guide:

• Text written in a FWF in GREEN is text file contents.
• Commands written in Consolas in BLUE are LOCAL Client Console Commands
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in BOLD are CUSTOM user variables that your should change for your configuration.

Helpful References, for Shadowsocks:

Shadowsocks-libev on Github
Simple-obfs on Github

HOW TO: Build Shadowsock-libev from source on Ubuntu 16.04 LTS | Specify Commit Version

Introduction and Summary:

Shadowsocks is an encrypted proxy for circumventing firewalls and geo-location restrictions.  Some Linux server repositories do not have packages or if they do, the packages might be old since the server and client are under constant development.  These commands will help to download and build a specific version/commit of Shadowsocks-libev server from source.  In this example, we will build version 3.1.3 (since I've had problems with every version after that).

Prerequisites:

• Linux Ubuntu 16.04 LTS Server
• Console Access
• Root level permissions
• The following Ubuntu packages (some might be obsolete or unnecessary, send me a message if you discover an improvement):

build-essential autoconf libtool libssl-dev libpcre3-dev zlib1g-dev libev-dev git automake init-system-helpers pkg-config asciidoc xmlto libmbedtls-dev libc-ares-dev

Instructions:

Chose the "commit" that you want to build from.  Here's the master list of commits available which are identified by a 7 digit code.  Replace 85a64ae with the commit version you want.

When logged into the server, run the following commands:

cd /tmp
git clone https://github.com/shadowsocks/shadowsocks-libev.git
cd shadowsocks-libev
git checkout 85a64ae
git submodule update --init --recursive
./autogen.sh
./configure
make
sudo make install
cd ..
rm shadowsocks-libev -r -f

NOTE: The entire set of commands can be chained together in one long command:
cd /tmp && git clone https://github.com/shadowsocks/shadowsocks-libev.git && cd shadowsocks-libev && git checkout 85a64ae && git submodule update --init --recursive && ./autogen.sh && ./configure && make && sudo make install && cd .. && rm shadowsocks-libev -r -f

Blog Formatting Guide:

• Commands written in Consolas in BLUE are LOCAL Client Shell Commands (I'm running an Ubuntu local client)
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in BOLD are CUSTOM user variables that your should change for your configuration.

Helpful References, for Shadowsocks:

Shadowsocks-libev on Github

Simple-obfs on Github

Step-by-Step: Install OpenWRT on the Xiaomi Wifi Mini router from a Windows Computer (2018 edition)

The Xiaomi Mini wifi router has a low price and spacious ROM (16MB) and RAM (128MB) for easily supporting a VPN service on a OpenWRT/LEDE installation.  Thanks to the hard work by the community it’s fairly straight-forward to replace the Stock OEM firmware with LEDE.

PREREQUISITES

A. A computer with a web browser for downloading the files, registering a Xiaomi account, and capable of running SSH.

B. The LEDE Installation File from | download.openwrt.org

• miwifi-mini-squashfs-sysupgrade.bin

C. A Xiaomi Account for registering the router | https://account.xiaomi.com/pass/register

D.  The Developer edition of the Router ROM from Xiaomi | http://www.miwifi.com/miwifi_download.html

• Click the ROM graphic, scroll down to find “ROM for Mini 开发版”

E. A mobile device with Xiaomi’s router app installed | Google Play Link | Xiaomi APK

F. The customized SSH BIN file from Xiaomi.  This file can only be downloaded after registering the router with the Router app on your phone or tablet and it’s unique to each router since the ROOT password is apparently baked into it | Xiaomi SSH Site

G. A physical Ethernet connection to the router’s LAN ports (white colored).  Note: You can do most of the process on Wifi, but not all of it, so I recommend starting with Ethernet to simplify the process.

H. A USB drive formatted to FAT32 for installing the SSH BIN and LEDE BIN files to the router. The file is small, so the USB drive doesn't have to be big.

I. PuTTY for Windows to run SSH commands | Download page

Initial Setup of the Router

1. Connect a network patch cable from your Internet equipment to the WAN Uplink port (blue colored).
2. Connect a Windows computer to one of the Ethernet switch ports on the router (white colored).
3. Connect the router to power.
4. Launch a web browser and connect to the administrator console by going to the following address:
http://192.168.31.1, If you cannot reach the administrator console:

• Check the IP address on your computer and verify the it starts with 192.168.31
• Verify that you have any VPN software running on your computer that it is turned OFF
• Verify that the OS Proxy settings are disabled in “Internet Options”

5. Follow the setup wizard and configure the router.

• Note: It will ask you to set the wifi and the administrator password. Set it to 1234567890
• Note: All settings made during this process will be erased later, so don’t spend time making it nice...

After you complete the setup wizard, the router will reboot.

Unlock the Router's Firmware

1. Download the "Developer" ROM from Xiaomi, if you haven’t already (see pre-reqs step D).
2. Manually flash the Developer ROM through the web app console.  (Select the 3rd tab option, select the last menu item, and click the first command button to upload the file, click the right-hand button to confirm the installation of the new ROM. (It’s all in Chinese, so I’m trying to be helpful here, not condescending :-)

4. Install the Mi Wifi app on a mobile device (see pre-reqs step E)

5. Connect your mobile device to the Xiaomi MINI's wifi network

6. Sign-in to the mobile app with a "Mi" account or create a new one and then sign in. As of this writing, no personal information is needed other than an email account.

7. The app should detect the router.  Sign into the Wifi Router inside the app using the admin password you set during setup (in this example we set it to 1234567890) .  The router should now be registered to your Xiaomi account, our “Bound” as Google Translate says.

8. Download the the SSH BIN patch file at https://d.miwifi.com/rom/ssh. It's supposed to be custom and unique for your router, so I'm imagining it's because the BIN files needs your router's root password baked in.

<screenshot>

9. While you're at the Xiaomi SSH site copy down the "root" password from the page; you’ll need it for SSH later.

<screenshot>

10. Copy the miwifi_ssh.bin file from your computer to a USB drive formatted with FAT32.

11. Insert this USB drive into the USB port on the back of the Xiaomi router

12. Unplug the router's power cable.

13. Use a tool (like a straightened paperclip) to press down the reset button on the back of the router and hold it in.  While keeping the button pressed down, reconnect the power cable...not a task for the manually challenged. The yellow status light will come on at the front of the router. After about 10 seconds the yellow status light will begin to blink indicating that the router is now installing the SSH BIN file.  Once the yellow light is flashing like this you can release the reset button.

14. After the SSH file is installed, the router will reboot and the yellow light will become steady again during the boot process.  After the router is done booting, the status LED will turn a steady "blue" color.

15. Sign into the router using PuTTY on your Windows computer, using SSH (not Telnet) and enter the enter the root password from the Xiaomi website

• Note: On Linux, you can run ssh root@192.168.31.1 (this command works on a Windows computer as well if you've installed OpenSSH for Powershell from github)

<screenshot>

You're in! Enjoy your new super powers wisely

Install OpenWRT Stable Edition

1. Download the following LEDE BIN files from the Stable tree | Download Page (see pre-reqs step B)

2. Remove the USB drive from the router, copy the BIN file to it, and then re-insert the USB drive back into the router.

3.  Within the SSH console in PuTTY, use these commands check the MTD layout on the router to make sure you don't damage it:

cat /proc/mtd

4. If you find a line "OS1" in the text output from this command go ahead with flashing:

cd /extdisks/sda1

ls

5. Verify that the BIN file is in this directory from the text output. If it is, flash the ROM:

mtd -r write /extdisks/sda1/lede-17.01.4-ramips-mt7620-miwifi-mini-squashfs-sysupgrade.bin OS1

After flashing is complete, the router will reboot several times. It can seem to take a long time and interrupting the process can damage your router, so wait about 10 minutes to ensure that the firmware is installed properly before doing anything to the router.  The WebUI is not always accessible on first boot after all of the flashing steps, so power off the router and power it back-on by disconnecting the power plug and plugging it back in.

Also, please note that the OpenWRT OS defaults to the red color of the led instead of the orange/blue during and after finishing the boot that Xiaomi uses. (Need to double-check this with the newest build)

You're now running OpenWRT! Enjoy your more secure and flexible router.

REFERENCES

Xiaomi SSH Website After Registering Your Device in the App

小米ID：0000000000
已绑定1台小米路由器
Xiaomi_AAAA(小米路由器3增强版)root密码 xxxxxxxx下载工具包
工具包使用方法：小米路由器需升级到开发版0.5.28及以上，小米路由器mini需升级到开发版0.3.84及以上，小米路由器3即将支持。注意：稳定版不支持。
请将下载的工具包bin文件复制到U盘（FAT/FAT32格式）的根目录下，保证文件名为miwifi_ssh.bin；
断开小米路由器的电源，将U盘插入USB接口；
按住reset按钮之后重新接入电源，指示灯变为黄色闪烁状态即可松开reset键；
等待3-5秒后安装完成之后，小米路由器会自动重启，之后您就可以尽情折腾啦 ：）

Google Translate of Website
Has been bundled with a millet router
Xiaomi_AAAA (millet router 3 enhanced version) root password xxxxxx download kit
Toolkit to use: millet router to be upgraded to the development version of 0.5.28 and above, millet router mini to be upgraded to the development version of 0.3.84 and above, millet router 3 is about to support. Note: Steady version does not support.
Please download the toolkit bin file to U disk (FAT / FAT32 format) root directory, to ensure that the file name miwifi_ssh.bin;
Disconnect the power supply millet router, the U disk into the USB interface;
Hold down the reset button and then re-access the power, the indicator light to yellow flashing state can release the reset button;
Wait 3-5 seconds after the installation is complete, millet router will automatically restart, then you can enjoy tossing :)

HOW TO: Enable Socket SO_REUSEPORT for Shadowsocks-libev on Ubuntu 16.04 Server

Introduction and Summary:

     Shadowsocks is an encrypted proxy for circumventing firewalls and geo-location restrictions.  There is an option to enable SO_REUSEPORT based on the adjustments to the Linux kernel in version 3.9 in 2013.  This option seems to be most useful on a Shadowsocks server when handling many users ("millions of packets per second") in order to deal with congestion on a multi-core server.  Some applications see a speed increase of about 20% of traffic throughput on a congested server.  A Cloudflare engineer summarized it with these statements:

When this flag is set on a socket descriptor, Linux will allow many processes to bind to the same port. In fact, any number of processes will be allowed to bind and the load will be spread across them. With SO_REUSEPORT each of the processes will have a separate socket descriptor. Therefore each will own a dedicated UDP receive buffer."

In the author's summary (Marek Majkowski), he concluding by recommending the SO_REUSEPORT option be enabled by saying:

Ensure traffic is distributed evenly across many RX queues and SO_REUSEPORT processes. In practice, the load usually is well distributed as long as there are a large number of connections (or flows).

It seems that for SO_REUSEPORT to have a significant effect on network performance, the following would need to be true of a Shadowsocks server:

1. There are many users connecting to the server
2. The server is equipped with multiple CPU cores for handling traffic
3. The OS is Linux running kernel 3.9 or higher (a technical requirement)

Matt Haigh reinforces my understanding when he said the following in a Stack Overflow thread:

[SO_REUSEPORT] was added during the 3.9 cycle in a series of patches by Tom Herbert, as you can see here, in order to better support multithreaded web servers.
...
And yes, you can use SO_REUSEPORT to bind a socket to the same address and port as another connection, as long as the initial connection also uses SO_REUSEPORT (and any other connections sharing the source address and port). This is done to prevent port hijacking by rogue applications.

Note: That being said, there seems to be no disadvantage to enabling it if you are connecting from a Linux client, like OpenWRT.  I've enabled it with no noticeable adverse effects.  Nevertheless it's hard for me to comment yet if it has a significant positive effect since very few people are using my server.

Prerequisites:

• Linux Ubuntu 16.04 LTS Server
• Linux Kernel must be > 3.9.0
• Console Access
• Root level permission

Instructions:

1. Simply add the following line to your config.json file (formatted to be in the middle of the config, not the end

  "reuse_port":true,

or add this option to the command line string if you are not using the config file:

  --reuse-port

As far as I can tell, there's nothing to enable in Linux.  If the process declare to the SO_REUSEPORT to the kernel, it will automatically load balance connections.

Blog Formatting Guide:

• Commands written in Consolas in BLUE are LOCAL Client Shell Commands (I'm running an Ubuntu local client)
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in BOLD are CUSTOM user variables that your should change for your configuration.

Helpful References, for Shadowsocks:

Shadowsocks-libev on Github

TCP Page on Ubuntu.com
SO_REUSEPORT Tech Briefing at lwn.net
Gitlabs report from a School in Illinois
Cloudflare blog post on SO_REUSEPORT
CTOLIB.COM's schematic drawing
Ubuntu MAN page on getsockopt
Ubuntu MAN page on Sockets
Github conversation on failback mentions REUSEPORT as "distributes datagrams evenly across all of the receiving threads."

HOW TO: Install Shadowsock-libev using APT on Ubuntu 16.04 LTS

Introduction and Summary:

Shadowsocks is an encrypted proxy for circumventing firewalls and geo-location restrictions.  Some Linux server repositories do not have packages or if they do, the packages might be old since the server and client are under constant development.  I've written another post on how to build the packages from source, but this should only be done if you are willing to troubleshoot issues and want to maintain updating the builds by yourself.  A much easier way to install shadowsocks-libev on an Ubuntu server is to add the PPA to your server and install with APT.

Note: These packages are considered "unsupported" and "untrusted" by Ubuntu.

Prerequisites:

• Linux Ubuntu 16.04.3 LTS Server
• Console Access
• Root level permission

Instructions:

1. When logged into the server, run the following commands:

sudo add-apt-repository ppa:max-c-lv/shadowsocks-libev

2. Update your local APT library with the new packages

sudo apt-get update

3. Install the packages:

sudo apt install libsodium mbedtls shadowsocks-libev

Note: You can run all three of these commands in one line by entering the following string:

sudo add-apt-repository ppa:max-c-lv/shadowsocks-libev && sudo apt-get update && sudo apt install libsodium mbedtls shadowsocks-libev

Blog Formatting Guide:

• Commands written in Consolas in BLUE are LOCAL Client Shell Commands (I'm running an Ubuntu local client)
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in BOLD are CUSTOM user variables that your should change for your configuration.

Helpful References, for Shadowsocks:

Shadowsocks-libev on Github

Simple-obfs on Github

Ubuntu PPA Help Page for Shadowsocks-libev

HOW TO: Enable rng-tools on Ubuntu VPS Server 16.04 for more "random" data

Introduction and Summary:

Headless VPS servers can suffer from a lack of random data inputs since there is little if any hardware to pull truly random data from (mouse, keyboard, webcams, entropy keys, etc.).  A hack for semi-random data to be generated in software is possible with rng-tools.  These instructions will help you install rng-tools on your VPS and hack the config to pull from the urandom file.

Note: This random data is not truly random and it is generally not considered secure for highly sensitive applications.  I'll be using it for generating obfuscation data, so true randomness is not important to me. Also, my paranoia level is in the mid to low range - if you're requiring higher security solutions, please skip this article.

Prerequisites:

• Linux Ubuntu 16.04 LTS Server
• Console Access
• Root level permissions

Instructions:

1. Connect to the server console (local or SSH)
2. Install the rng-tools package:

  sudo apt update && apt install rng-tools

Note: If you start the service, you will see that it fails:

systemctl status rng-tools
● rng-tools.service
   Loaded: loaded (/etc/init.d/rng-tools; bad; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2017-10-26 04:11:02 EDT; 36min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 299 ExecStart=/etc/init.d/rng-tools start (code=exited, status=1/FAILURE)
Oct 26 04:11:02 ubuntuvps systemd[1]: Starting rng-tools.service...
Oct 26 04:11:02 ubuntuvps rng-tools[299]: Starting Hardware RNG entropy gatherer daemon: (Hardware RNG device inode not found)
Oct 26 04:11:02 ubuntuvps rng-tools[299]: /etc/init.d/rng-tools: Cannot find a hardware RNG device to use.
Oct 26 04:11:02 ubuntuvps systemd[1]: rng-tools.service: Control process exited, code=exited status=1
Oct 26 04:11:02 ubuntuvps systemd[1]: Failed to start rng-tools.service.
Oct 26 04:11:02 ubuntuvps systemd[1]: rng-tools.service: Unit entered failed state.
Oct 26 04:11:02 ubuntuvps systemd[1]: rng-tools.service: Failed with result 'exit-code'.

3. Add the /dev/urandom path to the rng config file:

  sudo echo "HRNGDEVICE=/dev/urandom" >> /etc/default/rng-tools

4. Start the service again

  systemctl start rng-tools

5. Check the status of rng-tools

  systemctl status rng-tools

● rng-tools.service
   Loaded: loaded (/etc/init.d/rng-tools; bad; vendor preset: enabled)
   Active: active (exited) since Thu 2017-10-26 04:47:45 EDT; 2s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 562 ExecStart=/etc/init.d/rng-tools start (code=exited, status=0/SUCCESS)
Oct 26 04:47:45 ubuntuvps systemd[1]: Starting rng-tools.service...
Oct 26 04:47:45 ubuntuvps rng-tools[562]: Starting Hardware RNG entropy gatherer daemon: rngd.
Oct 26 04:47:45 ubuntuvps systemd[1]: Started rng-tools.service.
Oct 26 04:47:45 ubuntuvps rngd[564]: RNDADDENTROPY failed: Operation not permitted

Blog Formatting Guide:

• Commands written in Consolas in BLUE are LOCAL Client Shell Commands (I'm running an Ubuntu local client)
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in Courier and BOLD are CUSTOM user variables that your should change for your configuration.
• All text written in Courier and PURPLE are file contents as shown in a text editor.

Helpful References:

Jon Stephen's post. Thank's Jon!

HOW TO: Use BBR TCP Congestion for Improved Shadowsocks Performance on Ubuntu Server 16.04 LTS

Introduction and Summary:

Shadowsocks uses the TCP protocol for transmitting information over the Internet.  A new "congestion algorithm" has been released by Google that shows good potential for increasing maximum speeds for Shadowsocks, TCP BBR. Newer Linux Kernels support BBR natively, so the basic instructions below update your server to a recent kernel and then switch the TCP congestion controls to BBR.  This only needs to be done on the server side; the client will receive TCP traffic control information from the server automatically.

Note: This only works on servers that you have full control of, like a dedicated server or a KVM VPS. It will not work on OpenVZ VPS's or similar types of virtualization environments since you cannot modify most networking settings or the core OS.

Prerequisites:

• Linux Ubuntu 16.04 LTS Server
• Console Access
• Root level permissions
• The following Ubuntu packages (some might be obsolete or unnecessary, send me a message if you discover an improvement): wget

Instructions:

1. Update the Ubuntu Linux Kernel to the most recent generic release (Note: The minimum kernel is v4.9 - feel free to use newer or older kernels if you want. You can browse the current kernel builds here: http://kernel.ubuntu.com/~kernel-ppa/mainline/)

sudo apt install linux-image-generic-hwe-16.04-edge

2. The server will reboot. Sign-in to the server again (console, ssh, etc.)
3. Enable BBR congestion algorithm

echo "net.core.default_qdisc=fq" | sudo tee -a /etc/sysctl.conf
echo "net.ipv4.tcp_congestion_control=bbr" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

4. Now check the results; you should see bbr as the current congestion control setting.

sysctl net.ipv4.tcp_available_congestion_control
sysctl net.ipv4.tcp_congestion_control
lsmod | grep bbr

5. Reboot the server to finalize the new settings

sudo reboot

Blog Formatting Guide:

• Commands written in Consolas in BLUE are LOCAL Client Shell Commands (I'm running an Ubuntu local client)
• Commands written in Consolas in RED are REMOTE Server Shell Commands (usually in SSH)
• All commands in Courier and BOLD are CUSTOM user variables that your should change for your configuration.
• All text written in Courier and PURPLE are file contents as shown in a text editor.

Helpful References, for Shadowsocks:

Shadowsocks-libev on Github
Ubuntu Server documentation on Mainline Kernel implementations
Google's BBR code on Github
Google blog post on BBR
ACM.org blog post on BBR