Web proxy for linux

Использование HTTP proxy и SOCKS в Linux

В Linux существует много полезных консольных команд, которые при необходимости хотелось бы запустить через proxy. Некоторые приложения имеют встроенную поддержку proxy, а некоторые нет. Далее описано как пользоваться востребованными утилитами через proxy, даже теми, которые этой поддержки не имеют.

curl: передача данных через proxy

curl имеет полноценную поддержку как HTTP proxy так и SOCKS.

Для тестирования возможно использовать proxy сервера из бесплатных списков (socks — sockslist.net, и HTTP proxy — proxyhttp.net). Проверка IP адреса будет производиться с помощью ресурса check-host.net

# Проверить HTTP proxy curl --proxy 11.22.33.44:5555 check-host.net/ip # Тоже самое, но если для HTTP proxy требуется авторизация curl --proxy 11.22.33.44:5555 -U username:password check-host.net/ip # Проверить socks4 curl --socks4 11.22.33.44:5555 check-host.net/ip # Проверить socks5 curl --socks5 11.22.33.44:5555 check-host.net/ip # Тоже самое, только преобразование имен идет также через SOCKS # (подробнее о преобразовании имен можно прочитать ниже в подразделе "DNS запросы через proxy") curl --socks5-hostname 11.22.33.44:5555 check-host.net/ip 

Часть параметров curl можно записать в файл ~/.curlrc:

socks5 = 11.22.33.44:5555 proxy-user = username:password user-agent = "Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1" 

С помощью time и curl также можно замерить время отклика сервера:

# Без proxy: time curl check-host.net/ip # С proxy: time curl --socks5 11.22.33.44:5555 check-host.net/ip # Или любого сайта: time curl habrahabr.ru 

Результат будет выглядеть так:

real 0m0.307s user 0m0.000s sys 0m0.004s 

wget: закачка файлов через proxy

wget имеет встроенную поддержку proxy. Недостаток лишь в том, что только поддержку HTTP proxy. Для использования совместно с SOCKS рекомендуется использовать соксификатор dante.

 # Скачать файл через proxy: http_proxy="http://33.22.44.44:8080" wget http://www.google.com/favicon.ico Тоже самое, но для HTTPS https_proxy="http://33.22.44.44:8080" wget https://www.google.com/favicon.ico # Использовать proxy с авторизацией http_proxy="http://33.22.44.44:8080" wget --proxy-user=user --proxy-password=password http://www.google.com/favicon.ico 

Чтобы все время не указывать —proxy-user и —proxy-password можно их прописать в файл ~/.wgetrc:

proxy-user = username proxy-password = password user-agent = Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1 

ssh: доступ к серверам

Для доступа к серверам через ssh и proxy также лучше использовать соксификатор dante.

Соксификатор dante

apt-get install dante-client # пример для Debian-based систем 

# Если требуется зайти по ssh на сервер SOCKS_PASSWORD="" SOCKS_SERVER="11.22.33.44:1080" socksify ssh myserver # Тоже самое, только если для подключения к соксу требуется авторизация SOCKS_USERNAME="user" SOCKS_PASSWORD="password" SOCKS_SERVER="11.22.33.44:1080" socksify ssh myserver # Пример с использованием IRC клиента - irssi SOCKS_PASSWORD="" SOCKS_SERVER="11.22.33.44:1080" socksify irssi # Тоже самое, только с использованием HTTP proxy с поддержкой метода CONNECT HTTP_CONNECT_PROXY="http://11.22.33.44:8080" socksify irssi 

С помощью socksify можно направить через proxy почти любое приложение, не только консольное.

Чтобы все время не вводить данные о proxy можно создать файл /etc/socks.conf
Пример для SOCKS:

Пример для HTTP proxy с авторизацией:

А также экспортировать переменные SOCKS_USERNAME и SOCKS_PASSWORD, если для SOCKS или HTTP proxy требуется авторизация:

export SOCKS_USERNAME="username" export SOCKS_PASSWORD="password" 

DNS запросы через proxy

Часто требуется чтобы и преобразование имен происходило через proxy. Если использовать dante, то запрос на преобразование имен идет и через proxy, и через именной сервер указанный в /etc/resolv.conf . Понять почему же идет два одинаковых запроса вместо одного не удалось. Поэтому можно предложить два варианта:
1) Закомментировать именные сервера в файле /etc/resolv.conf, чтобы преобразование имен шло только через proxy. Это отразится на всей системе.
2) Изменить /etc/resolv.conf и выставить именные сервера необходимой страны, или просто отличные от серверов провайдера. Например установить сервера Google:

nameserver 8.8.8.8 nameserver 8.8.4.4 

Чтобы данные не были перезаписаны именными серверами провайдера (при переподключении), можно запретить обновление списка именных серверов сетевому менеджеру (NetworkManager/wicd) или DHCP-клиенту (спасибо ergil за корректировку).

Или воспользоваться «грубым» методом — запрещением изменения файла /etc/resolv.conf:

sudo chattr +i /etc/resolv.conf 

Если есть какие-то дополнения, пожалуйста, напишите, это будет полезно узнать и применить.

Источник

8 Top Open Source Reverse Proxy Servers for Linux

A reverse proxy server is a type of proxy server that is deployed between clients and back-end/origin servers, for example, an HTTP server such as NGINX, Apache, etc.. or application servers written in Nodejs, Python, Java, Ruby, PHP, and many other programming languages.

It is a gateway or an intermediary server that takes a client request, passes it on to one or more back-end servers, and subsequently fetches the response from the server and delivers it back to the client, thus making it appear as if the content originated from the reverse proxy server itself.

Generally, a reverse proxy server is an internal-facing proxy used as a ‘front-end‘ to control and protect access to back-end servers on a private network: it is typically deployed behind the network firewall.

It helps back-end servers to achieve anonymity to enhance their security. In an IT infrastructure, a reverse proxy can also function as an application firewall, load balancer, TLS terminator, web accelerator (by caching static and dynamic content), and much more.

In this article, we will review the 8 top open-source reverse proxy servers you can use on a Linux system.

1. HAProxy

HAProxy (HAProxy, which stands for High Availability Proxy), a free, open-source, very fast, reliable, and top-notch load balancer and proxying software for TCP and HTTP-based applications, built for high availability.

HAProxy is an HTTP reverse-proxy, a TCP proxy and normalizer, an SSL/TLS terminator/initiator/offloader, a caching proxy, an HTTP compression offloader, a traffic regulator, a content-based switch, a FastCGI gateway, and more. It is also a protection against DDoS and service abuse.

It is powered by an event-driven, non-blocking engine that combines a very fast I/O layer with a priority-based, multi-threaded scheduler which enables it to easily deal with tens of thousands of concurrent connections. Notably, HAProxy uses the PROXY protocol to pass the client’s connection information to backend or origin servers so that an application gets all the relevant information.

Some of HAProxy’s basic features include proxying, SSL support, monitoring both server states and its own state, high availability, load balancing, stickiness(maintain a visitor on the same server even across various events), content switching, HTTP rewriting, and redirection, server protection, logging, statistics, and much more.

2. NGINX

NGINX, a free, open-source, high-performance, and very popular HTTP server and reverse proxy. It also functions as an IMAP/POP3 proxy server. NGINX is well known for its high performance, stability, rich feature set, simple and flexible configuration, and low resource consumption (particularly small memory footprint).

Just like HAProxy, NGINX has an event-driven architecture so it has no problem dealing with tens of thousands of concurrent connections, as it uses HAProxy’s PROXY protocol.

NGINX supports accelerated reverse proxying with caching using the ngx_http_proxy_module module, which allows passing requests to another server over protocols other than HTTP, such as FastCGI, uwsgi, SCGI, and memcached.

Importantly, it supports load balancing and fault tolerance which are vital aspects of large-scale distributed computing systems. The ngx_http_upstream_module module allows for defining groups of backend servers to distribute the requests coming from clients. This makes your applications more robust, available and reliable, highly scalable, with response time and throughput. Additionally, concerning security, it supports SSL/TLS termination and so many other security features.

Useful articles on Nginx web server you might like to read:

3. Varnish HTTP Cache

Varnish HTTP Cache (or Varnish Cache or simply Varnish) is a free, open-source, high-performance, and very popular caching reverse-proxy software better known as a web application accelerator, designed to improve HTTP performance using server-side caching.

It is deployed between a client and an HTTP web server or application server; every time a client requests for information or a resource from a web server, Varnish stores a copy of the information, so the next time the client requests for the same information, Varnish will serve it without sending a request to the webserver thus reducing the load on the server and in turn speeding up web content delivery.

Varnish uses a flexible configuration language know as the Varnish Configuration Language (VLC) which among other things enables system administrators to configure how incoming requests should be processed, what content should be served, and from where, and how the request or response should be altered, and much more.

The varnish is also extensible – it can be extended using Varnish Modules (VMODs) and users can write their custom modules or use community provided modules.

The main limitation of Varnish is its lack of support for SSL/TLS. The only way to enable HTTPS is to deploy an SSL/TLS terminator or offloader such as HAProxy or NGINX in front of it.

4. Træfɪk

Træfɪk (pronounced Traffic) is a free, open-source, modern, and fast HTTP reverse proxy and load balancer for deploying micro-services that supports multiple load balancing algorithms. It can interface with various providers (or service discovery mechanisms or orchestration tools) such as Kubernates, Docker, Etcd, Rest API, Mesos/Marathon, Swarm, and Zookeper.

Its lovable feature is its ability to manage its configuration automatically and dynamically thus discovering the right configuration for your services. It does this by scanning your infrastructure to find relevant information and discovers which service serves which request from the external world. The providers tell Træfɪk where your applications or micro-services are located.

Træfɪk’s other features are supported for WebSockets, HTTP/2, and GRPC, and hot reloading (continuously updates its configuration without restarts), HTTPS using Let’s Encrypt certificates (wildcard certificate support), and exposes a REST API. It also keeps access logs, and it provides metrics (Rest, Prometheus, Datadog, Statsd, InfluxDB).

Also, Træfɪk ships with a simple HTML-based web user interface used to keep an eye on events. It also supports circuit breakers, retry requests, rate limiting, and basic authentication.

5. Apache Traffic Server

Formerly a commercial product owned by Yahoo which was later handed over to the Apache Foundation, Apache Traffic Server is a free, open-source, and fast caching forward and reverse-proxy server.

Traffic Server also works as a load balancer and can participate in flexible cache hierarchies. It is known to have handled over 400 TB a day of traffic at Yahoo.

It features a set of keep-alive, filtering, or anonymizing of content requests, and is extensible via an API that allows users to create custom plugins to modify HTTP headers, handle ESI requests, or design new cache algorithms.

6. Squid Proxy Server

Squid is a free, open-source, and well-known proxy server and Web cache daemon that supports various protocols such as HTTP, HTTPS, FTP, and more. It features a reverse proxy (httpd-accelerator) mode that caches incoming requests for outgoing data.

It supports rich traffic optimization options, access control, authorization, logging facilities, and much more.

7. Pound

A Pound is another free and open-source, lightweight reverse-proxy and load balancer and front-end for web servers. It is also an SSL terminator (that decrypts HTTPS requests from clients and sends them as plain HTTP to the back-end servers), an HTTP/HTTPS sanitizer(that verifies requests for correctness and accept only well-formed ones), and a fail-over server.

8. Apache HTTP Server

Last but not least, we have an Apache HTTP server (also known as HTTPD), the most popular web server on the globe. It can also be deployed and configured to act as a reverse proxy.

Additionally, you can also checkout Skipper, the new kid on the block. It is a free and open-source HTTP router and reverse proxy for service composition, including use cases like Kubernetes Ingress.

That’s all we had for you in this guide. For more information about each tool in this list, check out their respective websites. Do not forget to share your thoughts with us via the feedback form below.

Источник