Working Behind a Network Proxy: Difference between revisions

From Yocto Project
Jump to navigationJump to search
(Remove leading dot from domains in no_proxy, leading dot no longer works for many modern distros and this now matches W3C documentation on proxy variables. See discussion in https://bugzilla.yoctoproject.org/show_bug.cgi?id=10952)
 
(6 intermediate revisions by 4 users not shown)
Line 1: Line 1:
This page lists some configuration tips for working behind a proxy.
= Background =
The Yocto Project build system provides the tools and environment to build the packages that comprise a bootable Linux image. All the sources for the packages must be fetched through various means, including wget, git, and others. You need to configure your system to work with your corporate proxies and there are two options for doing this
# '''Environment variables and configuration files''' - this is the traditional and least intrusive option but involves a number of steps and it can fail in some situations.
# '''Chameleonsocks''' - this is [https://github.com/darkk/redsocks Redsocks] auto-configured in a [https://www.docker.com/ docker] container, the Yocto team's favorite option. If in doubt, go with this option.
 
= Option 1: Environment Variables and Configuration Files =


== HTTP/HTTPS/FTP Setup ==
== HTTP/HTTPS/FTP Setup ==
 
Set the following environment variables in your <tt>~/.bashrc</tt> file. This example uses the same proxy server and port number for all three protocols.
Set the following environment variables in your ~/.bashrc file. This example uses the same proxy server and port number for all three protocols.


  export http_proxy='http://myproxy.example.com:1080/'
  export http_proxy='http://myproxy.example.com:1080/'
  export https_proxy='https://myproxy.example.com:1080/'
  export https_proxy='https://myproxy.example.com:1080/'
  export ftp_proxy='http://myproxy.example.com:1080/'
  export ftp_proxy='http://myproxy.example.com:1080/'
  export no_proxy = '.example.com'
  export ALL_PROXY='socks://myproxy.example.com:1080/'
 
  export all_proxy='socks://myproxy.example.com:1080/'
== Git Setup (with socat)==
  export no_proxy='example.com'
 
First make sure you have the socat utility installed on your host (in Ubuntu, this should be a simple command "sudo apt-get install socat")
 
Create a script named ''git-proxy'' and put it in /usr/local/bin:
 
#!/bin/bash
# $1 = hostname, $2 = port
PROXY=myproxy.example.com
exec socat STDIO SOCKS4:$proxy:$1:$2
Then run the following command:
 
  git config  --global  core.gitProxy git-proxy
 
== Git Setup (with nc)==
 
First make sure you have the netcat utility (nc) installed on your host.
 
Create a script named ''git-proxy'' and put it in /usr/local/bin:
 
#!/bin/bash
PROXY=myproxy.example.com
  PORT=1080
case $1 in
        # list internal git servers here that you do not want to use
        # the proxy with, separated by a pipe character '|' as below:
internalgit1.example.com|internalgit2.example.com)
        METHOD="-X connect"
        ;;
*)
        METHOD="-X 5 -x ${PROXY}:${PORT}"
        ;;
esac
/usr/bin/nc $METHOD $*
 
Note that on some Linux distros, the nc binary is in /bin. You can also change the '5' in the second METHOD line to '4' if your proxy server only supports SOCKS v4.
 
Then set the environment variable GIT_PROXY_COMMAND in your ~/.bashrc file and point it to this script:


export GIT_PROXY_COMMAND=/usr/local/bin/git-proxy
== Git Proxy Configuration ==
export GIT_PROXY_IGNORE="example.com"
Git allows you to specify a command via GIT_PROXY_COMMAND to proxy its network communications. The script, [http://git.yoctoproject.org/cgit/cgit.cgi/poky/plain/scripts/oe-git-proxy oe-git-proxy], uses socat and standard proxy environment variables. Make sure socat is installed otherwise oe-git-proxy will fail silently.


== Subversion Setup ==
Copy [http://git.yoctoproject.org/cgit/cgit.cgi/poky/plain/scripts/oe-git-proxy oe-git-proxy] to a location in your PATH, typically ~/bin/ is a good choice. /usr/local/bin is another common option. oe-git-proxy is provided in poky/scripts/oe-git-proxy, but use the link below to get you bootstrapped. Replacing it with the latest from the poky git repository is recommended.


You'll need to have the following in your ~/.subversion/servers file:
$ sudo apt-get install socat
$ wget http://git.yoctoproject.org/cgit/cgit.cgi/poky/plain/scripts/oe-git-proxy
$ cp oe-git-proxy ~/bin
$ chmod +x ~/bin/oe-git-proxy


[global]
Update your .bashrc to setup the GIT_PROXY_COMMAND variable. You also need to set the '''NO_PROXY''' variable as oe-git-proxy does no recognize '''no_proxy'''
http-proxy-exceptions = *.exception.com, www.internal-site.org
http-proxy-host = myproxy.example.com
http-proxy-port = 1080


You can also set ''http-proxy-username'' and ''http-proxy-password'' if your proxy requires authentication.
export GIT_PROXY_COMMAND="oe-git-proxy"
export NO_PROXY=$no_proxy


== CVS Setup (this seems useless since we don't have any recipe requiring cvs now) ==
Be sure to '''log out and back in''' for the settings to take effect.


For CVS checkouts to work correctly, you need to add some options in your Poky ''local.conf'' file.
== Wget Proxy Configuration ==
Proxy environment variables are not propagated to all stages of the Yocto build process so sometimes wget cannot pick them up. To workaround this add the following to ~/.wgetrc.
https_proxy = http://socks.example.com:1080
http_proxy = http://socks.example.com:1080
ftp_proxy = http://socks.example.com:1080
no_proxy = example.com:1080;127.0.0.1
use_proxy = on


CVS_PROXY_HOST = "myproxy.example.com"
= Option 2: Chameleonsocks =
CVS_PROXY_PORT = "1080"
Chameleonsocks is a containerized proxy solution that is very easy to set up.  
* Get files from https://github.com/crops/chameleonsocks
* Follow instructions in [https://github.com/crops/chameleonsocks/blob/master/README.MD README]

Latest revision as of 16:50, 2 March 2017

Background

The Yocto Project build system provides the tools and environment to build the packages that comprise a bootable Linux image. All the sources for the packages must be fetched through various means, including wget, git, and others. You need to configure your system to work with your corporate proxies and there are two options for doing this

  1. Environment variables and configuration files - this is the traditional and least intrusive option but involves a number of steps and it can fail in some situations.
  2. Chameleonsocks - this is Redsocks auto-configured in a docker container, the Yocto team's favorite option. If in doubt, go with this option.

Option 1: Environment Variables and Configuration Files

HTTP/HTTPS/FTP Setup

Set the following environment variables in your ~/.bashrc file. This example uses the same proxy server and port number for all three protocols. 

export http_proxy='http://myproxy.example.com:1080/'
export https_proxy='https://myproxy.example.com:1080/'
export ftp_proxy='http://myproxy.example.com:1080/'
export ALL_PROXY='socks://myproxy.example.com:1080/'
export all_proxy='socks://myproxy.example.com:1080/'
export no_proxy='example.com'

Git Proxy Configuration

Git allows you to specify a command via GIT_PROXY_COMMAND to proxy its network communications. The script, oe-git-proxy, uses socat and standard proxy environment variables. Make sure socat is installed otherwise oe-git-proxy will fail silently.

Copy oe-git-proxy to a location in your PATH, typically ~/bin/ is a good choice. /usr/local/bin is another common option. oe-git-proxy is provided in poky/scripts/oe-git-proxy, but use the link below to get you bootstrapped. Replacing it with the latest from the poky git repository is recommended. 

$ sudo apt-get install socat
$ wget http://git.yoctoproject.org/cgit/cgit.cgi/poky/plain/scripts/oe-git-proxy
$ cp oe-git-proxy ~/bin
$ chmod +x ~/bin/oe-git-proxy

Update your .bashrc to setup the GIT_PROXY_COMMAND variable. You also need to set the NO_PROXY variable as oe-git-proxy does no recognize no_proxy 

export GIT_PROXY_COMMAND="oe-git-proxy"
export NO_PROXY=$no_proxy

Be sure to log out and back in for the settings to take effect.

Wget Proxy Configuration

Proxy environment variables are not propagated to all stages of the Yocto build process so sometimes wget cannot pick them up. To workaround this add the following to ~/.wgetrc. 

https_proxy = http://socks.example.com:1080
http_proxy = http://socks.example.com:1080
ftp_proxy = http://socks.example.com:1080
no_proxy = example.com:1080;127.0.0.1
use_proxy = on

Option 2: Chameleonsocks

Chameleonsocks is a containerized proxy solution that is very easy to set up.

Retrieved from "https://wiki.yoctoproject.org/wiki/index.php?title=Working_Behind_a_Network_Proxy&oldid=24637"