Lẩu thập cẩm: tháng 1 2014

Chủ Nhật, 19 tháng 1, 2014

Courses

https://education.mongodb.com/courses/

https://www.coursera.org/course/algs4partI?from_restricted_preview=1&course_id=970400&r=https%3A%2F%2Fclass.coursera.org%2Falgs4partI-003%2Flecture

Thứ Sáu, 17 tháng 1, 2014

Các bước cài đặt haproxy

1 - Get latest (stable) HAProxy:

$ curl -O http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.21.tar.gz
 $ tar xzpvf haproxy-1.4.21.tar.gz
 $ cd haproxy-1.4.21

2 - Compile:
$ make TARGET=generic ARCH=x86_64 USE_PCRE=1 3 - Install:
$ make install DESTDIR='/usr/local/haproxy' PREFIX=''
After installing HAProxy get the following sample configuration and save it to a file named haproxy.cfg .

Run HAProxy with:
$ ~/opt/haproxy/sbin/haproxy -f haproxy.conf

Nguồn: http://blog.pedrocarrico.net/post/25226892944/setting-up-haproxy-in-your-development-environment

Tạo file init.d để start|stop:

#!/bin/sh
#
# custom haproxy init.d script, by Mattias Geniar
#
# haproxy starting and stopping the haproxy load balancer
#
# chkconfig: 345 55 45
# description: haproxy is a TCP loadbalancer
# probe: true

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0

[ -f /usr/local/sbin/haproxy ] || exit 0

[ -f /usr/local/haproxy/haproxy.cfg ] || exit 0

# Define our actions
checkconfig() {
# Check the config file for errors
/usr/local/sbin/haproxy -c -q -f /usr/local/haproxy/haproxy.cfg
if [ $? -ne 0 ]; then
echo "Errors found in configuration file."
return 1
fi

# We're OK!
return 0
}

start() {
# Check config
/usr/local/sbin/haproxy -c -q -f /usr/local/haproxy/haproxy.cfg
if [ $? -ne 0 ]; then
echo "Errors found in configuration file."
return 1
fi

echo -n "Starting HAProxy: "
daemon /usr/local/sbin/haproxy -D -f /usr/local/haproxy/haproxy.cfg -p /var/run/haproxy.pid

RETVAL=$?
echo
[ $RETVAL -eq 0 ] && touch /var/lock/subsys/haproxy
return $RETVAL
}

stop() {
echo -n "Shutting down HAProxy: "
killproc haproxy -USR1

RETVAL=$?
echo
[ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/haproxy
[ $RETVAL -eq 0 ] && rm -f /var/run/haproxy.pid
return $RETVAL
}

restart() {
/usr/local/sbin/haproxy -c -q -f /usr/local/haproxy/haproxy.cfg
if [ $? -ne 0 ]; then
echo "Errors found in configuration file."
return 1
fi

stop
start
}

check() {
/usr/local/sbin/haproxy -c -q -V -f /usr/local/haproxy/haproxy.cfg
}

rhstatus() {
status haproxy
}

reload() {
/usr/local/sbin/haproxy -c -q -f /usr/local/haproxy/haproxy.cfg
if [ $? -ne 0 ]; then
echo "Errors found in configuration file."
return 1
fi

echo -n "Reloading HAProxy config: "
/usr/local/sbin/haproxy -f /usr/local/haproxy/haproxy.cfg -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)

success $"Reloading HAProxy config: "
echo
}

# Possible parameters
case "$1" in
start)
start
;;
stop)
stop
;;
status)
rhstatus
;;
restart)
restart
;;
reload)
reload
;;
checkconfig)
check
;;
*)
echo "Usage: haproxy {start|stop|status|restart|reload|checkconfig}"
exit 1
esac

exit 0

Thứ Hai, 13 tháng 1, 2014

Building HA Load Balancer with HAProxy and keepalived

For this tutorial I'll demonstrate how to build a simple yet scalable highly available HTTP load balancer using HAProxy [1] and keepalived [2], then later I'll show how to front-end HAProxy with Pound [5] and implement SSL termination and redirect the insecure connections from port 80 to 443.

Let's assume we have two servers LB1 and LB2 that will host HAProxy and will be made highly available through the use of the VRRP protocol [3] as implemented by keepalived. LB1 will have an IP address of 192.168.29.129 and LB2 will have an IP address of 192.168.29.130. The HAProxy will listen on the "shared/floating" IP address of 192.168.29.100, which will be raised on the active LB1. If LB1 fails that IP will be moved and raised on LB2 with the help of keepalived.

We are also going to have two back-end nodes that run apache - WEB1 192.168.29.131 and WEB2 192.168.29.132 - that will be receiving traffic from the HAProxy using round-robing load-balancing algorithm.

First let's install keepalived on both LB1 and LB2. We can either get it from the EPEL repo, or install it from source.

[root@lb1 ~] rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-5.noarch.rpm

[root@lb1 ~] yum install keepalived

Edit the configuration file on both servers to match except the priority parameter:

[root@lb1 ~] vi /etc/keepalived/keepalived.conf

vrrp_script chk_haproxy {

      script "killall -0 haproxy"

      interval 2

      weight 2

}

vrrp_instance VI_1 {

      interface eth0

      state MASTER

      virtual_router_id 51

      priority 101          # 101 on master, 100 on backup

      virtual_ipaddress {

           192.168.29.100

      }

      track_script {

           chk_haproxy

      }

}

Save the config on both servers and start keepalived:

[root@lb1 ~] /etc/init.d/keepalived start

Now that keepalived is running check that LB1 has raised 192.168.29.100:

[root@lb1 ~] ip addr show | grep 192.168.29.100

inet 192.168.29.100/32 scope global eth0

You can test if the IP will move from LB1 to LB2 by failing LB1 (shutdown or bring the network down) and running the above command on LB2.

Now that we have high availability of the IP resource we can install HAProxy on LB1 and LB2:

[root@lb1 ~] yum install haproxy

Edit the configuration file, and start HAProxy:

[root@lb1 ~] vi /etc/haproxy/haproxy.cfg

global

        log 127.0.0.1   local7 info

        maxconn 4096

        user haproxy

        group haproxy

        daemon

        #debug

        #quiet

defaults

        log     global

        mode    http

        option  httplog

        option  dontlognull

        retries 3

        option redispatch

        maxconn 2000

        contimeout      5000

        clitimeout      50000

        srvtimeout      50000

listen webfarm 192.168.29.100:80

      mode http

      balance roundrobin

      cookie JSESSIONID prefix

      option httpclose

      option forwardfor

      option httpchk HEAD /index.html HTTP/1.0

      server webA webserver1.example.net:80 cookie A check

      server webB webserver2.example.net:80 cookie B check

This is a very simplistic configuration that uses HTTP load-balancing with cookie prefixing. This is how it works:

- LB1 is VRRP master (keepalived), LB2 is backup. Both monitor the haproxy process, and lower their prio if it fails, leading to a failover to theother node.

- LB1 will receive clients requests on IP 192.168.29.100.

- both load-balancers send their checks from their native IP.

- if a request does not contain a cookie, it will be forwarded to a validserver

- in return, if a JESSIONID cookie is seen, the server name will be prefixedinto it, followed by a delimitor ('~')

- when the client comes again with the cookie "JSESSIONID=A~xxx", LB1 will know that it must be forwarded to server A. The server name will then be extracted from the cookie before it is sent to the server.

- if server "webA" dies, the requests will be sent to another valid serverand a cookie will be reassigned.

For more information and examples see [4].

Let's start HA proxy on both LB's:

[root@lb1 ~] /etc/init.d/haproxy start

To start it on LB2 you might have to fail LB1 first so that the shared IP moves to LB2 or make the following kernel change:

[root@web1 ~] sysctl -w net.ipv4.ip_nonlocal_bind=1

On the back-end apache nodes create a simple index.html like so:

[root@web1 ~] cat /var/www/html/index.html

This is Web Node 1

[root@web2 ~] cat /var/www/html/index.html

This is Web Node 2

Now hit 192.168.29.100 in your browser and refresh few times. You should see both nodes rotating in a round-robin fashion.

Also test the HA setup by failing one of the LB servers making sure that you always get a response back from the back-end nodes. Do the same for the back-end nodes.

To send logs from HAProxy to syslog-ng add the following lines to the syslog-ng config file:

[root@logserver ~] vi /etc/syslog-ng/syslog-ng.conf

source s_all {

    udp(ip(127.0.0.1) port(514));

};

destination df_haproxy { file("/var/log/haproxy.log"); };

filter f_haproxy { facility(local7); };

log { source(s_all); filter(f_haproxy); destination(df_haproxy); };

We can use Pound, which is a reverse proxy that supports SSL termination to listen for SSL connections on port 443 and terminate them using a local certificate. Pound will then insert a header in each HTTP packet called "X-Forwarded-Proto: https" that HAproxy will look for and if absent HAProxy will forward the insecure connections to port 443.
Installing pound is straight forward and can be done from a package or from source. Once installed the config file should look like this:

[root@lb1 ~] cat /etc/pound/pound.cfg 

User            "www-data"

Group           "www-data"

LogLevel        3

## check backend every X secs:

Alive           5

Control "/var/run/pound/poundctl.socket"

ListenHTTPS

        Address 192.168.29.100

        Port    443

        AddHeader "X-Forwarded-Proto: https"

        Cert    "/etc/ssl/local.server.pem"

        xHTTP           0

        Service

                BackEnd

                        Address 192.168.29.100

                        Port    80

                End

        End

End

[root@lb1 ~] /etc/init.d/pound start

Pound will now listen on port 443 for secure connections, terminate them using the local.server.pem certificate then inset the "X-Forwarded-Proto: https" header in the HTTP packet and forward it to HAProxy which is running and listening on the same host on port 80.
To make HAProxy forward all insecure connections from port 80 to port 443 all we need to do is create an access list that looks for the header that Pound inserts and if missing redirect the HTTP connections to Pound (listening on port 443).
The new config needs to look like this:

[root@lb1 ~] cat /etc/haproxy/haproxy.cfg

global

        log 127.0.0.1   local7 info

        maxconn 4096

        user haproxy

        group haproxy

        daemon

        #debug

        #quiet

defaults

        log     global

        mode    http

        option  httplog

        option  dontlognull

        retries 3

        option redispatch

        maxconn 2000

        contimeout      5000

        clitimeout      50000

        srvtimeout      50000

listen webfarm 192.168.29.100:80

      mode http

      balance roundrobin

      cookie JSESSIONID prefix

      option httpclose

      option forwardfor

      option httpchk HEAD /index.html HTTP/1.0

      acl x_proto hdr(X-Forwarded-Proto) -i https

      redirect location https://192.168.29.100/ if !x_proto

      server webA webserver1.example.net:80 cookie A check

      server webB webserver2.example.net:80 cookie B check

The two new lines at 31 and 32 create an access list that looks for (case insensitive -i) the https string in the X-Forwarded-Proto header. If the string is not there (meaning the connection came on port 80 directly hitting HAproxy) redirect to the secure SSL port 443 that Pound is listening on. This will ensure that each time a client hits port 80 the connection will be redirected to port 443 and secured. Same goes for if the client connects directly to port 443.
To generate a self-signed cert to use in Pound run this:

[root@lb1 ~] openssl req -x509 -newkey rsa:1024 -keyout local.server.pem -out local.server.pem -days 365 -nodes

Resources:

[1] http://haproxy.1wt.eu/

[2] http://www.keepalived.org/

[3] http://en.wikipedia.org/wiki/Virtual_Router_Redundancy_Protocol

[4] http://haproxy.1wt.eu/download/1.2/doc/architecture.txt
[5] http://www.apsis.ch/pound/

Nguồn: http://kaivanov.blogspot.com/2012/02/building-ha-load-balancer-with-haproxy.html

Thứ Hai, 6 tháng 1, 2014

The “right way” to handle file downloads in PHP

’ve seen many download scripts written in PHP, from simple one-liners to dedicated classes. Yet, at least half of them share common errors; in many cases programmers simply copy the code from something that works, without even attempting to understand what it really does.
What follows is not a complete working download script, but rather a set of issues you should be aware about and that will allow you to write better code.

1. Never accept paths as input

It’s very tempting to write something like

readfile($_GET['file']);

but before you do, think about it: anyone could request any file on the server, even if it’s outside the public html area. Guessing is not too difficult and in a few tries, an attacker could obtain configuration or password files.
You might think you’re being extra clever by doing something like

$mypath = '/mysecretpath/' .  $_GET['file'];

but an attacker can use relative paths to evade that.
What you must do – always – is sanitize the input. Accept only file names, like this:

$path_parts = pathinfo($_GET['file']);
$file_name  = $path_parts['basename'];
$file_path  = '/mysecretpath/' . $file_name;

And work only with the file name and add the path to it youserlf.
Even better would be to accept only numeric IDs and get the file path and name from a database (or even a text file or key=>value array if it’s something that doesn’t change often). Anything is better than blindly accept requests.
If you need to restrict access to a file, you should generate encrypted, one-time IDs, so you can be sure a generated path can be used only once.

2. Use headers correctly

This is a very widespread problem and unfortunately even the PHP manual is plagued with errors. Developers usually say “this works for me” and they copy stuff they don’t fully understand.
First of all, I notice the use of headers like Content-Description and Content-Transfer-Encoding. There is no such thing in HTTP. Don’t believe me? Have a look at RFC2616, they specifically state “HTTP, unlike MIME, does not use Content-Transfer-Encoding, and does use Transfer-Encoding and Content-Encoding“. You may add those headers if you want, but they do absolutely nothing. Sadly, this wrong example is present even in the PHP manual.
Second, regarding the MIME-type, I often see things like Content-Type: application/force-download. There’s no such thing and Content-Type: application/octet-stream (RFC1521) would work just as fine (or maybe application/x-msdownload if it’s an exe/dll). If you’re thinking about Internet Explorer, it’s even better to specify it clearly rather than force it to “sniff” the content. See MIME Type Detection in Internet Explorer for details.
Even worse, I see these kinds of statements:

header("Content-Type: application/force-download");
header("Content-Type: application/octet-stream");
header("Content-Type: application/download");

The author must have been really frustrated and added three Content-Type headers. The only problem is, as specified in the header() manual entry, “The optional replace parameter indicates whether the header should replace a previous similar header, or add a second header of the same type. By default it will replace“. So unless you specify header("Content-Type: some-value", FALSE), the new Content-Type header will replace the old one.

3. Forcing download and Internet Explorer bugs

What would it be like to not having to worry about old versions of Internet Explorer? A better world, that’s for sure.
To force a file to download, the correct way is:

header("Content-Disposition: attachment; filename=\"$file_name\"");

Note: the quotes in the filename are required in case the file may contain spaces.
The code above will fail in IE6 unless the following are added:

header("Pragma: public");
header("Cache-Control: must-revalidate, post-check=0, pre-check=0");

Now, the use of Cache-Control is wrong in this case, especially to both values set to zero, according to Microsoft, but it works in IE6 and IE7 and later ignores it so no harm done.
If you still get strange results when downloading (especially in IE), make sure that the PHP output compression is disabled, as well as any server compression (sometimes the server inadvertently applies compression on the output produced by the PHP script).

4. Handling large file sizes

readfile() is a simple way to ouput files files. Historically it had some performance issues and while the documentation claims there are no memory problems, real-life scenarios beg to differ - output buffering and other subtle things. Regardless, if you need byte ranges support, you still have to output the old-fashioned way.
The simplest way to handle this is to output the file in “chunks”:

set_time_limit(0);
$file = @fopen($file_path,"rb");
while(!feof($file))
{
 print(@fread($file, 1024*8));
 ob_flush();
 flush();
}

If you’re on Apache, there’s a very cool module called mod_xsendfile that makes the download simpler and faster. You just output a header and the module takes care of the rest. Of course, you must be able to install it and it also makes the code less portable so you probably won’t want to use this for redistributable code.

5. Disable Gzip / output compression / output buffering

This is the source of many seemingly obscure errors. If you have output buffering, the file will not be sent to the user in chunks but only at the end of the script. Secondly, you’re most likely to be outputting a binary file that does not need compression anyway. Thirdly, some older browser+server combinations might become confused that you’re requesting a text file (PHP) but you’re sending compressed data with a different content type.
To avoid this, assuming you’re using Apache, create a .htaccess file in the folder containing your download script with this directive:

SetEnv no-gzip dont-vary

This will disable compression in that folder.

6. Resumable downloads

For large files, it’s useful to allow downloads to be resumed. Doing so is more involved, but it’s really worth doing, especially if you serve large files or video/audio.
I’m not going to write a complete example, but to point you in the right direction.
First, you need to signal the browser that you support ranges:

header("Accept-Ranges: bytes");

Again, I’ve seen examples in which the actual byte range is given (e.g. 0-1000), which is wrong, according to the specs.
At the start of your script, after checking the file (if it exists, etc.), you have to check if a range is requested:

if (isset($_SERVER['HTTP_RANGE'])) 
 $range = $_SERVER['HTTP_RANGE'];

Ranges can be expressed like ‘bytes=-99‘ or ‘bytes=0-99‘ for the first 100 bytes, ‘bytes=100-‘ to skip the first 100 bytes, or ‘bytes=1720-8392‘ for something in the middle. Be aware that multiple ranges can be specified (e.g. ’100-200,400-’) but processing and especially delivering those ranges is more complicated so no one bothers.
So, now that you have the range, you have to make sure that’s expressed in bytes, that it does not contain multiple ranges and that the range itself is valid (end is greater that the start, start is not negative, and end is not larger than the file itself. Note that ‘bytes:-‘ is not a valid request. If the range is not valid, you must output

header('HTTP/1.1 416 Requested Range Not Satisfiable');

(yet again, many scripts get this wrong by sending 400 errors or other codes). Do not try to guess or fix the range(s) as it may result in corrupted downloads, which are more dangerous than failed ones.
Then, you must send a bunch of headers:

header('HTTP/1.1 206 Partial Content');
header('Accept-Ranges: bytes');
header("Content-Range: bytes $start-$end/$filesize");
$content_length = $end - $start + 1;
header("Content-Length: $length");

Every line contains a ‘gotcha’. Many developers forget to send the 206 code or the Accept-Ranges. Don’t forget that given a file size of 1000 bytes, a full range would be 0-999 so the Content-Range would be expressed as Content-Range: bytes 0-999/1000. Yet others forget that when you send a range, the Content-Length must match the length of the range rather than the size of the whole file.
You can output the file using the method described above, skipping until the start of the range and delivering the length of the range.

Closing thoughts

I did my best to provide only accurate information. It would be truly sad for me if an article about avoiding common PHP errors contained errors itself.
Regardless, my point stands: PHP makes it easy to hack together code that appears to be working, but developers should read and adhere to the official specifications.
UPDATE: I released a free script that adheres to the above guidelines.

Nguồn: http://www.richnetapps.com/the-right-way-to-handle-file-downloads-in-php/

How To Use PHP to Force a File Download

1. Never accept paths as input

It’s very tempting to write something like

readfile($_GET['file']);

$mypath = '/mysecretpath/' .  $_GET['file'];

but an attacker can use relative paths to evade that.
What you must do – always – is sanitize the input. Accept only file names, like this:

$path_parts = pathinfo($_GET['file']);
$file_name  = $path_parts['basename'];
$file_path  = '/mysecretpath/' . $file_name;

2. Use headers correctly

header("Content-Type: application/force-download");
header("Content-Type: application/octet-stream");
header("Content-Type: application/download");

3. Forcing download and Internet Explorer bugs

What would it be like to not having to worry about old versions of Internet Explorer? A better world, that’s for sure.
To force a file to download, the correct way is:

header("Content-Disposition: attachment; filename=\"$file_name\"");

Note: the quotes in the filename are required in case the file may contain spaces.
The code above will fail in IE6 unless the following are added:

header("Pragma: public");
header("Cache-Control: must-revalidate, post-check=0, pre-check=0");

4. Handling large file sizes

set_time_limit(0);
$file = @fopen($file_path,"rb");
while(!feof($file))
{
 print(@fread($file, 1024*8));
 ob_flush();
 flush();
}

5. Disable Gzip / output compression / output buffering

SetEnv no-gzip dont-vary

This will disable compression in that folder.

6. Resumable downloads

header("Accept-Ranges: bytes");

if (isset($_SERVER['HTTP_RANGE'])) 
 $range = $_SERVER['HTTP_RANGE'];

header('HTTP/1.1 416 Requested Range Not Satisfiable');

header('HTTP/1.1 206 Partial Content');
header('Accept-Ranges: bytes');
header("Content-Range: bytes $start-$end/$filesize");
$content_length = $end - $start + 1;
header("Content-Length: $length");

Lẩu thập cẩm

Chủ Nhật, 19 tháng 1, 2014

Courses

Thứ Sáu, 17 tháng 1, 2014

Các bước cài đặt haproxy

Thứ Hai, 13 tháng 1, 2014

Building HA Load Balancer with HAProxy and keepalived

Thứ Hai, 6 tháng 1, 2014

The “right way” to handle file downloads in PHP

1. Never accept paths as input

2. Use headers correctly

3. Forcing download and Internet Explorer bugs

4. Handling large file sizes

5. Disable Gzip / output compression / output buffering

6. Resumable downloads

Closing thoughts

How To Use PHP to Force a File Download

1. Never accept paths as input

2. Use headers correctly

3. Forcing download and Internet Explorer bugs

4. Handling large file sizes

5. Disable Gzip / output compression / output buffering

6. Resumable downloads

Closing thoughts

Thứ Sáu, 3 tháng 1, 2014

UNetbootin - Cài đặt Linux OS từ USB