Spark实践笔记1:环境准备

  1. ubuntu server镜像,ParallelsDesktop下快速安装。采用12.04 server 32位版本,手动安装,使用英文语言,时区选中国,预装服务选了 openssh
  2. 修改root用户密码,启用root账户:sudo passwd root 。此后使用root用户操作。
  1. 设置静态IP:vim /etc/network/interfaces
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
root@ubuntu1:~# more /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
#iface eth0 inet dhcp
iface eth0 inet static
address 10.211.55.21
gateway 10.211.55.1
netmask 255.255.255.0
network 10.211.55.0
broadcast 10.211.55.255
# This is an autoconfigured IPv6 interface
iface eth0 inet6 auto
  1. 安装FTP:
1
2
apt-get install vsftpd
vim /etc/vsftpd.conf

修改:
#anonymous_enable=YES
local_enable=YES
write_enable=YES
重启:
/etc/init.d/vsftpd restart

  1. 装JAVA环境:下载地址
1
2
3
mkdir /usr/lib/jvm
tar xvf jdk-7u7-linux-i586.tar -C /usr/lib/jvm
mv jdk1.7.0_07 java

添加环境变量

1
2
3
4
5
vim ~/.bashrc
export JAVA_HOME=/usr/lib/jvm/java
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH

配置默认JAVA版本

1
2
3
4
5
update-alternatives --install /usr/bin/java java /usr/lib/jvm/java/bin/java 300
update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/java/bin/javac 300
update-alternatives --install /usr/bin/jar jar /usr/lib/jvm/java/bin/jar 300
update-alternatives --install /usr/bin/javah javah /usr/lib/jvm/java/bin/javah 300
update-alternatives --install /usr/bin/javap javap /usr/lib/jvm/java/bin/javap 300

然后执行
update-alternatives --config java
验证
java -version

  1. 重复操作搞定三台虚拟机
  • 10.211.55.21 ubuntu1
  • 10.211.55.22 ubuntu2
  • 10.211.55.23 ubuntu3
  1. 配置免密登陆:
  • 修改hostname vi /etc/hostname本次不用修改
  • 配置hosts

    1
    2
    3
    4
    127.0.0.1 localhost
    10.211.55.21 ubuntu1
    10.211.55.22 ubuntu2
    10.211.55.23 ubuntu3
  • 分别生成公钥、密钥
    ssh-keygen

  • 将公钥导入到三台机器
    cat id_rsa.pub >> ~/.ssh/authorized_keys
  • 测试一下
    1
    2
    3
    ssh ubuntu1
    ssh ubuntu2
    ssh ubuntu3