旧情報実験機活用プロジェクト:pirikaインストール

概要

NFS サーバを設置します。 NFS サーバに関しては、安定性重視のため認証の統合化はせず、他のサーバに一切依存しないような構成にします。

事前準備

何でもいいのでとりあえずまともに動くパソコンを用意。

ep アカウント作成
デスクトップパソコン導入 (HP Compaq 8100 Elite SFF PC)
MAC アドレスをプライベート登録 (パソコン一台目の登録は本人にはできない仕様なので注意)
FreeBSD のインストール CD イメージをダウンロードして　CD-R に書き込む
CVSup のポート(5999)をあけてもらう

ハードウェア

色々と試したのですが、ここにある機械のうちで最もハイスペックなものを投入するのが一番安定性がよかったので、そのようにします。

CPU: Core i7 950 3.0GHz
Memory: DDR3-12800 2Gx6 (Triple channel)
MotherBoard: ASUS P6X58D-E
SSD: Intel SSDSA2M080G2GC (80G, for system)
HDD: 36.2 Tbytes ZFS (RAIDZ x 5, zfspool)
- onboard SATA: WD20EARS x 5, ZFS, RAIDZ(ad24, ad26, ad28, ad30) + spare(ad20)
- PCIe, hptrr0 (sx508x): HDS722020ALA330 x 8, ZFS, RAIDZ(da0, da1, da2, da3) RAIDZ(da4, da5, da6, da7)
- PCIe, hptrr1 (sx508x): HDS722020ALA330 x 8, ZFS, RAIDZ(da8, da9, da10, da11) RAIDZ(da12, da13, da14, da15)
LAN: onboard, Marvell Yukon 88E8056 Gigabit Ethernet (msk)

OS インストール

OS は FreeBSD, ファイルシステムには ZFS を使用します

８号館ネットにMACアドレスをプライベート登録
余っている CD ドライブをつないでFreeBSD をとりあえず SSD に最小インストール
- partition は適当に切る(/ を少し大きめに)
- swap は後で作るのでなし
- ユーザ aki:aki = 10001:10001 で作成
- ip は dhcp で取得
ハードディスク(20台)をつないで起動
dmesg でディスクが全部認識されていることを確認。
- デバイス名を確認
/etc/rc.conf に追加
```
zfs_enable="YES"
```

フォーマットする

#!/bin/sh

i=1
zfs=""
for p in \
da0 da1 da2 da3 da4 da5 da6 da7 da8 da9 da10 da11 da12 da13 da14 da15 \
ad24 ad26 ad28 ad30
do
  gpart create -s gpt $p
  gpart add -b 34 -s 64k -t freebsd-boot $p
  gpart add -t freebsd-zfs -l disk$i $p
  gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 $p
  echo "===> $p (disk$i) "; gpart show $p
  i=`expr $i + 1`
  zfs="$zfs ${p}p2"
done

zpool create zfspool raidz ad24p2 ad26p2 ad28p2 ad30p2
zpool add zfspool raidz da0p2 da1p2 da2p2 da3p2
zpool add zfspool raidz da4p2 da5p2 da6p2 da7p2
zpool add zfspool raidz da8p2 da9p2 da10p2 da11p2
zpool add zfspool raidz da12p2 da13p2 da14p2 da15p2
zfs set checksum=fletcher4 zfspool

zfs にシステム用ボリューム追加

zfs set mountpoint=legacy zfspool
zfs umount -a
zfs create -o mountpoint=/system zfspool/system
zfs create -o mountpoint=/home zfspool/home
zfs create -o mountpoint=legacy zfspool/system/pirika
zfs create -o mountpoint=legacy -o compression=off -o compression=off -o exec=off -o setuid=off zfspool/system/pirika/distfiles
zfs create -o mountpoint=legacy zfspool/system/pirika/local
zfs create -o mountpoint=legacy -o compression=lzjb zfspool/system/pirika/ports
zfs create -o mountpoint=legacy -o compression=off  zfspool/system/pirika/obj
zfs create -o mountpoint=legacy -o compression=lzjb zfspool/system/pirika/src
zfs create -V 1gb zfspool/system/pirika/swap
zfs set org.freebsd:swap=on zfspool/system/pirika/swap

/etc/fstab にエントリ追加

zfspool/home                        /home                zfs        rw        2        0
zfspool/system/pirika/local        /usr/local        zfs        rw        2        0
zfspool/system/pirika/obj        /usr/obj        zfs        rw        2        0
zfspool/system/pirika/src        /usr/src        zfs        rw        2        0
zfspool/system/pirika/ports        /usr/ports        zfs        rw        2        0
zfspool/system/pirika/distfiles        /usr/ports/distfiles        zfs        rw        2        0

再起動して自動で mount されるかを確認

カーネルとツールのコンパイル

ソースを持ってきてコンパイルする

sysinstall で ports と全てのソースを入れる
/usr/sup ディレクトリを作る

/usr/sup/src-supfile を書く

*default host=cvsup2.jp.FreeBSD.org
*default base=/var/db
*default prefix=/usr
*default release=cvs tag=RELENG_8
*default delete use-rel-suffix
*default compress
src-all

/usr/sup/ports-supfile を書く

*default host=cvsup3.jp.FreeBSD.org
*default base=/var/db
*default prefix=/usr
*default release=cvs tag=.
*default delete use-rel-suffix
*default compress
ports-all

csup -gL2 src-supfile を実行
/usr/src/sys/amd64 に移動してカーネルのコンパイル設定をする
```
cp GENERIC PIRIKA
vi PIRIKA
```

不必要なドライバをコメントアウトする

#
# GENERIC -- Generic kernel configuration file for FreeBSD/amd64
#
# For more information on this file, please read the config(5) manual page,
# and/or the handbook section on Kernel Configuration Files:
#
#    http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html
#
# The handbook is also available locally in /usr/share/doc/handbook
# if you've installed the doc distribution, otherwise always see the
# FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the
# latest information.
#
# An exhaustive list of options and more detailed explanations of the
# device lines is also present in the ../../conf/NOTES and NOTES files.
# If you are in doubt as to the purpose or necessity of a line, check first
# in NOTES.
#
# $FreeBSD: src/sys/amd64/conf/GENERIC,v 1.531.2.15.2.1 2010/12/21 17:09:25 kensmith Exp $

cpu             HAMMER
ident           PIRIKA

# To statically compile in device wiring instead of /boot/device.hints
#hints          "GENERIC.hints"         # Default places to look for devices.

# Use the following to compile in values accessible to the kernel
# through getenv() (or kenv(1) in userland). The format of the file
# is 'variable=value', see kenv(1)
#
# env           "GENERIC.env"

makeoptions     DEBUG=-g                # Build kernel with gdb(1) debug symbols

options         SCHED_ULE               # ULE scheduler
options         PREEMPTION              # Enable kernel thread preemption
options         INET                    # InterNETworking
options         INET6                   # IPv6 communications protocols
options         SCTP                    # Stream Control Transmission Protocol
options         FFS                     # Berkeley Fast Filesystem
options         SOFTUPDATES             # Enable FFS soft updates support
options         UFS_ACL                 # Support for access control lists
options         UFS_DIRHASH             # Improve performance on big directories
options         UFS_GJOURNAL            # Enable gjournal-based UFS journaling
options         MD_ROOT                 # MD is a potential root device
options         NFSCLIENT               # Network Filesystem Client
options         NFSSERVER               # Network Filesystem Server
options         NFSLOCKD                # Network Lock Manager
options         NFS_ROOT                # NFS usable as /, requires NFSCLIENT
options         MSDOSFS                 # MSDOS Filesystem
options         CD9660                  # ISO 9660 Filesystem
options         PROCFS                  # Process filesystem (requires PSEUDOFS)
options         PSEUDOFS                # Pseudo-filesystem framework
options         GEOM_PART_GPT           # GUID Partition Tables.
options         GEOM_LABEL              # Provides labelization
options         COMPAT_43TTY            # BSD 4.3 TTY compat (sgtty)
options         COMPAT_FREEBSD32        # Compatible with i386 binaries
options         COMPAT_FREEBSD4         # Compatible with FreeBSD4
options         COMPAT_FREEBSD5         # Compatible with FreeBSD5
options         COMPAT_FREEBSD6         # Compatible with FreeBSD6
options         COMPAT_FREEBSD7         # Compatible with FreeBSD7
options         SCSI_DELAY=5000         # Delay (in ms) before probing SCSI
options         KTRACE                  # ktrace(1) support
options         STACK                   # stack(9) support
options         SYSVSHM                 # SYSV-style shared memory
options         SYSVMSG                 # SYSV-style message queues
options         SYSVSEM                 # SYSV-style semaphores
options         P1003_1B_SEMAPHORES     # POSIX-style semaphores
options         _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
options         PRINTF_BUFR_SIZE=128    # Prevent printf output being interspersed.
options         KBD_INSTALL_CDEV        # install a CDEV entry in /dev
options         HWPMC_HOOKS             # Necessary kernel hooks for hwpmc(4)
options         AUDIT                   # Security event auditing
options         MAC                     # TrustedBSD MAC Framework
options         FLOWTABLE               # per-cpu routing cache
#options        KDTRACE_FRAME           # Ensure frames are compiled in
#options        KDTRACE_HOOKS           # Kernel DTrace hooks
options         INCLUDE_CONFIG_FILE     # Include this file in kernel

options         KDB                     # Kernel debugger related code
options         KDB_TRACE               # Print a stack trace for a panic

# Make an SMP-capable kernel by default
options         SMP                     # Symmetric MultiProcessor Kernel

# CPU frequency control
device          cpufreq

# Bus support.
device          acpi
device          pci

# Floppy drives
device          fdc

# ATA and ATAPI devices
device          ata
device          atadisk         # ATA disk drives
device          ataraid         # ATA RAID drives
device          atapicd         # ATAPI CDROM drives
device          atapifd         # ATAPI floppy drives
device          atapist         # ATAPI tape drives
options         ATA_STATIC_ID   # Static device numbering

# SCSI Controllers
#device         ahc             # AHA2940 and onboard AIC7xxx devices
#options        AHC_REG_PRETTY_PRINT    # Print register bitfields in debug
                                        # output.  Adds ~128k to driver.
#device         ahd             # AHA39320/29320 and onboard AIC79xx devices
#options        AHD_REG_PRETTY_PRINT    # Print register bitfields in debug
                                        # output.  Adds ~215k to driver.
#device         amd             # AMD 53C974 (Tekram DC-390(T))
#device         hptiop          # Highpoint RocketRaid 3xxx series
#device         isp             # Qlogic family
#device         ispfw           # Firmware for QLogic HBAs- normally a module
#device         mpt             # LSI-Logic MPT-Fusion
#device         ncr             # NCR/Symbios Logic
#device         sym             # NCR/Symbios Logic (newer chipsets + those of `ncr')
#device         trm             # Tekram DC395U/UW/F DC315U adapters

#device         adv             # Advansys SCSI adapters
#device         adw             # Advansys wide SCSI adapters
#device         aic             # Adaptec 15[012]x SCSI adapters, AIC-6[23]60.
#device         bt              # Buslogic/Mylex MultiMaster SCSI adapters

# SCSI peripherals
device          scbus           # SCSI bus (required for SCSI)
device          ch              # SCSI media changers
device          da              # Direct Access (disks)
device          sa              # Sequential Access (tape etc)
device          cd              # CD
device          pass            # Passthrough device (direct SCSI access)
device          ses             # SCSI Environmental Services (and SAF-TE)

# RAID controllers interfaced to the SCSI subsystem
#device         amr             # AMI MegaRAID
#device         arcmsr          # Areca SATA II RAID
#XXX it is not 64-bit clean, -scottl
#device         asr             # DPT SmartRAID V, VI and Adaptec SCSI RAID
#device         ciss            # Compaq Smart RAID 5*
#device         dpt             # DPT Smartcache III, IV - See NOTES for options
device          hptmv           # Highpoint RocketRAID 182x
device          hptrr           # Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx
#device         iir             # Intel Integrated RAID
#device         ips             # IBM (Adaptec) ServeRAID
#device         mly             # Mylex AcceleRAID/eXtremeRAID
#device         twa             # 3ware 9000 series PATA/SATA RAID

# RAID controllers
#device         aac             # Adaptec FSA RAID
#device         aacp            # SCSI passthrough for aac (requires CAM)
#device         ida             # Compaq Smart RAID
#device         mfi             # LSI MegaRAID SAS
#device         mlx             # Mylex DAC960 family
#XXX pointer/int warnings
#device         pst             # Promise Supertrak SX6000
#device         twe             # 3ware ATA RAID

# atkbdc0 controls both the keyboard and the PS/2 mouse
device          atkbdc          # AT keyboard controller
device          atkbd           # AT keyboard
device          psm             # PS/2 mouse

device          kbdmux          # keyboard multiplexer

device          vga             # VGA video card driver

device          splash          # Splash screen and screen saver support

# syscons is the default console driver, resembling an SCO console
device          sc

#device         agp             # support several AGP chipsets

# PCCARD (PCMCIA) support
# PCMCIA and cardbus bridge support
#device         cbb             # cardbus (yenta) bridge
#device         pccard          # PC Card (16-bit) bus
#device         cardbus         # CardBus (32-bit) bus

# Serial (COM) ports
device          uart            # Generic UART driver

# Parallel port
device          ppc
device          ppbus           # Parallel port bus (required)
device          lpt             # Printer
device          plip            # TCP/IP over parallel
device          ppi             # Parallel port interface device
#device         vpo             # Requires scbus and da
# If you've got a "dumb" serial or parallel PCI card that is
# supported by the puc(4) glue driver, uncomment the following
# line to enable it (connects to sio, uart and/or ppc drivers):
#device         puc

# PCI Ethernet NICs.
#device         de              # DEC/Intel DC21x4x (``Tulip'')
#device         em              # Intel PRO/1000 Gigabit Ethernet Family
#device         igb             # Intel PRO/1000 PCIE Server Gigabit Family
#device         ixgbe           # Intel PRO/10GbE PCIE Ethernet Family
#device         le              # AMD Am7900 LANCE and Am79C9xx PCnet
#device         ti              # Alteon Networks Tigon I/II gigabit Ethernet
#device         txp             # 3Com 3cR990 (``Typhoon'')
#device         vx              # 3Com 3c590, 3c595 (``Vortex'')

# PCI Ethernet NICs that use the common MII bus controller code.
# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs!
device          miibus          # MII bus support
#device         ae              # Attansic/Atheros L2 FastEthernet
#device         age             # Attansic/Atheros L1 Gigabit Ethernet
#device         alc             # Atheros AR8131/AR8132 Ethernet
#device         ale             # Atheros AR8121/AR8113/AR8114 Ethernet
#device         bce             # Broadcom BCM5706/BCM5708 Gigabit Ethernet
#device         bfe             # Broadcom BCM440x 10/100 Ethernet
#device         bge             # Broadcom BCM570xx Gigabit Ethernet
#device         dc              # DEC/Intel 21143 and various workalikes
#device         et              # Agere ET1310 10/100/Gigabit Ethernet
#device         fxp             # Intel EtherExpress PRO/100B (82557, 82558)
#device         jme             # JMicron JMC250 Gigabit/JMC260 Fast Ethernet
#device         lge             # Level 1 LXT1001 gigabit Ethernet
device          msk             # Marvell/SysKonnect Yukon II Gigabit Ethernet
#device         nfe             # nVidia nForce MCP on-board Ethernet
#device         nge             # NatSemi DP83820 gigabit Ethernet
#device         nve             # nVidia nForce MCP on-board Ethernet Networking
#device         pcn             # AMD Am79C97x PCI 10/100 (precedence over 'le')
device          re              # RealTek 8139C+/8169/8169S/8110S
device          rl              # RealTek 8129/8139
#device         sf              # Adaptec AIC-6915 (``Starfire'')
#device         sge             # Silicon Integrated Systems SiS190/191
#device         sis             # Silicon Integrated Systems SiS 900/SiS 7016
#device         sk              # SysKonnect SK-984x & SK-982x gigabit Ethernet
#device         ste             # Sundance ST201 (D-Link DFE-550TX)
#device         stge            # Sundance/Tamarack TC9021 gigabit Ethernet
#device         tl              # Texas Instruments ThunderLAN
#device         tx              # SMC EtherPower II (83c170 ``EPIC'')
#device         vge             # VIA VT612x gigabit Ethernet
#device         vr              # VIA Rhine, Rhine II
#device         wb              # Winbond W89C840F
#device         xl              # 3Com 3c90x (``Boomerang'', ``Cyclone'')

# ISA Ethernet NICs.  pccard NICs included.
#device         cs              # Crystal Semiconductor CS89x0 NIC
# 'device ed' requires 'device miibus'
#device         ed              # NE[12]000, SMC Ultra, 3c503, DS8390 cards
#device         ex              # Intel EtherExpress Pro/10 and Pro/10+
#device         ep              # Etherlink III based cards
#device         fe              # Fujitsu MB8696x based cards
#device         sn              # SMC's 9000 series of Ethernet chips
#device         xe              # Xircom pccard Ethernet

# Wireless NIC cards
#device         wlan            # 802.11 support
#options        IEEE80211_DEBUG # enable debug msgs
#options        IEEE80211_AMPDU_AGE # age frames in AMPDU reorder q's
#options        IEEE80211_SUPPORT_MESH  # enable 802.11s draft support
#device         wlan_wep        # 802.11 WEP support
#device         wlan_ccmp       # 802.11 CCMP support
#device         wlan_tkip       # 802.11 TKIP support
#device         wlan_amrr       # AMRR transmit rate control algorithm
#device         an              # Aironet 4500/4800 802.11 wireless NICs.
#device         ath             # Atheros pci/cardbus NIC's
#device         ath_hal         # pci/cardbus chip support
#options        AH_SUPPORT_AR5416       # enable AR5416 tx/rx descriptors
#device         ath_rate_sample # SampleRate tx rate control for ath
#device         ral             # Ralink Technology RT2500 wireless NICs.
#device         wi              # WaveLAN/Intersil/Symbol 802.11 wireless NICs.

# Pseudo devices.
device          loop            # Network loopback
device          random          # Entropy device
device          ether           # Ethernet support
device          vlan            # 802.1Q VLAN support
device          tun             # Packet tunnel.
device          pty             # BSD-style compatibility pseudo ttys
device          md              # Memory "disks"
device          gif             # IPv6 and IPv4 tunneling
device          faith           # IPv6-to-IPv4 relaying (translation)
device          firmware        # firmware assist module

# The `bpf' device enables the Berkeley Packet Filter.
# Be aware of the administrative consequences of enabling this!
# Note that 'bpf' is required for DHCP.
device          bpf             # Berkeley packet filter

# USB support
options         USB_DEBUG       # enable debug msgs
device          uhci            # UHCI PCI->USB interface
device          ohci            # OHCI PCI->USB interface
device          ehci            # EHCI PCI->USB interface (USB 2.0)
device          usb             # USB Bus (required)
#device         udbp            # USB Double Bulk Pipe devices
device          uhid            # "Human Interface Devices"
device          ukbd            # Keyboard
device          ulpt            # Printer
device          umass           # Disks/Mass storage - Requires scbus and da
device          ums             # Mouse
device          urio            # Diamond Rio 500 MP3 player
# USB Serial devices
device          uark            # Technologies ARK3116 based serial adapters
device          ubsa            # Belkin F5U103 and compatible serial adapters
device          uftdi           # For FTDI usb serial adapters
device          uipaq           # Some WinCE based devices
device          uplcom          # Prolific PL-2303 serial adapters
device          uslcom          # SI Labs CP2101/CP2102 serial adapters
device          uvisor          # Visor and Palm devices
device          uvscom          # USB serial support for DDI pocket's PHS
# USB Ethernet, requires miibus
device          aue             # ADMtek USB Ethernet
device          axe             # ASIX Electronics USB Ethernet
device          cdce            # Generic USB over Ethernet
device          cue             # CATC USB Ethernet
device          kue             # Kawasaki LSI USB Ethernet
device          rue             # RealTek RTL8150 USB Ethernet
device          udav            # Davicom DM9601E USB
# USB Wireless
#device         rum             # Ralink Technology RT2501USB wireless NICs
#device         uath            # Atheros AR5523 wireless NICs
#device         ural            # Ralink Technology RT2500USB wireless NICs
#device         zyd             # ZyDAS zb1211/zb1211b wireless NICs

# FireWire support
#device         firewire        # FireWire bus code
#device         sbp             # SCSI over FireWire (Requires scbus and da)
#device         fwe             # Ethernet over FireWire (non-standard!)
#device         fwip            # IP over FireWire (RFC 2734,3146)
#device         dcons           # Dumb console driver
#device         dcons_crom      # Configuration ROM for dcons

/etc/make.conf を編集

CPUTYPE?=nocona
KERNCONF?=PIRIKA

MASTER_SITE_BACKUP?=\
ftp://ftp.yz.yamagata-u.ac.jp/pub/FreeBSD/ports/distfiles/${DIST_SUBDIR}/
MASTER_SITE_OVERRIDE?=  ${MASTER_SITE_BACKUP}

/usr/src/ に移動して

make buildworld
make buildkernel 
make installkernel 
mergemaster -pi
make installworld
mergemaster -i
make delete-old
reboot

途中でエラーになったらやりなおす
uname -a で STABLE になっていることをを確認する

NFS の設定

LANを抜く

/etc/rc.conf を書き換える

hostname="pirika.hokudai.in-cps"
ifconfig_msk0="inet 10.0.1.1 netmask 255.255.0.0"
defaultrouter="10.0.1.0"
keymap="jp.106"
moused_enable="NO"
sshd_enable="YES"
zfs_enable="YES"
nfs_server_enable="YES"
nfs_server_flags="-u -t -n 16"
rpc_lockd_enable="YES"
rpc_statd_enable="YES"
rpcbind_enable="YES"

再起動する
ifconfig -a で ip アドレスが設定されていることを確認
ps axuww で nfsd が起動していることを確認

/system/debian32 を作成してエクスポート

zfs create zfspool/system/debian32
zfs set sharenfs=on zfspool/system/debian32
zfs set sharenfs="-maproot=0 -network 10.0.0.0 -mask 255.255.0.0" zfspool/system/debian32
zfs set sharenfs=on zfspool/home
zfs set sharenfs="-maproot=0 -network 10.0.0.0 -mask 255.255.0.0" zfspool/home

/etc/zfs/exports を確認
cps 裏ネットにつなぐ
cps裏ネットの他のホストに ping してみる
他のホストで mount -t nfs 10.0.1.1:/home /mnt してみる

ファイルサーバの構築

概要

事前準備

ハードウェア

OS インストール

カーネルとツールのコンパイル

NFS の設定