Home COMSC-171 <- Prev Next ->

Storage

drive

block device
block sizes are most commonly 512 or 4096 (4K) bytes but other values are possible
all reads and writes are performed on whole blocks
LBA (logical block addressing) has replaced cylinder-head-sector addressing
Linux device names
SATA, USB, and SCSI drives: sda, sdb, ...
NVMe drives: nvme0n1, nvme1n1, ...(n is namespace)
FreeBSD device names
SATA drives: ada0, ada1, ...
USB and SCSI drives: da0, da1, ...
NVMe drives: nvd0, nvd1, ...

partition

set of consecutive blocks on a drive
GPT (GUID partition table)
64-bit block address (nearly unlimited partition size and number of partitions)
partitions have type, unique ID, attributes, name
MBR (Master Boot Record) partition table
legacy from IBM PC hard disk
32-bit block address (partitions limited to 2TB with 512-byte blocks)
Linux device names
sda1, sda2, nvme0n1p1, nvme0n1p2, ...
Linux tools
gfdisk: GPT partitions, line interface
cgdisk: GPT partitions, curses (full screen) interface
fdisk: MBR partitions, line interface
cfdisk: MBR partitions, curses (full screen) interface
parted: GPT or MBR partitions, subcommands
FreeBSD device names
ada0p1, ada0p2, da0p1, da0p2, nvd0p1, nvd0p2, ...
FreeBSD tools
gpart: GPT or MBR partitions, subcommands
By default FreeBSD puts all filesystems in one partition called the BSD slice

volume

set of drives or partitions or extents (finer divisions)
may be created by a volume manager or an advanced filesystem
each volume has a device name, details vary
Linux tools
lvm and related commands (logical volume manager)
FreeBSD tools
zpool (part of ZFS filesystem)

RAID (redundant array of independent devices)

set of drives or partitions or volumes combined for better performance and/or reliability
levels
RAID0: striped, data spread over multiple drives for speed, 2 or more components
RAID1: mirror, data duplicated over multiple drives for reliability, 2 or more components
RAID5: striped with parity for speed and reliability, 3 or more components
RAID6: striped with dual parity for speed and reliability, 4 or more components
RAID10: nested, RAID0 of RAID1 arrays for speed and reliability, 4 or more components
other levels are less common
Linux device names
md0, md1, ...
Linux tools
mdadm (multiple device administration)
FreeBSD tools
zpool (part of ZFS filesystem)

filesystem

scheme for managing data on a storage device
Linux types
ext2, ext3, ext4, btrfs, xfs (from SGI)
Linux tools
mkfs (filesystem-specific options)
FreeBSD types
UFS, ZFS
FreeBSD tools
bsdlabel, newfs, zfs
Windows types (some UNIX/Linux/BSD support)
FAT, ExFAT, NTFS
optical types
iso9660 (CDROM), UDF (DVD)
network types
NFS, SMB/CIFS (Windows)
distributed types
GlusterFS, CephFS, more

using filesystems

mount
attaches a filesystem to a (usually empty) node on the directory tree
umount
detaches a filesystem from the directory tree
/etc/fstab
file which describes filesystems (device, mount point, type, options, dump frequency, check pass)
fsck
checks for errors and (sometimes) repairs
often a wrapper which runs a filesystem-specific program
startup scripts usually do this periodically and on filesystems which were not unmounted cleanly
zfs
multi-purpose command for ZFS filesystems
other
many tools for tuning, resizing, etc.

filesystem planning

space
a full filesystem (especially /) will cause all sorts of problems, 80% is a reasonable maximum
data will not spill from one filesystem to another so space must be planned
purpose
/ (root): kernel, configuration, core utilities, booting, maintenance, everything not mounted elsewhere
/usr: utilities, applications, libraries, docs, source, etc. (all static)
/var: mail, news, logs, spool files, etc. (all dynamic)
/home: user home directories
/tmp: temporary files (newer systems use a pseudo-filesystem in RAM)
Linux applications sometimes install in /opt, may need a separate filesystem
BSD applications often install in /usr/local, may need a separate filesystem
server data may be in /var or may need a separate filesystem

ZFS

advanced filesystem developed at Sun
open source but CDDL licence not compatible with Linux GPL license
standard on FreeBSD, available for Linux but not merged into Linux kernel
Linux alternatives btrfs and bcachefs are not complete
features
large data (264 byte files, 2128 byte volumes)
copy on write (modified data written to another location) for integrity
checksums on data and metadata for reliability
includes volume management and RAID
flexible allocation of space to datatasets (filesystems) in pools (volumes)
compression
snapshots and replication
caching
deduplication
encryption