notes update && created mock practice exam

2025-03-06 18:40:54 -06:00
parent c29910ad0e
commit 1c0abd53fe
4 changed files with 780 additions and 4 deletions
@@ -14,6 +14,7 @@
 - [Practice Scenarios v4](practice-scenarios/rhcsa_practice_scenarios_v4.md)
 - [Practice Scenarios v5](practice-scenarios/rhcsa_practice_scenarios_v5.md)
 - [Mock RHCSA Exam](practice-scenarios/mock-rhcsa-exam.md)
 - [Mock RHCSA Exam v2](practice-scenarios/mock-rhcsa-exam-v2.md)
 # My Solutions Portfolio
 - [Portfolio Overview](solutions/README.md)
@@ -26,3 +27,4 @@
 - [System Recovery Solutions [🔧]](solutions/system-recovery.md)
 - [Shell Scripting Solutions [📜]](solutions/shell-scripting.md)
 - [Mock Exam Solutions [🎯]](solutions/mock-exam-solutions.md)
 - [Mock Exam v2 Solutions [🎯]](solutions/mock-exam-solutions-v2.md)
@@ -1,26 +1,30 @@
 # RHCSA Practice Scenarios
-This directory contains various practice scenarios and a mock exam to help prepare for the Red Hat Certified System Administrator (RHCSA) exam.
+This directory contains various practice scenarios and mock exams to help prepare for the Red Hat Certified System Administrator (RHCSA) exam.
 ## Available Practice Materials
 - **Practice Scenarios v1-v5**: Incremental practice scenarios covering individual RHCSA objectives
 - **Mock RHCSA Exam**: A comprehensive mock exam that simulates the actual RHCSA exam environment and tasks
 - **Mock RHCSA Exam v2**: An advanced mock exam with more complex tasks covering additional RHCSA objectives
 ## How to Use These Materials
 1. Start with the practice scenarios to build specific skills
-2. Once comfortable with individual components, attempt the mock exam
+2. Once comfortable with individual components, attempt the mock exams
 3. Time yourself to simulate exam conditions (2.5 hours)
 4. Review and practice areas where you struggled
 5. Use the second mock exam to further challenge yourself with more advanced tasks
 ## Key Focus Areas
 - User and group management
- Storage configuration (partitions, LVM, STRATIS)
+- Storage configuration (partitions, LVM, STRATIS, VDO, encryption)
 - Service management and automation
 - Network configuration
- Security (firewall, SELinux)
+- Security (firewall, SELinux, SSH)
 - Container management
 - System boot and troubleshooting
 - Advanced shell scripting
 Remember that the actual RHCSA exam is performance-based, so hands-on practice in a lab environment is essential for success. 
@@ -0,0 +1,156 @@
 # Mock RHCSA (EX200) Practice Exam - Version 2
 ## Introduction
 This is the second version of our mock RHCSA exam, featuring new tasks to further test your Red Hat system administration skills. Like the first version, it covers the key objectives of the official RHCSA exam (EX200).
 **Exam Guidelines:**
 - Duration: 2.5 hours
 - All configurations must persist after system reboot
 - Complete as many tasks as possible - 70% is the passing score
 - You may use Red Hat documentation available on the system
 **Exam Environment:**
 - Red Hat Enterprise Linux 9
 - Virtual machine with 4GB RAM, 40GB disk space
 - Internet access is NOT available
 - Root access is available
 ## Exam Tasks
 ### Task 1: User and Group Management
 1. Create a user named `devops` with UID 3030
 2. Set the password for `devops` to `DevOps2023`
 3. Configure the `devops` account to expire on December 31, 2023
 4. Create a group named `developers` with GID 4040
 5. Create 3 users: `dev1`, `dev2`, and `dev3`, all belonging to the `developers` group as their primary group
 6. Ensure the primary group for all new users created in the system is `developers`
 7. Configure all members of the `developers` group to have a password change required every 60 days
 ### Task 2: File System Management
 1. Create a directory `/projects` owned by `devops` with `developers` as the group owner
 2. Set special permissions on the `/projects` directory so all new files created within it inherit the `developers` group ownership
 3. Configure permissions so that:
   - Members of the `developers` group can create files in the directory
   - Only file owners can delete their own files (using the sticky bit)
   - Users outside the `developers` group cannot access the directory
 4. Create a file `/projects/readme.md` with the content "Development Projects Directory"
 5. Set ACLs on the `/projects` directory to give user `dev1` full control, while `dev2` and `dev3` have read and execute permissions only
 ### Task 3: LVM Storage Configuration
 1. Create a 2GB partition on an available disk
 2. Create a volume group named `vg_projects` using this partition
 3. Create two logical volumes in the `vg_projects` volume group:
   - `lv_data` (800MB)
   - `lv_backup` (400MB)
 4. Format `lv_data` with XFS filesystem and `lv_backup` with ext4 filesystem
 5. Mount `lv_data` at `/projects/data` and `lv_backup` at `/projects/backup` permanently
 6. Configure the system to automatically snapshot `lv_data` daily (hint: you'll need to leave free space in the volume group)
 ### Task 4: Storage with VDO and Stratis
 1. Install the necessary packages for VDO (Virtual Data Optimizer)
 2. Create a 1GB partition on an available disk
 3. Configure VDO on this partition with a logical size of 3GB
 4. Format the VDO volume with XFS and mount it at `/vdo` permanently
 5. Install the Stratis storage management tools
 6. Create a Stratis pool named `stratis_pool` using another 1GB partition
 7. Create a Stratis filesystem named `stratis_fs` and mount it at `/stratis` permanently
 ### Task 5: Advanced Container Management
 1. Install the container tools (podman)
 2. Configure a persistent container storage location at `/container_storage`
 3. Pull the MariaDB container image
 4. Run a MariaDB container named `db_server` with:
   - The container accessible on port 3306
   - A root password of `dbpassword`
   - A persistent volume at `/container_storage/mysql_data` mounted to the container's `/var/lib/mysql`
   - Environment variables set for a database named `webapp` and a database user `webuser` with password `webpass`
 5. Configure the container to start automatically at system boot using systemd
 6. Create a simple bash script `/usr/local/bin/db_backup.sh` that creates a backup of the MariaDB container data
 ### Task 6: Automating System Tasks
 1. Create a systemd timer that runs a filesystem usage check every 4 hours and logs the results to `/var/log/disk_usage.log`
 2. Configure a cron job for the `devops` user to run every Monday at 2:30 AM that:
   - Archives files older than 30 days in `/projects/data` 
   - Moves the archives to `/projects/backup`
 3. Configure a systemd service called `project-backup` that:
   - Creates a tar archive of the `/projects` directory
   - Runs automatically at system shutdown
   - Places the backup in `/var/backups` with a timestamp in the filename
 ### Task 7: System Boot Configuration
 1. Configure the system to boot into emergency mode with a 5-second timeout
 2. Create a new GRUB menu entry that boots the system with the kernel parameter `enforcing=0`
 3. Set the default boot target to multi-user (non-graphical)
 4. Create a custom systemd target called `developer-mode.target` that:
   - Starts all the services of multi-user target
   - Additionally starts the MariaDB container
   - Mounts all project-related filesystems
 ### Task 8: Network Configuration and Services
 1. Configure the primary network interface with:
   - Static IP: 192.168.10.200/24
   - Gateway: 192.168.10.1
   - DNS servers: 192.168.10.1 and 8.8.8.8
 2. Configure the hostname as `rhcsa-server`
 3. Add a second IP address (192.168.10.201/24) to the same interface
 4. Install and configure an NFS server that:
   - Exports `/projects/data` as read-write to 192.168.10.0/24
   - Exports `/projects/backup` as read-only to 192.168.10.0/24
 5. Configure the firewall to allow NFS and MariaDB traffic from the local network only
 ### Task 9: Security Configuration
 1. Configure SELinux contexts so that:
   - The web server can access files in `/projects/data`
   - The MariaDB container can access `/container_storage/mysql_data`
 2. Configure SSH to:
   - Allow access only for the `devops` user
   - Disable password authentication (key-based only)
   - Listen on an alternate port (2222)
 3. Set up key-based authentication for the `devops` user
 4. Configure the firewall to allow SSH on the new port
 5. Create a sudo configuration that allows `devops` to run commands as root without a password, but logs all commands
 ### Task 10: System Analysis and Troubleshooting
 1. Configure system logging to:
   - Keep logs for 90 days
   - Forward system and audit logs to the `rsyslog` service
   - Create separate log files for container operations
 2. Create a script `/usr/local/bin/system_health.sh` that:
   - Checks system load average
   - Monitors memory usage
   - Verifies connectivity to the gateway
   - Reports if any key services are down
   - Emails the report to `root@localhost`
 3. Create a systemd service and timer to run this health check script every hour
 ### Task 11: Advanced Shell Scripting
 1. Create a script `/usr/local/bin/user_report.sh` that:
   - Finds all users in the `developers` group
   - Reports their last login time
   - Lists all files owned by each user in the `/projects` directory
   - Sorts the output by file size
   - Saves a formatted report to `/var/reports/user_report.txt`
 2. Make the script executable and set up a weekly cron job to run it
 3. Create a directory `/var/reports` with appropriate permissions for storing reports
 ### Task 12: Advanced FileSystem Management
 1. Create a 500MB file using `dd` to serve as a loopback device
 2. Configure this file as a loopback device
 3. Create a LUKS encrypted container on the loopback device
 4. Format the encrypted container with XFS
 5. Configure the system to automatically unlock the encrypted container at boot using a key file
 6. Mount the encrypted filesystem at `/projects/secure` permanently
 ## Scoring Guide
 Each task has approximately equal weight. To pass this mock exam, you need to successfully complete 70% of the tasks.
 After completing the exam, test your configurations by:
 1. Rebooting the system to verify persistence
 2. Logging in as the created users to verify access
 3. Testing that services start correctly
 4. Verifying container operation
 5. Checking encrypted storage and mounts
 Good luck with your practice exam! 
@@ -0,0 +1,614 @@
 # Mock RHCSA Exam v2 - Solutions
 This document provides solutions for the [Mock RHCSA Exam - Version 2](../practice-scenarios/mock-rhcsa-exam-v2.md). Use these solutions to check your work after attempting the exam yourself.
 ## Task 1: User and Group Management
 ```bash
 # Create user with specific UID
 sudo useradd -u 3030 devops
 # Set password
 sudo passwd devops
 # Enter DevOps2023 when prompted
 # Set account expiration date
 sudo chage -E 2023-12-31 devops
 # Create group with specific GID
 sudo groupadd -g 4040 developers
 # Create users with developers as primary group
 sudo useradd -g developers dev1
 sudo useradd -g developers dev2
 sudo useradd -g developers dev3
 # Set default group for new users
 sudo sed -i 's/^GROUP=.*/GROUP=developers/' /etc/default/useradd
 # Configure password expiration for developers group members
 for user in $(grep "developers" /etc/group | cut -d: -f4 | tr ',' ' '); do
  sudo chage -M 60 $user
 done
 ```
 ## Task 2: File System Management
 ```bash
 # Create directory with proper ownership
 sudo mkdir -p /projects
 sudo chown devops:developers /projects
 # Set SGID bit for group inheritance
 sudo chmod 2775 /projects
 # Set sticky bit and appropriate permissions
 # 2 = SGID, 7 = rwx for owner, 7 = rwx for group, 5 = r-x for others
 # t (sticky bit) makes it so only owners can delete their files
 sudo chmod 2775 /projects
 sudo chmod +t /projects
 # Create file with content
 echo "Development Projects Directory" | sudo tee /projects/readme.md
 # Set ACLs
 sudo setfacl -m u:dev1:rwx /projects
 sudo setfacl -m u:dev2:r-x /projects
 sudo setfacl -m u:dev3:r-x /projects
 ```
 ## Task 3: LVM Storage Configuration
 ```bash
 # Check available disks
 lsblk
 # Create partition (assuming /dev/sdb is available)
 sudo fdisk /dev/sdb
 # n (new), p (primary), 1 (partition number), enter (default), +2G, w (write)
 # Create volume group
 sudo vgcreate vg_projects /dev/sdb1
 # Create logical volumes
 sudo lvcreate -L 800M -n lv_data vg_projects
 sudo lvcreate -L 400M -n lv_backup vg_projects
 # Format filesystems
 sudo mkfs.xfs /dev/vg_projects/lv_data
 sudo mkfs.ext4 /dev/vg_projects/lv_backup
 # Create mount points
 sudo mkdir -p /projects/data /projects/backup
 # Add to fstab for persistence
 echo "/dev/vg_projects/lv_data   /projects/data   xfs   defaults   0 0" | sudo tee -a /etc/fstab
 echo "/dev/vg_projects/lv_backup /projects/backup ext4  defaults   0 0" | sudo tee -a /etc/fstab
 # Mount filesystems
 sudo mount -a
 # Configure daily snapshots
 # Create a script for creating and managing snapshots
 cat << 'EOF' | sudo tee /usr/local/bin/lvm_snapshot.sh
 #!/bin/bash
 # Create a snapshot with date in the name
 DATE=$(date +%Y%m%d)
 # Remove old snapshots
 sudo lvremove -f /dev/vg_projects/lv_data_snap_* 2>/dev/null
 # Create new snapshot (100MB size)
 sudo lvcreate -L 100M -s -n lv_data_snap_$DATE /dev/vg_projects/lv_data
 EOF
 # Make script executable
 sudo chmod +x /usr/local/bin/lvm_snapshot.sh
 # Create cron job for daily snapshots
 echo "0 1 * * * root /usr/local/bin/lvm_snapshot.sh" | sudo tee /etc/cron.d/lvm-snapshots
 ```
 ## Task 4: Storage with VDO and Stratis
 ```bash
 # Install VDO packages
 sudo dnf install -y vdo kmod-kvdo
 # Create a partition (assuming /dev/sdc is available)
 sudo fdisk /dev/sdc
 # n (new), p (primary), 1 (partition number), enter (default), +1G, w (write)
 # Create VDO volume
 sudo vdo create --name=vdo_vol1 --device=/dev/sdc1 --vdoLogicalSize=3G
 # Format VDO volume with XFS
 sudo mkfs.xfs -K /dev/mapper/vdo_vol1
 # Create mount point
 sudo mkdir -p /vdo
 # Add to fstab for persistence
 echo "/dev/mapper/vdo_vol1 /vdo xfs defaults,x-systemd.requires=vdo.service 0 0" | sudo tee -a /etc/fstab
 # Mount VDO
 sudo mount -a
 # Install Stratis
 sudo dnf install -y stratisd stratis-cli
 # Create a partition (assuming /dev/sdd is available)
 sudo fdisk /dev/sdd
 # n (new), p (primary), 1 (partition number), enter (default), +1G, w (write)
 # Start Stratis service
 sudo systemctl enable --now stratisd
 # Create Stratis pool
 sudo stratis pool create stratis_pool /dev/sdd1
 # Create Stratis filesystem
 sudo stratis filesystem create stratis_pool stratis_fs
 # Create mount point
 sudo mkdir -p /stratis
 # Add to fstab for persistence
 # Get UUID for the stratis filesystem
 STRATIS_UUID=$(sudo stratis filesystem list | grep stratis_fs | awk '{print $4}')
 echo "UUID=$STRATIS_UUID /stratis xfs defaults,x-systemd.requires=stratisd.service 0 0" | sudo tee -a /etc/fstab
 # Mount Stratis
 sudo mount -a
 ```
 ## Task 5: Advanced Container Management
 ```bash
 # Install container tools
 sudo dnf install -y podman
 # Create persistent storage location
 sudo mkdir -p /container_storage/mysql_data
 sudo chown -R 27:27 /container_storage/mysql_data  # MySQL standard user/group ID
 # Pull MariaDB image
 sudo podman pull mariadb:latest
 # Run MariaDB container
 sudo podman run -d --name db_server \
  -p 3306:3306 \
  -v /container_storage/mysql_data:/var/lib/mysql:Z \
  -e MYSQL_ROOT_PASSWORD=dbpassword \
  -e MYSQL_DATABASE=webapp \
  -e MYSQL_USER=webuser \
  -e MYSQL_PASSWORD=webpass \
  mariadb:latest
 # Create systemd service for auto-start
 sudo mkdir -p /etc/systemd/system
 sudo podman generate systemd --name db_server --files --new
 # Copy and enable the service
 sudo cp container-db_server.service /etc/systemd/system/
 sudo systemctl enable container-db_server.service
 # Create backup script
 cat << 'EOF' | sudo tee /usr/local/bin/db_backup.sh
 #!/bin/bash
 DATE=$(date +%Y%m%d)
 BACKUP_DIR="/container_storage/backups"
 mkdir -p $BACKUP_DIR
 sudo podman exec db_server mysqldump -u root -pdbpassword --all-databases > $BACKUP_DIR/mysql_all_$DATE.sql
 find $BACKUP_DIR -name "mysql_all_*.sql" -mtime +7 -delete
 EOF
 # Make script executable
 sudo chmod +x /usr/local/bin/db_backup.sh
 ```
 ## Task 6: Automating System Tasks
 ```bash
 # Create filesystem usage check script
 cat << 'EOF' | sudo tee /usr/local/bin/disk_check.sh
 #!/bin/bash
 DATE=$(date +"%Y-%m-%d %H:%M:%S")
 echo "Filesystem usage report - $DATE" > /var/log/disk_usage.log
 df -h >> /var/log/disk_usage.log
 echo "----------------------------" >> /var/log/disk_usage.log
 EOF
 # Make script executable
 sudo chmod +x /usr/local/bin/disk_check.sh
 # Create systemd service
 cat << 'EOF' | sudo tee /etc/systemd/system/disk-check.service
 [Unit]
 Description=Check disk usage
 [Service]
 Type=oneshot
 ExecStart=/usr/local/bin/disk_check.sh
 EOF
 # Create systemd timer
 cat << 'EOF' | sudo tee /etc/systemd/system/disk-check.timer
 [Unit]
 Description=Run disk usage check every 4 hours
 [Timer]
 OnBootSec=10min
 OnUnitActiveSec=4h
 [Install]
 WantedBy=timers.target
 EOF
 # Enable and start timer
 sudo systemctl enable --now disk-check.timer
 # Create archive script for cron job
 cat << 'EOF' | sudo tee /usr/local/bin/archive_old_files.sh
 #!/bin/bash
 DATE=$(date +%Y%m%d)
 find /projects/data -type f -mtime +30 -print0 | xargs -0 tar czf /projects/backup/old_files_$DATE.tar.gz 2>/dev/null
 find /projects/data -type f -mtime +30 -delete
 EOF
 # Make script executable
 sudo chmod +x /usr/local/bin/archive_old_files.sh
 # Create cron job for devops user
 sudo -u devops crontab -l > /tmp/devops-crontab
 echo "30 2 * * 1 /usr/local/bin/archive_old_files.sh" >> /tmp/devops-crontab
 sudo -u devops crontab /tmp/devops-crontab
 rm /tmp/devops-crontab
 # Create project backup script
 cat << 'EOF' | sudo tee /usr/local/bin/backup-projects.sh
 #!/bin/bash
 DATE=$(date +%Y%m%d-%H%M%S)
 mkdir -p /var/backups
 tar czf /var/backups/projects-$DATE.tar.gz /projects
 EOF
 # Make script executable
 sudo chmod +x /usr/local/bin/backup-projects.sh
 # Create systemd service for shutdown backup
 cat << 'EOF' | sudo tee /etc/systemd/system/project-backup.service
 [Unit]
 Description=Backup projects directory
 DefaultDependencies=no
 Before=shutdown.target reboot.target halt.target
 [Service]
 Type=oneshot
 ExecStart=/usr/local/bin/backup-projects.sh
 TimeoutStartSec=0
 [Install]
 WantedBy=shutdown.target reboot.target halt.target
 EOF
 # Enable the service
 sudo systemctl enable project-backup.service
 ```
 ## Task 7: System Boot Configuration
 ```bash
 # Edit grub configuration
 sudo cp /etc/default/grub /etc/default/grub.backup
 sudo sed -i 's/GRUB_TIMEOUT=.*/GRUB_TIMEOUT=5/' /etc/default/grub
 sudo sed -i 's/GRUB_CMDLINE_LINUX=.*/GRUB_CMDLINE_LINUX="rd.break"/' /etc/default/grub
 # Create custom menu entry
 cat << 'EOF' | sudo tee /etc/grub.d/40_custom
 #!/bin/sh
 exec tail -n +3 $0
 # This file provides an easy way to add custom menu entries.
 menuentry 'RHEL 9 with SELinux Permissive' {
    linux /boot/vmlinuz-$(uname -r) root=/dev/mapper/rhel-root ro enforcing=0
    initrd /boot/initramfs-$(uname -r).img
 }
 EOF
 # Update grub configuration
 sudo chmod +x /etc/grub.d/40_custom
 sudo grub2-mkconfig -o /boot/grub2/grub.cfg
 # Set default target
 sudo systemctl set-default multi-user.target
 # Create custom systemd target
 cat << 'EOF' | sudo tee /etc/systemd/system/developer-mode.target
 [Unit]
 Description=Developer Mode
 Requires=multi-user.target container-db_server.service
 After=multi-user.target container-db_server.service
 Wants=container-db_server.service
 [Install]
 WantedBy=multi-user.target
 EOF
 # Create symlinks for filesystem mounts
 sudo ln -sf /usr/lib/systemd/system/local-fs.target /etc/systemd/system/developer-mode.target.wants/
 ```
 ## Task 8: Network Configuration and Services
 ```bash
 # Configure network interface (assuming eth0 is the interface name)
 sudo nmcli connection add con-name static-eth0 ifname eth0 type ethernet ip4 192.168.10.200/24 gw4 192.168.10.1
 sudo nmcli connection modify static-eth0 ipv4.dns "192.168.10.1 8.8.8.8"
 sudo nmcli connection up static-eth0
 # Add secondary IP
 sudo nmcli connection modify static-eth0 +ipv4.addresses 192.168.10.201/24
 sudo nmcli connection up static-eth0
 # Set hostname
 sudo hostnamectl set-hostname rhcsa-server
 # Install NFS server
 sudo dnf install -y nfs-utils
 # Create exports file entries
 echo "/projects/data 192.168.10.0/24(rw,sync,no_root_squash)" | sudo tee -a /etc/exports
 echo "/projects/backup 192.168.10.0/24(ro,sync)" | sudo tee -a /etc/exports
 # Start and enable NFS server
 sudo systemctl enable --now nfs-server
 # Export the shares
 sudo exportfs -ra
 # Configure firewall
 sudo firewall-cmd --permanent --add-service=nfs
 sudo firewall-cmd --permanent --add-service=rpc-bind
 sudo firewall-cmd --permanent --add-service=mountd
 sudo firewall-cmd --permanent --add-service=mysql
 sudo firewall-cmd --permanent --add-source=192.168.10.0/24
 sudo firewall-cmd --reload
 ```
 ## Task 9: Security Configuration
 ```bash
 # Configure SELinux context for web server access
 sudo semanage fcontext -a -t httpd_sys_content_t "/projects/data(/.*)?"
 sudo restorecon -Rv /projects/data
 # Configure SELinux context for container storage
 sudo semanage fcontext -a -t container_file_t "/container_storage/mysql_data(/.*)?"
 sudo restorecon -Rv /container_storage/mysql_data
 # Configure SSH
 sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.backup
 sudo sed -i 's/#Port 22/Port 2222/' /etc/ssh/sshd_config
 sudo sed -i 's/#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config
 sudo sed -i 's/#PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config
 # Create AllowUsers directive
 echo "AllowUsers devops" | sudo tee -a /etc/ssh/sshd_config
 # Set up key-based authentication for devops
 sudo -u devops mkdir -p /home/devops/.ssh
 sudo -u devops chmod 700 /home/devops/.ssh
 sudo -u devops ssh-keygen -t rsa -f /home/devops/.ssh/id_rsa -N ""
 sudo -u devops cat /home/devops/.ssh/id_rsa.pub >> /home/devops/.ssh/authorized_keys
 sudo -u devops chmod 600 /home/devops/.ssh/authorized_keys
 # Configure firewall for SSH
 sudo firewall-cmd --permanent --remove-service=ssh
 sudo firewall-cmd --permanent --add-port=2222/tcp
 sudo firewall-cmd --reload
 # Create sudo configuration for devops
 echo "devops ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/devops
 echo "Defaults:devops logfile=/var/log/devops_sudo.log" | sudo tee -a /etc/sudoers.d/devops
 # Restart SSH service
 sudo systemctl restart sshd
 ```
 ## Task 10: System Analysis and Troubleshooting
 ```bash
 # Configure system logging
 sudo cp /etc/rsyslog.conf /etc/rsyslog.conf.backup
 # Set log rotation to 90 days
 sudo sed -i 's/^\$FileOwner.*/$FileOwner root\n$MaxFileDays 90/' /etc/rsyslog.conf
 # Create separate log file for containers
 cat << 'EOF' | sudo tee /etc/rsyslog.d/container.conf
 if $programname contains 'podman' or $programname contains 'container' then /var/log/container.log
 & stop
 EOF
 # Restart rsyslog
 sudo systemctl restart rsyslog
 # Create system health check script
 cat << 'EOF' | sudo tee /usr/local/bin/system_health.sh
 #!/bin/bash
 DATE=$(date +"%Y-%m-%d %H:%M:%S")
 REPORT_FILE="/tmp/system_health_$$.tmp"
 echo "System Health Report - $DATE" > $REPORT_FILE
 echo "============================" >> $REPORT_FILE
 # Load average
 echo "" >> $REPORT_FILE
 echo "System Load:" >> $REPORT_FILE
 uptime | awk '{print $10 $11 $12}' >> $REPORT_FILE
 # Memory usage
 echo "" >> $REPORT_FILE
 echo "Memory Usage:" >> $REPORT_FILE
 free -h >> $REPORT_FILE
 # Disk usage
 echo "" >> $REPORT_FILE
 echo "Disk Usage:" >> $REPORT_FILE
 df -h >> $REPORT_FILE
 # Network connectivity
 echo "" >> $REPORT_FILE
 echo "Network Connectivity:" >> $REPORT_FILE
 ping -c 1 192.168.10.1 >> $REPORT_FILE 2>&1
 # Service status
 echo "" >> $REPORT_FILE
 echo "Service Status:" >> $REPORT_FILE
 systemctl status nfs-server container-db_server | grep Active >> $REPORT_FILE
 # Send email
 mail -s "System Health Report" root@localhost < $REPORT_FILE
 rm $REPORT_FILE
 EOF
 # Make script executable
 sudo chmod +x /usr/local/bin/system_health.sh
 # Create systemd service
 cat << 'EOF' | sudo tee /etc/systemd/system/system-health.service
 [Unit]
 Description=System Health Check
 [Service]
 Type=oneshot
 ExecStart=/usr/local/bin/system_health.sh
 EOF
 # Create systemd timer
 cat << 'EOF' | sudo tee /etc/systemd/system/system-health.timer
 [Unit]
 Description=Run system health check hourly
 [Timer]
 OnBootSec=5min
 OnUnitActiveSec=1h
 [Install]
 WantedBy=timers.target
 EOF
 # Enable and start timer
 sudo systemctl enable --now system-health.timer
 ```
 ## Task 11: Advanced Shell Scripting
 ```bash
 # Create directory for reports
 sudo mkdir -p /var/reports
 sudo chmod 755 /var/reports
 # Create user report script
 cat << 'EOF' | sudo tee /usr/local/bin/user_report.sh
 #!/bin/bash
 REPORT_FILE="/var/reports/user_report.txt"
 DATE=$(date +"%Y-%m-%d %H:%M:%S")
 echo "User Activity Report - $DATE" > $REPORT_FILE
 echo "===============================" >> $REPORT_FILE
 # Get all users in the developers group
 USERS=$(grep "^developers" /etc/group | cut -d: -f4 | tr ',' ' ')
 for user in $USERS; do
  echo "" >> $REPORT_FILE
  echo "User: $user" >> $REPORT_FILE
  echo "----------------" >> $REPORT_FILE
  # Last login time
  echo "Last Login:" >> $REPORT_FILE
  lastlog -u $user | tail -1 >> $REPORT_FILE
  # Files owned by user
  echo "" >> $REPORT_FILE
  echo "Files in /projects owned by $user (sorted by size):" >> $REPORT_FILE
  find /projects -user $user -type f -exec ls -lh {} \; | sort -k5 -h >> $REPORT_FILE
 done
 EOF
 # Make script executable
 sudo chmod +x /usr/local/bin/user_report.sh
 # Create weekly cron job
 echo "0 0 * * 0 root /usr/local/bin/user_report.sh" | sudo tee /etc/cron.d/user-report
 ```
 ## Task 12: Advanced FileSystem Management
 ```bash
 # Create file for loopback
 sudo dd if=/dev/zero of=/root/secure.img bs=1M count=500
 # Configure as loopback device
 sudo losetup -f /root/secure.img
 LOOP_DEV=$(sudo losetup -a | grep secure.img | cut -d: -f1)
 # Create key file for encryption
 sudo dd if=/dev/urandom of=/root/luks-key bs=512 count=4
 sudo chmod 400 /root/luks-key
 # Create LUKS encrypted container
 sudo cryptsetup luksFormat --batch-mode --key-file=/root/luks-key $LOOP_DEV
 # Open LUKS container
 sudo cryptsetup luksOpen --key-file=/root/luks-key $LOOP_DEV secure-volume
 # Format with XFS
 sudo mkfs.xfs /dev/mapper/secure-volume
 # Create mount point
 sudo mkdir -p /projects/secure
 # Configure auto-unlock in crypttab
 echo "secure-volume $LOOP_DEV /root/luks-key luks" | sudo tee -a /etc/crypttab
 # Add to fstab for persistence
 echo "/dev/mapper/secure-volume /projects/secure xfs defaults,x-systemd.requires=cryptsetup.target 0 0" | sudo tee -a /etc/fstab
 # Mount
 sudo mount -a
 ```
 ## Verification Steps
 ```bash
 # Reboot to verify persistence
 sudo reboot
 # Test user login
 su - devops
 su - dev1
 su - dev2
 su - dev3
 # Test services
 systemctl status nfs-server
 systemctl status container-db_server
 systemctl status stratisd
 systemctl status vdo
 # Test file systems
 df -h
 ls -la /projects
 ls -la /projects/data
 ls -la /projects/backup
 ls -la /projects/secure
 ls -la /vdo
 ls -la /stratis
 # Test container
 sudo podman ps
 mysql -h 127.0.0.1 -P 3306 -u webuser -pwebpass
 # Test encryption
 sudo cryptsetup status secure-volume
 ```