Deploying a local apt source on Raspberry Pi

Preface#

The Raspberry Pi officially joins the internal network environment, preparing to deploy the apt source to provide source repository services for other Debian-based devices or virtual machines in the internal network environment, such as Proxmox. If you want to simulate an enterprise data center environment, it is necessary to completely isolate the internal and external networks.

Understanding Local Sources#

Local sources refer to the sources of software packages. Local source devices provide source services for devices that cannot connect to the internet in the internal network environment. Why can't they connect to the internet? In data centers or other internal network environments, some devices cannot connect to the internet for security reasons; it's not that they cannot, but that they are not allowed to connect, with a focus on security. Therefore, local sources become the only access point for devices in the internal network environment to update software packages. Thus, it is necessary to download the sources from the internet to the device storing the local source. For example, I installed Proxmox on my server, which is a virtual machine system based on the Debian operating system, and it is in the internal network environment. In the future, it will serve as the main system for internal network services, virtualizing multiple devices to provide different services. However, after installation, it has various software missing issues and cannot connect to the internet for installation, so a local source device is needed to meet its software installation needs.

How to Deploy Local Sources#

First, there are tutorials online that can be referenced for deployment issues. The Raspberry Pi system is installed on a TF card, which is generally 32G when purchased. With this capacity, let alone deploying a local source, just putting a few system image packages on it will fill it up. Therefore, we need to add a large-capacity backpack to the Raspberry Pi so that it can take on this task. Before deploying the local source, we need to mount an external hard drive to the Raspberry Pi. I happen to have a 512G solid-state drive that can be used as its external disk.

# Find the disk
root@raspberrypi:/# fdisk -l
# Results omitted
Disk identifier: 0x6aef5cb3

Device         Boot  Start      End  Sectors  Size Id Type
/dev/mmcblk0p1        8192   532479   524288  256M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      532480 62333951 61801472 29.5G 83 Linux

Disk /dev/sda: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
Disk model: Generic
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt

Disk identifier: 8C903F36-23B9-44CB-99B8-D122B713E560Device     Start        End    Sectors   Size Type
/dev/sda1   2048 1000215182 1000213135 476.9G Linux filesystem

I found a disk sda, which was previously used with the Linux file system. First, clear the data and reformat it.

root@raspberrypi:/# mkfs -t ext3 /dev/sda
mke2fs 1.46.2 (28-Feb-2021)
Creating filesystem with 125026902 4k blocks and 31260672 inodes
Filesystem UUID: 2898dbe1-7892-4424-9fee-ff8b78b8b32e
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000

Allocating group tables: done
Writing inode tables: done
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done

Create a mount directory and mount the disk.

root@raspberrypi:/# mkdir /opt/sda1
root@raspberrypi:/# chmod 777 /opt/sda1
root@raspberrypi:/# mount /dev/sda /opt/sda1

Query the disk's UUID.

root@raspberrypi:/# blkid
/dev/mmcblk0p1: LABEL_FATBOOT="boot" LABEL="boot" UUID="0F92-BECC" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="6aef5cb3-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="41c98998-6a08-4389-bf74-79c9efcf0739" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="6aef5cb3-02"
/dev/sda: UUID="2898dbe1-7892-4424-9fee-ff8b78b8b32e" BLOCK_SIZE="4096" TYPE="ext3"

Set the disk to mount at boot, as it would be troublesome if it fails to mount after a reboot and you don't know where the problem is.

root@raspberrypi:/etc# vim fstab
# Add the following content to the last line of fstab, with the UUID being the sda information queried in the previous step
UUID=2898dbe1-7892-4424-9fee-ff8b78b8b32e /opt/sda1 ext3 defaults        0       0

The disk is mounted, and we can start deploying the apt local source. Generally, there are two ways to set up a local source:

On-demand pulling, where internal network devices install packages through remote copying as needed. References 1, 2, and 3 in the materials are all this method.
Directly deploying a local repository, pulling the majority of software packages. Reference 4 in the materials is this method. Since we have prepared a large-capacity hard drive, we are definitely considering the latter. Moreover, since it mainly serves the Proxmox system based on Debian, we need to replace the source target during the deployment process.

root@raspberrypi:/# apt install apt-mirror
root@raspberrypi:/# cd /etc/apt
root@raspberrypi:/etc/apt# vim mirror.list
# Modify the mirror.list file
# After modification, execute apt-mirror
root@raspberrypi:/# apt-mirror

The content of the file is as follows, changing the original official US source to the Chinese source.

############# config ##################
#
set base_path    /opt/sda1/pve_mirror
#
#set mirror_path   $base_path/mirror
# set skel_path    $base_path/skel
# set var_path     $base_path/var
# set cleanscript $var_path/clean.sh
# set defaultarch  <running host architecture>
# set postmirror_script $var_path/postmirror.sh
# set run_postmirror 0
set nthreads     20
set _tilde 0
#
############# end config ##############

deb http://ftp.cn.debian.org/debian unstable main contrib non-free
deb-src http://ftp.cn.debian.org/debian unstable main contrib non-free

# mirror additional architectures
#deb-alpha http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-amd64 http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-armel http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-hppa http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-i386 http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-ia64 http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-m68k http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-mips http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-mipsel http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-powerpc http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-s390 http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-sparc http://ftp.us.debian.org/debian unstable main contrib non-free

clean http://ftp.us.debian.org/debian

After executing apt-mirror, it will take a very long time. The first time I thought it failed and terminated it with ctrl+C. Upon executing it again, I found that the originally required 102G had become 92G, indicating that although it was slow, it was indeed downloading.

After spending a day, it finally finished downloading. Next, we need to deploy an Apache proxy for internal network devices to access.

root@raspberrypi:/# apt-get install apache2
root@raspberrypi:/# ln -s /opt/sda1/pve_mirror/mirror/ftp.cn.debian.org/debian /var/www/html/proxmox

Once the symbolic link is created, the software source on this Raspberry Pi can be accessed through a browser. The access address is http://ip/pve.

Of course, there are other methods. If configuring apache2 or nginx doesn't work, there are other ways. Installing Node.js and using an open-source file service system to set up links is also feasible, such as hope-server on GitHub.

root@raspberrypi:/# apt install nodejs
root@raspberrypi:/# apt install npm
root@raspberrypi:/# apt install yarn
root@raspberrypi:/# yarn global install hope-server
root@raspberrypi:/# hope-server -d <filepath> -p 8080 -o localhost
# filepath is the software source directory, which can be the disk mount path or the soft link path after mounting the disk.
# However, this method has certain drawbacks; when starting hope-server, it will occupy the console, so you can use nohup + crontab.
root@raspberrypi:~# nohup hope-server -d /var/www/proxmox/ -p 8080 -o localhost &
[3] 6843
---------------------------------------------------
root@raspberrypi:~# crontab -e
no crontab for root - using an empty one

Select an editor.  To change later, run 'select-editor'.
  1. /bin/nano        <---- easiest
  2. /usr/bin/vim.basic
  3. /usr/bin/vim.tiny
  4. /bin/ed

Choose 1-4 [1]: 2
# Add the following line at the end of the editor
@reboot nohup hope-server -d /var/www/proxmox/ -p 8080 -o localhost &
# This way, it will execute every time it restarts.

How to Modify the PVE Source#

After deploying the local source, the next step is to modify the source address on the client, which is the PVE system in the internal network, located at /etc/apt/sources.list.

root@proxmox:/# nano /etc/apt/sources.list
# Add source addresses similar to the following in the editor
deb [trusted=yes] http://172.20.0.2:80/proxmox/mirror/mirrors.tuna.tsinghua.edu.cn/debian bullseye main contrib

deb [trusted=yes] http://172.20.0.2:80/proxmox/mirror/mirrors.tuna.tsinghua.edu.cn/debian bullseye-updates main contrib

deb [trusted=yes] http://172.20.0.2:80/proxmox/mirror/mirrors.tuna.tsinghua.edu.cn/debian bullseye-backports main contrib

deb [trusted=yes] http://172.20.0.2:80/proxmox/mirror/mirrors.tuna.tsinghua.edu.cn/debian bullseye-security main contrib

Although it seems successful up to this point, it still reported an error 😅.

Reading package lists... Done
E: Release file for http://172.20.0.2:80/proxmox/mirror/mirrors.tuna.tsinghua.edu.cn/debian/dists/bullseye/InRelease is not valid yet (invalid for another 33d 17h 53min 33s). Updates for this repository will not be applied.
E: Release file for http://172.20.0.2:80/proxmox/mirror/mirrors.tuna.tsinghua.edu.cn/debian/dists/bullseye-updates/InRelease is not valid yet (invalid for another 130d 21h 52min 40s). Updates for this repository will not be applied.
E: Release file for http://172.20.0.2:80/proxmox/mirror/mirrors.tuna.tsinghua.edu.cn/debian/dists/bullseye-backports/InRelease is not valid yet (invalid for another 130d 21h 52min 40s). Updates for this repository will not be applied.

It seems to be a time issue, probably due to selecting the wrong timezone when installing the PVE system? Let's check the current time of the PVE system.

root@proxmox:~# date
Mon 08 Aug 2022 12:29:02 AM CST

Modify it.

root@proxmox:/etc# vim /etc/systemd/timesyncd.conf
# Set the NTP address to the soft router
root@proxmox:/etc# systemctl restart systemd-timesyncd.service
root@proxmox:/etc# systemctl enable systemd-timesyncd.service
root@proxmox:/etc# date
Sat 17 Dec 2022 07:21:07 AM CST

Finally, apt updates normally.

Aki