pvr allow you to create squash file systems used by pantavisor as a read-only file system for the running containers, but sometimes if you have a big container, as could be the case for a full fetch ubuntu container, the resulting squashfs could be 900mb+, and therefore every time you deploy a new revision of your container, even though you only change 1 file inside the container, the latest update will download the whole new squashfs, that means will use more storage on your pantacor hub account, will be slower because it needs to download a bigger object and will use more disk on the board to maintain a previous version to rollback.
That is why we have a new feature in our development branch of pvr, that allows you two this:
- Create patch updates to your containers: That means it will create overlay squashfs that only have the files that change against the base squashfs in the container.
- Create an application container as an extension of another application: This will create a new application that only has the overlay squashfs with the diff between the base container and the new one. This will be useful if you have several containers that have the same base dependencies like libraries, binaries, and more. but they differ in a couple of configuration files or new packages installed.
To test this new feature you will need to have pvr installed (how to install pvr) and you will need to switch to the development branch and do a self-upgrade of your pvr cli.
pvr global-config set DistributionTag=develop pvr self-upgrade
This new feature is early and easy to use you only need to add
--patch argument when you run the
app update command.
pvr app update --patch mycroft
To see the difference that can make only sending the overlay patch instead of the whole squashfs less take the mycroft example.
Mycroft is an Open Source project for a speaker assistant called Mark II. The Mark II is a state-of-the-art, privacy-respecting smart speaker designed for you and your family, from kids to grandparents. The Mark II provides a great voice experience without sacrificing privacy.
In this product repository, where you can find out how they build their container to manage the whole Mark II device.
Then lest take this device as an example:
We can clone using:
pvr clone https://pvr.pantahub.com/mycroft-ci/myc200_dev/163 myc200_dev cd myc200_dev
This will generate a folder structure like this:
[4.0K] . ├── [4.0K] awconnect │ ├── [ 695] lxc.container.conf │ ├── [108M] root.squashfs │ ├── [ 126] root.squashfs.docker-digest │ ├── [ 510] run.json │ └── [1.1K] src.json ├── [4.0K] bsp │ ├── [ 348] build.json │ ├── [ 28M] firmware.squashfs │ ├── [ 22M] kernel.img │ ├── [ 15M] modules.squashfs │ ├── [3.3M] pantavisor │ ├── [ 172] run.json │ └── [ 106] src.json ├── [4.0K] _config │ └── [4.0K] awconnect │ ├── [4.0K] etc │ │ ├── [4.0K] modules-load.d │ │ │ └── [ 85] mark-ii.conf │ │ └── [4.0K] wifi-connect │ │ └── [ 54] config.json │ └── [4.0K] opt │ └── [4.0K] wifi-connect │ └── [4.0K] ui │ └── [4.0K] build │ └── [4.0K] static │ ├── [ 17K] favicon.ico │ ├── [4.8K] favicon.png │ └── [7.5K] theme.css ├── [4.0K] mycroft │ ├── [ 903] lxc.container.conf │ ├── [ 417] mdev.json │ ├── [978M] root.squashfs │ ├── [ 88] root.squashfs.docker-digest │ ├── [ 388] run.json │ ├── [1.1K] src.json │ └── [ 788] v0alpha.app.manifest.json ├── [ 2] network-mapping.json └── [ 2] storage-mapping.json 13 directories, 26 files
You can see the
978M and the
root.squashfs.docker-digest that save the docker digest of the source container. When we run the
pvr app update the cli will check what is the digest of the latest container in the same tag, if is different is going to download the new container and create the squashfs, but when we use the
--patch instead of creating a full
root.squashfs it will create a
root.ovl.squashfs a lot smaller.
[4.0K] mycroft/ ├── [ 938] lxc.container.conf ├── [ 417] mdev.json ├── [ 32K] root.ovl.squashfs ├── [ 88] root.ovl.squashfs.docker-digest ├── [978M] root.squashfs ├── [ 88] root.squashfs.docker-digest ├── [ 322] run.json ├── [1.1K] src.json └── [ 788] v0alpha.app.manifest.json
Only 32K of
root.ovl.squashfs will be the only part that will be sent to the device as an update, making the process of updating a lot more faster and efficient. An using the
unsquashfs command we can see the files that changed between the two container versions.
$ unsquashfs -l mycroft/root.ovl.squashfs squashfs-root squashfs-root/etc squashfs-root/etc/mycroft squashfs-root/etc/mycroft/build-info.json squashfs-root/usr squashfs-root/usr/share squashfs-root/usr/share/fonts squashfs-root/usr/share/fonts/truetype squashfs-root/usr/share/fonts/truetype/noto-sans squashfs-root/usr/share/fonts/truetype/noto-sans/.uuid squashfs-root/var squashfs-root/var/cache squashfs-root/var/cache/fontconfig squashfs-root/var/cache/fontconfig/1055c6cb-ce21-4f17-83fb-04817aec2d90-le64.cache-7 squashfs-root/var/cache/fontconfig/80708cb0-69bf-42ac-80c2-199c7c9ba90a-le64.cache-7 squashfs-root/var/cache/fontconfig/cafc8e5e-8030-4ab8-ac42-6431b99e4ad7-le64.cache-7
Making this method a great option to update your application inside a pantavisor device. If after several updates have an overlay that is growing to a point that is almost the same size as the
root.squashfs we can reset the system to create a new
root.squashfs that will be used in the future to create the new patches. In order to do this we need to run:
pvr app update --newbase mycroft
Creating applications that extend from another container is pretty easy. You need to follow two steps:
- Create the container that will be used as the base for the other applications.
pvr app add --group=data --status-goal=MOUNTED --from=nginx:stable-alpine nginx
- Create the extension application using the argument
--baseto refer to the application that we will extend.
You need to create this container as a normal extension in your Dockerfile
FROM nginx:stable-alpine COPY nginx.conf /etc/nginx/conf.d/default.conf COPY index.html /usr/share/nginx/html/index.html
And then you will create a new app inside your device description using pvr with the --base argument.
pvr app add --base nginx --from=highercomve/hello_world_nginx:latest hello_world_nginx
This will create two folders inside the device definition structure.
├── [4.0K] nginx │ ├── [7.0M] root.squashfs │ ├── [ 77] root.squashfs.docker-digest │ ├── [ 288] run.json │ └── [1.2K] src.json ├── [4.0K] hello_world_nginx │ ├── [ 697] lxc.container.conf │ ├── [4.0K] root.ovl.squashfs │ ├── [ 101] root.ovl.squashfs.docker-digest │ ├── [ 324] run.json │ └── [ 962] src.json
As you can see the second container is only 4.0K and the named
root.ovl.squashfs if we run a
unsquashfs -l hello_world_nginx/root.ovl.squashfs
We can see the only two files that are inside the root.ovl.squashfs are the ones we copied inside the Dockerfile of the second container.
squashfs-root squashfs-root/etc squashfs-root/etc/nginx squashfs-root/etc/nginx/conf.d squashfs-root/etc/nginx/conf.d/default.conf squashfs-root/usr squashfs-root/usr/share squashfs-root/usr/share/nginx squashfs-root/usr/share/nginx/html squashfs-root/usr/share/nginx/html/index.html
We hope these two new features will allow all the developers inside the pantavisor community to manage better their devices resources and update process.