From here.

The Ubuntu MATE team offers a bespoke images for UMPCs like GPD Pocket. You can download the ISO files here. The direct downloading link for “GPD Pocket” is this.

The hot keys to access the boot menu & BIOS are:

1
2
3
4

GPD Pocket: "Fn + F7"
GPD Pocket 2: "Fn + F12"
GPD Pocket 3: "Fn + F7"
...

So all you need to do is to download the ISO file, dump it to a flash drive, and install it.