How to compact VirtualBox’s VDMK file size

This guide explains how to shrink (compact) the virtual disk files (files having extension .vmdk) of your VirtualBox virtual machines so they consume less disk space. It is also easier to upload and share them with other people.

Fill the free space with zeros
VirtualBox only knows that the space is really free if it’s been set to zero, and standard deletion won’t do this. Login into the virtual machine and run the following command:

cat /dev/zero > zero.fill; sync; sleep 1; sync; rm -f zero.fill

Navigate to VirtualBox virtual machine folder
The virtual machines you have configured in VirtualBox are stored in folder “VirtualBox VMs” inside your home directory (or at least when Ubuntu is your host machine). Every machine has its own directory with name starting with the machine name. You have to navigate to the folder containing the files of the virtual machine having disk you want to compact and open terminal.

Get the UUID of the virtual disk
You have to obtain the UUID of the disk you want to shrink. I will explain you why later.
The command to do this is:

vboxmanage showhdinfo box-disk1.vmdk

This command assumes that your working directory is the directory that contains the vmdk file. Of course, you have to change the name of the vmdk file if it is different.
Write down the reported UUID because you will need it later.

Convert it to vdi
In order to compact the disk, you need to convert it to VDI format using this command

VBoxManage clonehd box-disk1.vmdk box-disk1.vdi --format vdi

Compact

VBoxManage modifyhd box-disk1.vdi --compact

Convert it back to vmdk

rm box-disk1.vdi
VBoxManage clonehd box-disk1.vdi box-disk1.vmdk --format vmdk

Set the original UUID
You converted the original vmdk disk file to vdi, compacted it and created new vmdk file from the compacted one. So, basically, you have created new virtual disk with new UUID which will not be recognized by VirtualBox unless you set the original disk’s UUID to the newly created one using the following command:

vboxmanage internalcommands sethduuid ./box-disk1.vmdk <original UUID here>

Voilà! The virtual disk size is successfully reduced!

Leave me a comment if you have any troubles or ideas how to improve this article. I am not VirtualBox expert and so I can’t guarantee that the above guide is the easiest way to get the job done.

How to restore a vagrant box using vmdk disk

The proper way to backup (and later restore) vagrant box is by using vagrant package command. This command creates a re-usable box file that can be easily transfered and used on other machines.

I have two Vagrant boxes I use on Ubuntu host machine. The last time I reinstalled my host machine OS I forgot to backup the boxes using vagrant package command, but saved the ~/VirtualBox VMs/ folder and all the files in it. This folder contains the virtual machines I amusing with VirtualBox (the virtual HDs to be more specific), so I used them to restore my boxes. Below are the steps I used to recover the boxes using the virtual HDs (vmdk files)

1. Import the machine
Go to ~/VirtualBox VMs/<name of the machine>/ and double click on the file with vbox extension. This should open VirtualBox GUI and import the machine.

2. Create base box file using the virtual machine you just imported
Use the following command to create a box file.

vagrant package --base <name of running virtual machine> --output ubuntu.box

3. Import the base box
First at all, you have to know the name of vagrant box you are restoring. Just open the Vagrantfile and look for line like this

config.vm.box = "ubuntu/trusty64"

As you can see, the box I am trying to restore is named “ubuntu/trusty64” and so I have to run the following command:

vagrant box add "ubuntu" ./ubuntu.box

Of course, you have to adjust the file path to the ubuntu.box file you have created in step 2.
To confirm that box is successfully added, you can run

vagrant box list

.

If everything is OK, you can now start your box in the usual way – using vagrant up command. Enjoy!

Add GIT branch information to Bash prompt

I’ve seen such as fancy Bash prompts on various tutorials and Linux examples over the Internet and I’ve always wondered how is achieved. I never really had a enough free time to learn more about it and explore the options. But being jobless for a month gave me opportunity to play with the thinks I like 🙂

My solution is pretty simple: when you navigate to git controlled folder, the bash prompt will show “@ branch” after the directory name. Nothing fancy.

Just open your ~/.bashrc configuration file with your favorite editor and add the following:

get_git_branch () {
git name-rev HEAD 2> /dev/null | sed "s/[a-zA-Z0-9]\+\ \(.*\)/ @ \1/"
}

than put this into your PS1 string:

$(get_git_branch)

so it become something like that

PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w$(parse_git_branch) \n\$\[\033[00m\] '

Restart your terminal or type bash to start new bash session. Navigate to git controlled folder to test. It should look like this:

bash-git

VBoxManage.exe: error: Failed to open/create the internal network ‘HostInterfaceNetworking-VirtualBox’

If you use Vagrant on Windows 10, there is big chance of getting error like this when you try to start the machine.

vagrant up
    Bringing machine 'default' up with 'virtualbox' provider...
    ==> default: Clearing any previously set forwarded ports...
    ==> default: Clearing any previously set network interfaces...
    ==> default: Preparing network interfaces based on configuration...
        default: Adapter 1: nat
        default: Adapter 2: hostonly
    ==> default: Forwarding ports...
        default: 19021 (guest) => 19021 (host) (adapter 1)
        default: 4567 (guest) => 4567 (host) (adapter 1)
        default: 3000 (guest) => 3000 (host) (adapter 1)
        default: 3003 (guest) => 3003 (host) (adapter 1)
        default: 6379 (guest) => 6379 (host) (adapter 1)
        default: 9000 (guest) => 9000 (host) (adapter 1)
        default: 27017 (guest) => 37017 (host) (adapter 1)
        default: 22 (guest) => 2222 (host) (adapter 1)
    ==> default: Running 'pre-boot' VM customizations...
    ==> default: Booting VM...
    There was an error while executing `VBoxManage`, a CLI used by Vagrant
    for controlling VirtualBox. The command and stderr is shown below.

    Command: ["startvm", "8dd3dbf3-efe3-4183-98d6-7679f9e83238", "--type", "headless"]

    Stderr: VBoxManage.exe: error: Failed to open/create the internal network 'HostInterfaceNetworking-VirtualBox Host-Only Ethernet Adapter #9' (VERR_INTNET_FLT_IF_NOT_FOUND).
    VBoxManage.exe: error: Failed to attach the network LUN (VERR_INTNET_FLT_IF_NOT_FOUND)
    VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component ConsoleWrap, interface IConsole

The problem appears only when Vagrant is configured to use private_network (i.e. Host-Only Ethernet Adapter). If you change your Vagrantfile configuration so that bridged network is used instead of private one, the machine will boot up without any problem. But how to fix the issue with the private network?
It’s more easy than you can imagine!

1. Go to start menu -> Control Panel -> Network and Internet -> Network and Sharing Center -> click on “Change adapter settings” form the menu on the left sidebar. List of available network adapters will show up.
2. Double-click the “VirtualBox Host-Only Ethernet Adapter” from which the error is comming. Please note that you may have multiple VirtualBox ethernet adapters and so you should read more carefully the error message you get to determine the problematic one.
3. Click the “Properties” button
4. Tick the checkbox for “VirtualBox NDIS6 Bridged Networking driver”
5. Press the OK button.
6. Try vagrant up command now. The machine should boot normally.

Voila!

Mock Yii2 components

The well written code is also testable code – it is loosely coupled to the other components, it follows the single responsibility principle, there are no explicit dependencies and so on. We all know the rules, but there is always an exception.

It’s pretty common situation to unit test component or controller that depends on another core Yii component – for example using \yii\web\Request to get query string or $_GET variable.

Rewriting your library in order to remove the explicit dependencies is not always option, because it may lead to massive refactoring and actually break more things than benefits.

In such as case a what I really want to do is to assign temporarily \Yii::$app->request PHPUnit Mock Class, so I can override the return values e.g.

$request = $this->getMock('\yii\web\Request', ['getUserIP', 'getUserAgent', 'getBodyParams']);

$request->expects($this->any())
 ->method('getUserIP')
 ->will($this->returnValue('127.0.0.1'));

$request->expects($this->any())
 ->method('getUserAgent')
 ->will($this->returnValue('Dummy User Agent'));

$request->expects($this->any())
 ->method('getBodyParams')
 ->will($this->returnValue([]));


\Yii::$app->set('request', $request);

Enjoy!

Disabling graphical login in Raspbian

If you are using Raspberry PI as headless machine, you do not need to start the graphical server at all. It is useless when Rasbian is used as server OS and disabling it will free ~40MB of RAM.
To do so, you need to edit the /etc/X11/default-display-manager file using your favorite text editor.

$ sudo vim /etc/X11/default-display-manager

Then comment out the line

#/usr/sbin/lightdm

and add a new one

/bin/false

Save the file and reboot the machine. Voila!

Update

There is a easier way of doing it…
Just run the Rasbian configuration utility

$ sudo raspi-config

In boot option menu choose:

  1. Enable Boot to Desktop/Scratch and enable Console Text.
  2. Also check SSH by going in ssh and selecting Enable or disable ssh server.
  3. After you quit, you’ll be asked to reboot, choose Yes.

 

Symfony 2 : Redirect to Referer

There are few methods of redirecting to referer.
Inside your controller action you can use:

return $this->redirect(
              $this->getRequest()
                   ->headers
                   ->get('referer')
           );

Alternatively you can write it like this:

 $request = $this->getRequest();
 return $this->redirect($request->server->get('HTTP_REFERER'));

How to create user without useradd command in OpenWRT

There is no useradd command in OpenWRT. Instead, shadow user package is available but I found it overkill. Anyway, if you are interested in using it – check out this tutorial,
Follow these steps to create a user without using useradd command in OpenWRT. Continue reading “How to create user without useradd command in OpenWRT” »

Generate huge JSON files with custom PHP >5.3 class

Lately, I’ve been working on transitioning XML feeds to JSON format on big video site.  We generate these feeds in order to feed external search service with results. It’s similar to sitemap, but it provides more detailed information about the pages.

This task is challenging because of the following problems that need to be resolved:

  1. The feed need to represent over 500 000 database entries i.e. videos. It’s just not possible to generate huge PHP multidimensional array with more than 500 000 elements and pass it to json_encode(). Obviously, you need to generate small JSON objects (chunks) concatenated with hand-coded strings and so build the full feed.
  2. The development and production servers we use are equipped with outdated PHP version 5.3.27. That means:
    – No meaningful error messages because json_last_error_msg() function it’s not available prior PHP 5.5
    – No JSON_PRETTY_PRINT, JSON_UNESCAPED_SLASHES, and JSON_UNESCAPED_UNICODE
  3. The code should be easy to test and maintain, so it should provide meaningful debug information and error handling.

Continue reading “Generate huge JSON files with custom PHP >5.3 class” »