LinuxPlanet Blogs

By Linux Geeks, For Linux Geeks.

Archive for the ‘technology’ Category

Optimal Sailfish SDK workflow with QML auto-reloading

without comments

SailfishOS IDE open. Just press Ctrl+S to save and see app reloading!

SailfishOS IDE open. Just press Ctrl+S to save and see app reloading!

Sailfish is the Linux based operating system used in Jolla phones. Those who develop apps for Jolla use the Sailfish SDK (software development kit), which is basically a customized version of Qt Creator. Sailfish OS apps are written using the Qt libraries and typically in the C++ programming language. The user interfaces of Sailfish apps are however written in a declarative language called QML. The syntax of QML is a custom markup language and includes a subset of CSS and JavaScript to define style and actions. QML files are not compiled but stay as plain text files when distributed with the app binaries and are interpreted at run-time.

While SailfishOS IDE (Qt Creator) is probably pretty good for C++ programming with Qt libraries, and the Sailfish flavour comes nicely bundled with complete Sailfish OS instances as virtual machines (one for building binaries and one for emulating running the binaries) the overall workflow is not very optimal from a QML development point of view. Each time a developer presses the Play button to launch his app, Qt Creator builds the app from scratch, packages it, deploys it on the emulator (or a real device if set up to do so) and only then actually runs the app. After doing some changes to the source code, the developer needs to remember to press Stop and then press Play to build, deploy and start the app again. Even on a super fast machine this cycle takes at least 10 seconds.

It would be a much more optimal workflow if relaunching the app after QML source code changes would happen in only 1 second or even less. Using Entr it is possible.

Enter Entr

Entr is an multi platform app which uses the operating system facilities to watch for file changes and to run a command the instant a watched file is changed. To install Entr on a Sailfish OS emulator or device, ssh to the emulator or device, add the community repository chum and install the package entr with (Note the chum for 1.0.5.16 also exists, but the repo is empty.):

ssh nemo@xxx.xxx.xxx.xxx
ssu ar chum http://repo.merproject.org/obs/sailfishos:/chum:/1.0.4.20/1.0.4.20_armv7hl/
pkcon refresh
pkcon install entr

After this change to the directory where your app and it’s QML files reside and run entr:

cd /usr/share/harbour-seravo-news/qml/
find . -name *.qml | entr -r /usr/bin/harbour-seravo-news

The find command will make sure all QML files in current or any subdirectory will be watched. Running entr with parameter -r will make sure it kills the program before running it again. The name of our app in this example here is seravo-news (available in the Jolla store if you are interested).

With this the app would automatically reload it any of the QML files change. To do this mount the app directory on the emulator (or device) to your local system using SSH:

mkdir mountpoint
sshfs nemo@xxx.xxx.xxx.xxx:/usr/share/harbour-seravo-news mountpoint/

Then finally open Qt Creator, point it to the files in the mountpoint directory and start editing. Every time you’ve edited QML files and you feel like you want to see how the result looks like, simply press Ctrl+S to save and watch the magic! It’s even easier than what web developers are used to do then pressing F5 to reload, because on the emulator (or device) there is nothing you need to do, just look at it while the app auto-restarts directly.

Remember to copy or directly git commit your files from the mountpoint directory when you’re completed writing the QML files.

Entr has been packaged for SailfishOS by Seravo staff. Subscribe to our blog to get notified when we post about how to package and build RPM packages using the Open Build System and submit them for inclusion in SailfishOS Chum repository.

Written by Otto Kekäläinen

April 15th, 2014 at 2:37 am

Open source for office workers

without comments

Document Freedom logoOpen source software is great and it’s not only great for developers who can code and use the source directly. Open source is a philosophy. Open source is for technology like what democracy is for society: it isn’t magically superior right away, but it enables a process which over time leads to best results – or at least avoids the worst results. A totalitarian regime might be efficient and benevolent, but there is a big risk it will become corrupt and get bad. And then a totalitarian regime gets bad, it can be really, really ugly.

Because of this philosophy even regular office workers should strive for maximizing their use of open source software. To help ordinary non-technical people Seravo has contributed to the VALO-CD project, which in 2008-2013 created a collection of the best Free and Open Source Software for Windows, which is available both in Finnish and English. The CD (contents suitable also for a USB stick) and related materials are still available for download.

We have also participated in promoting open standards. Most recently we helped the Free Software Foundation Europe publish a press releases in Finland regarding the Document Freedom Day. Also the theme of our latest Seravo-salad was the OpenDocument Format. Open standards are essential in making sure users can access their own data and open files in different programs. Open standards is also about programs being able to communicate with each other directly using publicly defined protocols and interfaces.

Information technology is rather challenging, and understanding abstract principles like open source and open standards does not happen in one go. Seravo is proud to support the oldest open source competence center in Europe, the Finnish Center for Open Systems and Solutions COSS ry which has promoted open technologies in Finland since 2003.

When it comes down to details, training is needed. This and last year we have cooperated with the Visio educational centre in Helsinki to provide courses on how to utilize open source software in non-profit organizations.

Learn more

We have recently published the following presentations in Finnish so people can learn more by themselves:





Written by Otto Kekäläinen

March 31st, 2014 at 4:56 am

How’s (battery) life with Jolla?

without comments

Some years ago Nokia conducted a large survey among its customers on what people most liked about Nokia phones. One of the top features turned out to be their unrivaled battery life. Despite the hype around screen resolutions, processor performance and software versions, one of the most important features for a mobile device is simply how long until you have to charge it again.

Jolla phone uptime shows device has been continously on for 8 days and 13 hours

Jolla phone uptime shows the device has continuously been turned on for 8 days and 13 hours

Back in 2012 we wrote about how to get an Android phone last for a week without recharging. On a mobile phone, the single most significant power hog is the display. On the other hand with the display turned off, the biggest energy hogs are the wireless radio devices. The Android phone in our example lasted for an entire week after locking it in 2G mode only, thus utilising only the most basic GSM network connection with all other forms of connectivity disabled.

The Nokia Maemo series devices and the Meego N9 smart phone already sported a feature where if the device was not in active use, it would automatically downgrade the network connections or disable them. When a user the opened an application that requires network access, the network was re-enabled automatically without extra user intervention. This feature is also present in Jolla phones. This is the reason why Jolla users every now and then see the “Connecting” notification; the connections are disabled but are automatically invoked upon request for network access.

We tested this feature by having all networks (3G, WLAN, Bluetooth, GPS) enabled in the Jolla phone settings and by having e-mail updates active with instant message presence turned on, but with no further active usage of the device. The results showed that the Jolla phone battery lasted for over 8 days! The screenshot attached is from SailTime by Mikko Ahlroth, which visualises the output of the uptime Linux command.

Keeps on running…

But wait, that was not all! The unique hardware feature present in the Jolla Phone is, of course The Other Half (abbreviated TOH), an exchangeable back side of the device. One of the connectors of TOH is a I2C connection, which is able to transfer both data and power. This makes possible TOHs that supplement the main battery within the device. In fact, the main battery is also located on the backside, so it could completely be replaced with a TOH that connects directly to the connectors the original battery would use.

First examples of this have already emerged. Olli-Pekka Heinsuo has created two battery supplementing Other Halves: the Power Half, which holds an extra battery for increased capacity and the Solar Half, which hosts a solar panel that directly charges the device. Olli-Pekka is attending the Seravo sponsored Jolla and Sailfish Hack Day next Saturday. If you wish to attend, please make sure to swiftly register to the event as the attendance capacity is limited!

Solar Half Power Half

Written by Otto Kekäläinen

March 27th, 2014 at 6:20 am

Installing Node.js on SUSE Linux Enterprise

without comments

SUSE logo
The officially supported collection of software in SUSE Linux Enterprise Linux 11 Service Pack 3 does not contain all conceivable Linux software, but in the Open Build System there are tons of software that is build for SLES 11SP3. Installing these software packages and repositories is of course on your own risk, as they are not a part of the officially supported offering. In practice however it works quite well to have a officially supported SLES base and on top of that a handful of additions that you care about yourself.

Node.js on SUSE

The JavaScript server side engine Node.js is an example of a program that you might need to complete your mission, but which isn’t in SUSE by default. You could of course download and install the sources from nodejs.org, but that isn’t a very good option in terms of maintenance, security and automatic updates. The optimal way is to browse the public SUSE instance of the Open Build Service at software.opensuse.org. By entering the search term “nodejs” and from the results clicking “Show other versions” and opening the section “SUSE SLE-11 SP 3″ and “Show unstable versions” you can see all the repositories that contain SLES11SP3 compatible packages with the name “nodejs”.

The first colon separated part in the repository names indicate what the type of the repository is. Repos named “home:” something belong to individual users (similar to a PPA repository at Launchpad.net for those who are familiar with Ubuntu). Other names are project names, and thus more likely to have a group of maintainers and thus preferred over individual users repositories. In this case the best repository is likely to be the official devel tools subproject nodejs at “devel:languages:nodejs”.

Importing the openSUSE repository public key

Importing the openSUSE repository public key

Once the correct line is identified simply click on the “1 Click Install” link and a .ymp file will be downloaded and opened with the SUSE package manager. This .ymp file contains both the package name and repository information. If installation is executed, the repository will permanently be added to the system and the package in question installed, and in future also automatically updated. Just like PPAs in Ubuntu this repository is single-purpose and only contains a Node.js packages so no other package on the system will be affected of overridden by updates from this repository, so it is fairly safe to use. One click install also has a command line tool, so alternatively you could run:

$ OCICLI http://software.opensuse.org/ymp/devel:languages:nodejs/SLE_11_SP3/nodejs.ymp

Git and SUSE

When deploying your Node.js apps you most likely also need the Git version management software. With the same principles above you can simply install it by running:

$ OCICLI http://software.opensuse.org/ymp/devel:tools:scm/SLE_11_SP3/git.ymp

Software.openSUSE.org

Using these same principles you can install any software from the openSUSE instance of the Open Build Service. Just browse to http://software.opensuse.org/find (often abbreviated as s.o.o) and start searching!

Written by Otto Kekäläinen

March 3rd, 2014 at 5:13 am

Jolla and Sailfish Hack Day 2014-02-22

without comments

Would you like to learn how to write a Sailfish OS app and get it into the Jolla store? Or are you already working on a Sailfish app but need help with it? Join the Jolla and Sailfish Hack Day to code and learn from other SailfishOS and QML enthusiasts. And if you already have an app in the Jolla store, please come and share your expertise!

Sailfish OS logoThe day will start at 12:00 with Artem Marchenko’s presentation “How to make a Sailfish app that passes Harbour QA (and would be good enough for the Jolla store)” where he presents his Hello World Pro example.

After this tutorial everybody can open SailfishOS SDK on their laptop and start coding. During the day experts will walk around and help out. You are also encouraged to help your peers with what you can! Together will make great apps.

Requirements:

  • some priori coding experience and courage to dive into SailfishOS SDK
  • an idea for an app
  • your own laptop to code on
  • limited seats - participants must be registered

Not required:

  • Jolla phone – you can write and run your app with the SailfishOS SDK and get your Jolla device later
  • food or drink – there will be pizza, beer and other refreshments sponsored by Seravo

Register now. Participant amount is limited.

Organized by Devaamo rySeravo Oy and Arch Red Oy

Written by Otto Kekäläinen

February 11th, 2014 at 7:57 am

The Jolla phone – first impressions

without comments

Jolla phone lineupThe first device running SailfishOS, the successor of Meego, has finally been released. It’s elegant and beautiful both on the outside and inside. It has multiple unique features that makes it unlike any mobile device we’ve seen so far.

We have been waiting for Jolla to release their phone for more than a year and finally it has happened. It is certainly not an easy task to make the world’s greatest mobile device and fulfill all the expectations people have for Jolla, but they have indeed succeeded in doing something amazing. The last Nokia Meego device N9 was very good and appraised for its gesture based interface and now two years later we can find that all new Google apps, Windows Metro and Ubuntu phone among others are built around swiping. In Jolla SailfishOS the gesture based interface is refined and feels almost magical to use.

The hardware

The device rocks an clean and elegant Nordic design. It is simply beautiful. There are no front facing buttons but on the side there is a power key and volume buttons. The volume buttons double as camera buttons if the camera is open. The back facing camera has an LED flash, auto-focus and an 8 megapixel sensor which is more than enough. In fact, the default wide screen camera mode takes 6,1 megapixel pictures, which many agree is the most optimal file size-to-quality ratio. There is also a smaller front facing camera for video call use. The internal storage is 16 GB and there is a slot for an external microSD card where users can attach e.g. a 64 GB card. The screen resolution is 960×540 pixels which looks very sharp on the beautiful 4,5 inch screen – no pixels can be distinguished with the bare eye. All the usual sensors are included (compass, gyrometer, acceleration, ambient light). The GPS is a dual chip with Glonass, so it will be able to figure out the location quickly – even inside vehicles or indoors. The battery is user-replaceable and has a capacity of 2100 mAh.

Jolla and Other Halves

Jolla and Other Halves

All of the above can be found in other top-of-line smart phones as well. However, there is one hardware feature that is unique to Jolla: The Other Half. The Other Half is a concept of user-changeable smart back cover. The basic covers included in the package feature an embedded NFC chip that makes the Sailfish UI change color and theme (called Ambience). However, the Other Half could connect to the main device also using a I2C connector. I2C is a bus standard common in all sorts of electronic devices and it can transmit both power and communication. Using this bus, anybody could make all kinds of imaginative Other Halves. Hopefully a keyboard will be one of the first Other Halves to come to the market. The battery connectors are also facing the back cover, so it should be easy for any manufacturer to make a giant 10 000 mhA battery filled Other Half. And of course the Other Half could also use Bluetooth or other generic means to communicate with the main device. The official specs will soon be released, so even home 3D-printing enthusiasts may produce their own Other Halves. It will be very exciting to see what kind of Other Halves start to appear in the future.

The software: SailfishOS

The software is indeed something altogether unique. Most readers of this blog are familiar with the story of Maemo-Meego-Eflop. Now the big question is, does SailfishOS offer something it’s competition does not? – Yes! The swipe based UI is a bit weird the first 5 minutes you use it, but once your muscle memory catches up, you’ll notice your fingers swiping all devices you touch and your brain wondering why those other devices require unnecessary amounts of thought to use. Android is easy to use, but even after using SailfishOS just for one day, going back to the Android world of multiple desktops, widgets, app menus and such starts to look rather complex. SailfishOS is just so natural you need to experience it yourself.

Jolla main screen showing off multitaskingVisualise this: you take your device out of your pocket and then double tap on the screen with your thumb to activate it. Then, without moving the place your thumb is at you just swipe down a little bit and feel the device vibrate three times as the selection passes over the pull down menu options. Even without looking at the device you know when the third option (camera) is selected. Then you simply lift your thumb and the camera opens. Then you point at something, move your thumb slightly down and then touch to take the picture. Then you might want to look at the picture you just took? Just swipe left and the picture is visible in full screen. Pinch to zoom. When you are done and you e.g. want to look at the clock, just swipe left starting from the edge of the screen and you get to see the main view with time, battery status, open apps overview etc. Maybe in the middle of that you decide you still have time to capture some more photos. Instead of swiping all the way to the end, you reverse direction in the middle and return back to the camera. All this with very smooth animations and responsive feeling.

On day-to-day basis a very important part of the user experience is the on-screen keyboard. SailfishOS uses the same Maliit keyboard familiar to many from the N9, which is excellent. It is strange that this open source keyboard component hasn’t been picked up in other mobile Linux “distributions”.

Jolla themselves talk a lot about the UI theme system called Ambiance. Basically you pick a background picture and then ambiance applies automatically the picture and matching colors all over the user interface. Most of the time it is stylish, but sometimes the system thinks bright red is a good colour for UI elements and you wish you could set the colours manually; but you can’t.

Multitasking is a feature that sounds technical, but when you have the app overview open and see the real time minified versions of the apps (think Gnome Shell) the feature and its advantages are easy to understand. It was great to be able to have Osmand download a 150 MB offline map file uninterrupted while browsing some photos at the same time. It was great to be able to load up a music video on YouTube and be able to listen to it without having the screen turned on at all or while reading an e-mail in another app.

The settings center is fantastic. In fact, the whole way the app settings, contacts, accounts and everything in general seems to be integrated very well; but then again that was already true with Nokia N900. This is an area where the Maemo-Meego was a pioneer and the competition hasn’t catched up yet. Linux geeks will love that in the settings there is an option to enable the developer mode. With developer mode enabled, the app Terminal is visible. This app is in fact the famous FingerTerm app with a special four row keyboard with all the special keys needed in terminal use. The keyboard sits translucent on top of the content, so the maximal screen area is available for terminal output and the app works well in both landscape and portrait mode. Linux geeks and developers will also respect the fact that SailfishOS is a true GNU/Linux system running Linux kernel 3.4 with a fully functional shell, software is managed as RPM packages with Zypper and the SailfishOS project is very open to new contributors. If you want to read about how this software is shipped, checkout the SailfishOS site and upstream projects Mer and Nemo.

The SailfishOS label says ‘beta’ but the system itself seems mature and stable, at least in our use so far. All the functionality that belongs to a modern mobile OS is there, but when it comes to things apps should do, there are still many things lacking.

The software: HTML5, Jolla and Android apps

Sailfish architecture

Sailfish architecture

Jolla has its own app store where you can browse and install native Qt/QML-based apps. At the moment however, there are very few apps, but then again all the basic apps you need like an alarm clock, e-mail, maps, calendar etc. are included.

At Seravo we hold the belief that in the long term, browser based HTML5 apps will be more important than native apps. Therefore to us the mobile browser is more important than any of the native apps combined. It should be is easy to search the web, enter URLs, open new tabs, save bookmarks etc. All of this can be done with the current browser in SailfishOS. The browser seems to work pretty fast and flawlessly. We heard at the launch event that the rendering engine is Gecko (same as in Firefox) so it is likely to have good support for HTML5 features. However in terms of usability, Chrome for Android is still the best mobile browser we’ve used so far. In particular the SailfishOS browser does not seem to have support for landscape mode. Hopefully while SailfishOS matures, the browser will grow to be more polished as well.

Android apps can be run using Alien Dalvik (probably some sort of virtual machine layer). You can either get both free and paid apps from the bundled Yandex store or you can get individual .apk files from other sources and install them manually. If you prefer open source only, then one option would be to use F-Droid, which can be installed by simply opening in the browser f-droid.org, downloading the apk and activating it.

Interesting times ahead

On the SailfishOS website it states that “We believe this will act as a refreshing sea-wind that will help push the industry forward.” Indeed it is refreshing and is leads the industry forward by leaps and bounds. At the same time there are also interesting developments going on with FirefoxOS, Ubuntu and Tizen. Whenever these are compared, Jolla seems to get the best reviews. However, SailfishOS and particularly the ecosystem around it has just started to grow so nothing is certain yet. The only thing we can be sure of is that we live in very interesting times. In the coming years, billions of people are going to buy new mobile phones and to many of them, their mobile phone is going to be their primary device to get online and get involved in the information society. If you wish to become part of this, you can get involved in the SailfishOS community. SailfishOS can, in fact, be installed on other devices as well as the Jolla phone, but nevertheless we recommend, in particular to Linux fans, to scroll down at Jolla.com and sign up for availablility notifications so you can eventually get a Jolla phone for yourself.

Gallery

Jolla phone lineup Close up back Close up front Jolla and the Other Half Jolla and Other Halves Sailfish architecture Jolla main screen showing off multitasking SailfishOS settings menu SailfishOS dialer with Snow Ambiance SailfishOS dialer with Dark Ambiance Maliit keyboard Vibrating pull down menu Remorce notification allows user to cancel action within 5 seconds Seravo.fi in browser Browser tabs Main screen with some apps open Photos app F-Droid on SailfishOS Osmand on SailfishOS Developer mode Jolla with terminal open (Fingerterm) Jolla and Dell XPS 13 with Gnome Shell

Official video

Written by Otto Kekäläinen

November 28th, 2013 at 7:38 am

Dual-booting Ubuntu 12.04 and Windows 8 with Restricted Boot

without comments

Windows 8 Secure Boot and Ubuntu 12.04In parallel with the introduction of Windows 8, Microsoft also introduced new hardware requirements to all manufacturers who sell pre-installed Windows 8 computers. As Microsoft Windows still has a monopoly in the desktop and laptop computer market, lots of people who buy computers suffer from a new anti-feature Microsoft introduced: Secure Boot.

All Windows 8 computers now come with a new kind of BIOS called UEFI, and with a feature called “Secure Boot” activated. The Free Software Foundation in the U.S. calls this “Restricted Boot“. This feature basically means, that every new computer gets an Microsoft master encryption key embedded at the factory and later on each time the computer powers on and starts the operating system found on the hard disk, it will only do so if the operating system signature is one of those signed by the Microsoft master key. To avoid new monopoly accusations, Microsoft has agreed to sign the boot keys of many Linux distributions and you can still install Linux on computers that are built for Windows 8. On all machines we have seen it is also still possible to disable Secure Boot. However the complexity of the overall installation procedure has increased, which hurts end users and stifles competition. We’ve also stumbled into cases which indicate that there exists multiple Microsoft master keys while Linux distributions have been signed with only one, and the Linux-compatible key is not included in all Windows 8 computers.

If you can avoid buying a pre-installed Windows 8 machine, then don’t buy one! If you want a Linux machine, we warmly recommend the Dell XPS 13 that is available with pre-installed Ubuntu. There are also companies like Teraset, ZaReason, ThinkPenguing and System76 that sell pre-installed Linux laptops. Buy your computer with Linux pre-installed and you’ll avoid all installation related issues.

The instructions below might be useful also when installing other Linux distributions, but for simplicity we focus on one use case: how to install Ubuntu 12.04 alongside a Windows 8.

Procedure

1. To free some disk space for the Ubuntu installation, resize the hard drive using the disk tools in Windows 8.

2. Boot the computer and open the (UEFI) BIOS by pressing F2. If F2 does not work, try other keys like DEL or F8 or consult the manual of your device. Also make sure that the device is really booting and not just continuing from a suspended state by forcefully turning off the computer from the power button (or remove the battery if possible).

3. In the BIOS settings, find the option “Secure Boot” and disable it.

4. Reboot the computer, with a USB stick containing the latest Ubuntu 12.04.3 LTS point release. If your computer does not boot from the USB stick, check out BIOS settings or access the boot menu directly with F8 or some other key, depending on your device.

5. Once you get Ubuntu booted, install it into the free space created earlier.

6. After installation, reboot into Ubuntu. Open a console and disable the Grub OS prober by adding into the file /etc/defaulf/grub the line DISABLE_OS_PROBE=”true”

7. To be able to dual-boot with Windows, add the manual boot entry by appending the file /etc/grub.d/40_custom with the following lines:

menuentry "Windows 8" {
 insmod part_gpt
 insmod chain
 set root='(hd0,gpt2)'
 chainloader /EFI/Microsoft/Boot/bootmgfw.efi
}

8. Run sudo update-grub so that the changes take effect and then reboot.

Now you’ll see the boot menu when the computer starts and you can successfully boot either Ubuntu or Windows 8.

Typical errors

If when booting into Windows 8 you see likes like:

error: unknown command 'drivemap'
error: invalid EFI file path

or

/EndEntire
file path: /ACPI(a0341d0,0)/PCI(2,1f)/UnknownMessaging(12)/HD(2,96800,32000,
7c043777b8608641,87,f6)/File(\EFI\Microsoft\Boot)/File(bootmgfw.efi)/EndEntire
error: cannot load image

This means that the file /EFI/Microsoft/Boot/bootmgfw.efi was not found on the hard disk and partition (hd0,gpt2). Use the disk tool in Ubuntu to mount all partitions and search them to find the correct location of that file, and update the custom boot entry accordingly.

See also

Written by Otto Kekäläinen

September 20th, 2013 at 4:32 am

Tips to optimize images for faster loading websites

without comments

Image compressionFor a multitude of reasons ranging from user conversion rate to search engine ranking, it is important for web sites to load fast. The first rule of thumb to having fast loading pages is to keep the amount of data the users need to transfer small. The smaller the web page file and its dependencies are, the faster it will load. Saving bandwidth is always good, both from the service provider perspective, who saves in data center costs, and from the end user perspective, in particular for those users who have slow mobile connections.

On any average site, most of the transferred data is due to image files. For example loading the front page of the Finnish broadcasting company yle.fi totals 1,4 MB of data, which of 0,9 MB is image files. Another example, the front page of the Finnish Defence Forces totals 365 KB of data which of 284 KB are due to images.

Therefore, a quick and easy way to make your site load faster is to optimize the images. Linux users can easily do this using the programs optipng and jpegoptim, which are available in most Linux distributions. After installing the programs, open a console and go to the directory where your web site is, e.g. cd /var/www/. Then run the following commands to automatically search for all png and jpeg files in the directory and its subdirectories, and optimize them in-place:

find . -iname '*.png' -print0 | xargs -0 optipng -o7 -preserve
find . -iname '*.jpg' -print0 | \
 xargs -0 jpegoptim --max=90 --strip-all --preserve --totals

Both optipng and jpegoptim work losslessly, meaning that the visible image quality will not decrease even though the file sizes decrease. Additionally jpegoptim also has options to do lossy compression, meaning that the visible image quality can be reduced a little bit, but usually the loss of quality is very small but file size savings big. In our example above the use the option –max=90 which will decrease the JPEG image quality to 90 for pictures that are above it. This is pretty safe, since quality level 90 is still very high and no website visitor is likely to spot any defects in the image. If more compression is needed, it is better to experiment e.g. using the Gimp and the Save for web feature with live preview.In the jpegoptim command, the option –strip-all will strip any metadata including Exif data from images. For websites JPEG metadata is most likely not needed, so it is a preffy safe decision to strip them.If a server administrator wants to automatically optimize all images on a server (like we do on Seravo’s servers), create a script with the contents:

#!/bin/bash
echo `date` >> /root/optipng.log
find /var/www/ -mtime -2 -iname '*.png' -print0 | \
 xargs -0 optipng -o7 -log /root/optipng.log -preserve
echo `date` >> /root/jpegoptim.log
find /var/www/ -mtime -2 -iname '*.jpg' -print0 | \
 xargs -0 jpegoptim --max=90 --preserve --totals >> /root/jpegoptim.log

Then make this into a cronjob by adding a line to cron like:

# m h dom mon dow command
0 1 * * * /root/optimize-images.sh

With these in place, the script will be run every night at 01:00 and it will find all PNG and JPEG images that have been created or modified during the last two days, and then it will optimize them in-place and preserving all other file attributes. Exif and metadata is not stripped, as in this scenario we cannot be sure that all images stored in website folders are meant only for web site viewing.

Update 2013-12-04: As of jpegoptim version v1.3.0 a new option ‘–all-progressive’ is available, which is recommended to use as the JPEG images will the load progressively and thus provide a better end-user visual experience during page load.

If the server has some publicly uploadable folder it might be a good idea to downscale images for web use, as 8Mpix images directly uploaded from a digital camera are way too big for any reasonable web use. To automatically scale all images in a directory to that the longer edge is at max 1000 pixels run Mogrify (part of GraphicsMagic):

mogrify -resize "1000x1000>" *.jpg

Other programs also exist, like pngcrush and jpegtran, but in our tests they where inferior to optipng and jpegoptim. However it seems that there is also some room for improvement for these tools, as for example the Yahoo Smush.it service can do a better job in particular in small file size PNG optimizations. Even more compression for PNG files can be achieved using the TinyPNG service, but it works in a lossy way and thus images might get visible defects. In most cases, the optipng and jpegoptim commands above will do a great job.

One question however remains: why do not all image editors have excellent optimization functions built-in in the first place?

CSS sprites, embedded images and glyph fonts

If you have multiple small images you can combine them into CSS sprites with GraphicMagick using this command:

gm convert +append *.png sprite.png

Now the sprite.png will be a vertically combined image of your PNG images. The resulting file size will be smaller than the sum of the small images, and it will also load faster because the browser needs to make only one HTTP round-trip to fetch the file. See article at A List Apart on what CSS markup to use with CSS sprites.

If all there is on a site is a few small images, they could even be embedded into the HTML code using the data URI scheme.

Another option is to use glyph fonts like Font Awesome. They have the added benefit of scaling to any size, as font files are vector graphics. If the graphic designer is clever, the whole site could be made using only CSS tricks and custom fonts, with perhaps only two or three actual images, and thus load blazing fast. That we leave up as an exercise to our readers.

Related articles

You might also be interested in these related articles:

Written by Otto Kekäläinen

August 22nd, 2013 at 4:07 am

Dell XPS 13 Ubuntu Edition – first impressions

without comments

Update October 2013: We bought a new Dell XPS 13 in October with Ubuntu pre-installed but unfortunately it seems to have problems with the wireless: wifi connections seem to disconnect at random and thus wifi is useless. We found out that the in the new laptop shipped to us this fall the Intel wifi card has been replaced with an Atheros wifi card, which unfortunately does not have a proper Linux driver, thus the connection issues. It seems Dell has been aware of Atheros wifi card problems since July as a thread in the Dell support forum shows, but still they ship the “downgraded” crappy model. With some tweaking, you may or may not get the Atheros wifi model working.

As the whole point of buying a pre-installed Linux laptop is to get hardware that is guaranteed to work with Linux, we can no longer recommend this laptop. It is a shame. Our earlier Dell XPS 13 still works perfectly as it has the Intel wifi card.

We received our first Dell XPS 13 Ubuntu Edition laptop last week. We decided to publish a quick review on it, as it is a very impressive product and in general every pre-installed Linux laptop is newsworthy as they are so rare, at least here in Finland.Dell XPS 13 power leds

This Dell XPS 13 Ubuntu Edition was released early this year globally and this month it became available in Finland. It is actually already the second generation in a Dell-Canonical-cooperation on creating the ultimate laptop for web developers.

We purchased our piece directly from Dell Finland but they only sell to corporate customers, so if you are a private person you need to buy from one of their re-sellers. After calling around all the re-sellers in Finland we found out most resellers refuse to sell the Ubuntu model, only two of sell it on request, and only a single re-seller has it readily available as an ready to order item on their website. We recommed every Finnish Linux users to visit www.teraset.net/linux.php which lists three pre-installed Dell models ready to order.

From package to ready-to-use in less than 20 minutes

When we finally got the device, the experience was great. It arrived a week earlier than promised, neatly packaged and ready to use with full battery and pre-installed Ubuntu. The version of Ubuntu is 12.04 with some Dell additions, among others a Dell end user license agreement screen inside the standard Ubuntu first-boot installer. There was also a cool animation on the first boot which we’ve never seen before in the standard Ubuntu OEM-installer. The default file system was ext4, which is probably in 12.04 still a better choice than btrfs. The default partition layout included some Dell specific recovery partitions, but we didn’t test how they work so we can’t say how useful the feature is. The level of localization was good and a Finnish speaking persons would have no problems in setting up and running their Dell XPS 13 Ubuntu Edition laptop.Dell XPS 13 package

In general, the overall user experience of the product from packaging to having it fully running and configured was excellent, beating anything we’ve ever seen before. The whole process was also really fast, from opening the package to using the laptop it took less than 20 minutes. The only thing where there is room for improvement is that the keyboard still had an Windows button, but luckily we had Linux stickers to cover it. This is one of the details that Linux-only laptop sellers like ZaReason and System76 do better, but otherwise nobody else can offer an ultrabook anywhere close to the Dell XPS 13. We also noticed that the Getting Started -leaflet included only instructions on how to get started with the Windows 8 version of the Dell XSP 13 laptop and nothing about Ubuntu was mentioned, but that we can forgive as we understand that grasping Windows 8 is difficult and many users install the open source Classic Shell in Windows 8 while Ubuntu is user-friendly enough to get by without any printed manuals.

Beautiful hardware externally and internally

Externally the Dell XPS 13 reminds quite a lot an MacBook Air, but slightly better. The size of the laptop is just perfect: it long edge is one centimeter shorter on a 13 inch MacBook Air but unlike the smaller 11 inch MacBook Air, Dell XPS 13 is still big enough to have a decent screen size and there is very little marginal from the screen edges to the case edges, so all space is efficiently used in an optimal way. Also the screen resolution of 1920×1080 pixels (full HD) is excellent and significantly better than the 1366×768 or 1440×900 common in laptops with similar physical dimensions. Dell XPS 13 is also big enough to host a full sized keyboard which makes it pleasant to use.

Animation at first boot

Animation at first boot

360 degrees view

360 degrees view

As the device is so thin, there is no room for a DVD tray, but in the rare occasions you might need one, you can use an external CD/DVD drive. There is also no ethernet port, but that too can be achieved with an external USB dongle if needed. Also there are no dock connectors in the bottom of the laptop, but even when one of our previous Dell laptops had the dock connectors, we never came around to buy a docking station, so we won’t be missing that feature either. The essential connectors that one really needs are exactly the ones found on this device:

  • one USB3 port with power (even then the laptop is turned off)
  • a second USB3 port (no powered)
  • a DisplayPort for external screens
  • a combined mic/speaker port for headset use
  • a power cord port

The Dell XPS 13 has also a neat hardware feature we haven’t seen before: a button you can press and a led scale that will indicate the battery charge level, even when the laptop is turned off.

As the laptop is Ubuntu certified, all the keyboard function buttons work as expected and you can for example turn on and off the keyboard backlight  The trackpad feels nice and works nicely with the multi-touch trackpad Linux driver, enabling two-finger scrolling in both directions. Naturally the Linux drivers for everything else was also working perfectly.

The SSD disk has TRIM support, but for some reason if was not enabled in the Ubuntu install by default. Even though we haven’t  yet noticed any problems with drivers, it is comforting to know that on Launchpad.net there is a custom Sputnik kernel PPA and all the Dell XPS 13 related bugs are carefully followed. We are sure no other laptop gets that much attention from Canonical engineers. Since fixed go upstream to mainline Linux, this laptop is an excellent choice also for users of other Linux distros, like Debian or OpenSUSE.

Performance-wise this laptop came with:

  • Intel Core i7-3537U CPU (2.00 GHz x 4)
  • 8 GB RAM
  • 256 GB Samsung PM830 mSATA SSD

The Dell XPS 13 Ubuntu Edition is a bit more expensive than the cheapest  MacBook Air, but it packs with more impressive hardware, so the Dell XPS is well worth its price.

Ten hour battery life

When it comes to laptops, one feature we in particular value is battery life. A previous Dell Latitude E5420 (a pre-installed Ubuntu model bought in 2011) with a 9-cell battery we have had as new a maximal battery life of 9,5 hour. With this much smaller Dell XPS 13 with a 45  Wh battery we get a maximal 11 hours battery life. The power usage is pretty well optimized out-of-the box, and after installing TLP, tweaking a little with Intel PowerTop and having the screen backlight turned off (which leaves text on the screen still readable) we got a power consumption figure as low as 4,7 watts and calculated battery life of 670+ minutes.  Amazing!Dell XPS 13 Ubuntu Edition at best 4,47 watts

Of course this figure is only valid if you sit on a long flight and spend all 11 hours only reading a simple text document. In reality while working you’ll have three virtual machines running, nine shell windows and twenty browser tabs open, so you’ll battery wouldn’t last the whole day, but luckily the power adapter of Dell XPS 13 is smaller than anything we’ve seen before and weights only 280 grams, so your whole toolkit with laptop and power adapter with cables will be just a mere 1,5 kg in total.

We also tried to measure how much power it draws in standby, but after having it over the weekend in suspend mode, when we opened it the battery status indicated the same level as at the time when the lid was closed. We could not calculate how long the battery lasts when the laptop is suspended, but we guarantee it is long enough, probably weeks. Besides, it only takes 13-16 seconds to boot, so doing a real shutdown won’t loose you much time either if you prefer it (for example due to security reasons).

Gallery

Four pre-installed Linux computers in comparison: Dell Latitude E5420, Asus EeePC, MeeGo tablet and Dell XPS 13 Ubuntu Edition Animation at first boot USB gives power even then laptop is shutdown. Default partition layout One small glitsh in packaging fixed Bottom flap At best only 4,47 watts! Nice leds indicate power in cord and charging or ready Packaging. Note how small the power adapter is. 360 degrees view 10 hour battery life Screen edge of Dell XPS 13 is really small. Comparison to Latitude E5420. Small laptop, but full sized keyboard. Dell XPS in comparison to Latitude E5420. A small fan can be seen in the bottom, but the laptop is practically soundless. Battery level button pressed, LEDs show charge level

Videos

Here are boot and suspend videos to give you a sense of the speed:

Where to buy pre-installed Linux laptops and other computers?

Currently the best directory of vendors can be found at Linuxpreloaded.com. If you live in Finland we recommend either the pre-installed Ubuntu computers from Dell or alternatively GGS-Data, a Swedish company that ships to Finland and provides keyboards and power plugs that suites Finns.

Written by Otto Kekäläinen

April 23rd, 2013 at 1:53 am

Improve MySQL performance to speed up websites

without comments

In every business the rule is that if your service is slow, you’ll loose customers, and it certainly applies online too. If your website takes two seconds to load instead of a half second, you’ll loose visitors and business. Even if it is just about improving by a few seconds, on the web it does matter.

We’ve written before about the web server speed and application level performance, now it’s time to tackle the most common bottle neck in larger web apps: the database. This time we did some benchmarks of our WordPress hosting optimized server, running Linux,  Nginx, MySQL and PHP. Since the LAMP-stack is the most common application stack on the web and WordPress might be the most common web server app, we hope the performance tips below will be useful for many. The research presented here will for sure be closer to real life scenarios than generic benchmarks done by The Transaction Processing Performance Council.

For the benchmarks we disabled our FastCGI cache and started out by taking the baseline figures for the seravo.fi/blog page, which is somewhat database heavy and most likely to reflect possible speed gains. Testing was done with Apache Bench, measuring 500 page loads in 10 concurrent connections:

$ ab -n 500 -c 10 -g blogpage-baseline.dat http://seravo.fi/blog

This was run twice and the first run was considered a warm-up, so the results where recorded only for the latter run. Baseline speed was 8.23 requests per second and 1211 milliseconds per request (mean values).

MariaDB logoAs the first step in MySQL optimization we installed MariaDB, a drop-in-replacement of MySQL. It is basically a fork of MySQL, but maintained and developed by the original MySQL developers, many of who fled from Oracle. When Oracle acquired Sun and thus also their most serious  competitor, the MySQL database and trademark, a concern rose whether or not Oracle is serious about developing MySQL, or will it simply keep MySQL just enough alive to shake off the competition authorities. You can’t yet directly say that MySQL has gone sour, but there are some concerning signs, like Oracle adding close source extensions to MySQL and Oracle handling security updates poorly. Luckily all the security vulnerabilities are fixed in MariaDB and it is easy to migrate to it by installing MariaDB via the MariaDB.org website and repositories.

After simply switching to MariaDB the speed was 8.64 requests per second and 1151 milliseconds per request (mean values). Not a big change, but still in the right direction. The next thing was to change the storage engine from WordPress default MyISAM to XtraDB (as InnoDB in called in MariaDB). That was done using this little spell:

$ sudo -s
$ DATABASENAME="wp"
$ echo 'SHOW TABLES;' \
 | mysql --defaults-file=/etc/mysql/debian.cnf ${DATABASENAME} \
 | awk '!/^Tables_in_/ {print "ALTER TABLE `"$0"` ENGINE = InnoDB;"}' \
 | column -t \
 | mysql --defaults-file=/etc/mysql/debian.cnf ${DATABASENAME}
$ exit

This change alone however didn’t have any notable gains, as the speed was 8.55 req/s and 1165 ms/req. Next we increased the InnoDB caching capabilities by editing file /etc/mysql/my.cnf and increasing the sizes of the following options:

innodb_log_file_size = 32M
innodb_buffer_pool_size = 1024M
innodb_log_buffer_size = 4M

Since innodb_log_file size cannot be applied via simple restart, we ran sudo /etc/init.d/mysql stop, edited options, deleted old log files out of the way with sudo rm /var/lib/mysql/ib_logfile* and then restarted with sudo /etc/init.d/mysql start. A good log file size depends on the amount of traffic your site gets. In addition to InnoDB changes we enabled the option that allows to log long running queries:

slow_query_log=1

The file /var/log/mysql/mariadb-slow.log was empty however.

With these changes we stood at 8.73 req/s and 1141 ms/req. As the last thing we increased the query cache:

query_cache_limit = 512K
query_cache_size = 128M

After the last round the final score was  8.87 req/s and 1124 ms/req for our 500 request benchmark. All in all the gains were very small (8.87 vs. 8.23 req/s is below 8%) and no single significant option was found. Graph of all benchmarks below:

MySQL and MariaDB tuning benchmark

From the graph we can conclude that our optimization was successful for high loads. At the left we can see that the addition of query cache will slow things down slightly on low loads, so you might want to consider leaving out our last optimization tip in your setup.

Note that this article was about choosing your MySQL flavor and settings. There is still many other things you can do to optimize your database speed. You could for example time all SQL queries and see which are slow and which could be rewritten or perhaps have an index added for a common WHERE statement. For example for WordPress there are many handy plugins available for analyzing SQL queries, which we will present in a later article. Stay tuned!

Written by Otto Kekäläinen

March 15th, 2013 at 4:31 am