LinuxPlanet Blogs

By Linux Geeks, For Linux Geeks.

Archive for the ‘technology’ Category

Philosophy And Servers – Part 1

without comments

Hello everyone, I hope you’re well. I’ve written a lot about my health situation lately but I also promised I would write properly about technology as soon as time allowed. I’m pleased to say that day has finally arrived. I want to tell you about the changes I’ve made to my computing setup in the last 6-9 months and the thinking behind it, I’ll also dish the dirt on this new Pixel C tablet I’m currently toting. That sounds like a lot to cram in so I’ll probably split this into 2 or 3 posts, we’ll see. So let’s start with the new computing philosophy I’ve come to and the reasons for it.

A side on photo of the HP Microserver Gen8 with the case open

HP Microserver Gen8 – case open

In the last couple of years I’ve been lucky enough to have the use of a Google Nexus 10 tablet belonging to my employers. Prior to this I hadn’t really gotten on board with the whole tablet computing revolution. I remember the previous false dawns and promises of tablets a decade ago. I’d had a couple of Nexus smartphones and maintained an an active interest in Android, even trying Android x86 on a netbook, but despite the excitement surrounding iPads and other shiny new devices I just didn’t see the point. “I have a phone and a laptop, what else would I need?” was my mindset. Over time though I noticed I was using the Nexus tablet more and more, rarely even turning on the laptop at home. The purchase of a bluetooth keyboard case for the Nexus 10 helped a lot with this, it felt much more like a mini laptop anyway. I could already easily do all of the following: Email, social media, web browsing, podcast aggregation, media consumption (Netflix, Plex, YouTube, BBC iPlayer etc), casual gaming (only solitaire or Angry Birds but I have a PS4 for serious gaming), time management and organisation (calendar, shopping lists, TODOs etc). The only things I couldn’t do on Android were audio production, web development and running virtual machines. I have a pretty powerful desktop PC that can take care of the podcast production and audio editing, probably the other bits too. I don’t do as much development these days anyway but when I do it’s carried out in an SSH session to a remote server anyway. That just left VMs and I figured a server could also take care of that. With the proliferation of faster broadband speeds, cloud computing, thin clients and an effective return to the client/server model it all seemed to be heading one way. I don’t need to carry a full computer when I can have a cool mobile device and can use it as a thin client anyway if I need to.

I’ve long thought that our phones will become our main computers eventually anyway. 5 or 10 years ago that seemed far fetched but these days you hardly need to be Nostradamus to see that one coming. A few companies have already attempted to create smartphones you can dock into a keyboard and large display, so far it hasn’t really worked out but sooner or later someone will get it right. My hunch is “sooner”, in the next year or two. I know this is a big part of the Ubuntu phone strategy, running Android and Ubuntu side by side, it remains to be seen whether that works. Tablets are certainly ripe for this kind of market. I also have a feeling Google are looking to merge Android and ChromeOS in the very near future, making a more serious play for the desktop and laptop market. New Google CEO Sundar Pichai is clearly keen. It’s silly to have these 2 separate products confusing the market, and while you could argue that they serve different purposes and different users, I don’t really buy that. “Convergence”, “synergy” and other such PR buzz words have long been in fashion. Now even Microsoft has realised that having 15 different editions of Windows is stupid. Have the same core OS across phone, tablet, laptop and desktop just brand it all the same. I can see why servers might still be a different kettle of fish but it makes sense to keep everything together. I’m excited about what Android N could bring in May and I’ve taken a gamble investing in the Pixel C. Fingers crossed.

Anyway, enough marketing and business speculation nonsense, where did all this ruminating really leave me? I’d already been running my own server for backups, media management and other things. It was a Buffalo Linkstation Duo NAS I hacked to allow full SSH access for backups (gotta love rsync) but it was low powered and also ARM based. It couldn’t handle increasingly essential software for me like Plex, Syncthing and ZeroTier (more on those in the next post). I had an old Lenovo netbook lying around and my makeshift solution was to set this up with Linux Mint MATE edition and then mount the NAS automatically on boot. It worked but if I was to truly embrace thin client computing and KVM I knew I had to step up my server game.

The Kernel Virtual Machine logo, Tux juggling next to the word KVM

Tux Likes To Juggle Apparently

My home computing set up this time last year was: a powerful studio desktop PC for podcasting/music which we can leave out of the equation as I was always going to keep that, this cobbled together Buffalo NAS and Lenovo netbook server, my ASUS laptop (Core i3, 4Gb RAM, not terrifically powerful but enough) and the Nexus 10 for casual stuff. After much experimentation in the last year I now have a powerful server with KVM to suit my desktop needs and a Google Pixel C tablet for most of my mobile computing needs. I still have the laptop of course, I haven’t thrown it away, it just hasn’t been switched on in almost a month.

I figured I should talk you through my transition in these next 2 articles. It’s still an experiment really. In the next post I’ll explain how I got hold of an HP Microserver and upgraded the hardware, installed all the appropriate software, broke it, fixed it, broke it again and so on. What I’ve learned about Plex, KVM, Zero Tier and other things. Finally in the 3rd post I’ll talk about my recent purchase of the Google Pixel C and review the device properly, I’m actually writing this on it right now. I plan to get all this done before I go into hospital in 2 weeks so stay tuned.

Until then take care everyone,

Dan

Written by Dan

January 26th, 2016 at 9:51 am

Philosophy & Servers – Part 1

without comments

Hello everyone, I hope you’re well. I’ve written a lot about my health situation lately but I also promised I would write properly about technology as soon as time allowed. I’m pleased to say that day has finally arrived. I want to tell you about the changes I’ve made to my computing setup in the last 6-9 months and the thinking behind it, I’ll also dish the dirt on this new Pixel C tablet I’m currently toting. That sounds like a lot to cram in so I’ll probably split this into 2 or 3 posts, we’ll see. So let’s start with the new computing philosophy I’ve come to and the reasons for it.

A side on photo of the HP Microserver Gen8 with the case open

HP Microserver Gen8 – case open

In the last couple of years I’ve been lucky enough to have the use of a Google Nexus 10 tablet belonging to my employers. Prior to this I hadn’t really gotten on board with the whole tablet computing revolution. I remember the previous false dawns and promises of tablets a decade ago. I’d had a couple of Nexus smartphones and maintained an an active interest in Android, even trying Android x86 on a netbook, but despite the excitement surrounding iPads and other shiny new devices I just didn’t see the point. “I have a phone and a laptop, what else would I need?” was my mindset. Over time though I noticed I was using the Nexus tablet more and more, rarely even turning on the laptop at home. The purchase of a bluetooth keyboard case for the Nexus 10 helped a lot with this, it felt much more like a mini laptop anyway. I could already easily do all of the following: Email, social media, web browsing, podcast aggregation, media consumption (Netflix, Plex, YouTube, BBC iPlayer etc), casual gaming (only solitaire or Angry Birds but I have a PS4 for serious gaming), time management and organisation (calendar, shopping lists, TODOs etc). The only things I couldn’t do on Android were audio production, web development and running virtual machines. I have a pretty powerful desktop PC that can take care of the podcast production and audio editing, probably the other bits too. I don’t do as much development these days anyway but when I do it’s carried out in an SSH session to a remote server anyway. That just left VMs and I figured a server could also take care of that. With the proliferation of faster broadband speeds, cloud computing, thin clients and an effective return to the client/server model it all seemed to be heading one way. I don’t need to carry a full computer when I can have a cool mobile device and can use it as a thin client anyway if I need to.

I’ve long thought that our phones will become our main computers eventually anyway. 5 or 10 years ago that seemed far fetched but these days you hardly need to be Nostradamus to see that one coming. A few companies have already attempted to create smartphones you can dock into a keyboard and large display, so far it hasn’t really worked out but sooner or later someone will get it right. My hunch is “sooner”, in the next year or two. I know this is a big part of the Ubuntu phone strategy, running Android and Ubuntu side by side, it remains to be seen whether that works. Tablets are certainly ripe for this kind of market. I also have a feeling Google are looking to merge Android and ChromeOS in the very near future, making a more serious play for the desktop and laptop market. New Google CEO Sundar Pichai is clearly keen. It’s silly to have these 2 separate products confusing the market, and while you could argue that they serve different purposes and different users, I don’t really buy that. “Convergence”, “synergy” and other such PR buzz words have long been in fashion. Now even Microsoft has realised that having 15 different editions of Windows is stupid. Have the same core OS across phone, tablet, laptop and desktop just brand it all the same. I can see why servers might still be a different kettle of fish but it makes sense to keep everything together. I’m excited about what Android N could bring in May and I’ve taken a gamble investing in the Pixel C. Fingers crossed.

Anyway, enough marketing and business speculation nonsense, where did all this ruminating really leave me? I’d already been running my own server for backups, media management and other things. It was a Buffalo Linkstation Duo NAS I hacked to allow full SSH access for backups (gotta love rsync) but it was low powered and also ARM based. It couldn’t handle increasingly essential software for me like Plex, Syncthing and ZeroTier (more on those in the next post). I had an old Lenovo netbook lying around and my makeshift solution was to set this up with Linux Mint MATE edition and then mount the NAS automatically on boot. It worked but if I was to truly embrace thin client computing and KVM I knew I had to step up my server game.

The Kernel Virtual Machine logo, Tux juggling next to the word KVM

Tux Likes To Juggle Apparently

My home computing set up this time last year was: a powerful studio desktop PC for podcasting/music which we can leave out of the equation as I was always going to keep that, this cobbled together Buffalo NAS and Lenovo netbook server, my ASUS laptop (Core i3, 4Gb RAM, not terrifically powerful but enough) and the Nexus 10 for casual stuff. After much experimentation in the last year I now have a powerful server with KVM to suit my desktop needs and a Google Pixel C tablet for most of my mobile computing needs. I still have the laptop of course, I haven’t thrown it away, it just hasn’t been switched on in almost a month.

I figured I should talk you through my transition in these next 2 articles. It’s still an experiment really. In the next post I’ll explain how I got hold of an HP Microserver and upgraded the hardware, installed all the appropriate software, broke it, fixed it, broke it again and so on. What I’ve learned about Plex, KVM, Zero Tier and other things. Finally in the 3rd post I’ll talk about my recent purchase of the Google Pixel C and review the device properly, I’m actually writing this on it right now. I plan to get all this done before I go into hospital in 2 weeks so stay tuned.

Until then take care everyone,

Dan

Written by Dan

January 26th, 2016 at 9:51 am

The Best of 2015

without comments

 

Another new year has started, and we think it is time to look a bit back and review what became the most popular content in Seravo Blog in 2015.

The five most popular blog posts in Seravo.fi in 2015

1. 10 reasons to migrate to MariaDB (if still using MySQL)

The most popular post last year – and actually the first article written in 2015 – was the one praising MariaDB and providing a list of reasons to start using it as your database. The article also created lively discussion in the comment section!

2. Ubuntu Phone and Unity vs Jolla and SailfishOS

In August we compared two Linux-based mobile operating systems that stand as competitors to Android. As a conclusion we claimed that  “Jolla and SailfishOS would be the technically and usabilitywise superior alternative, but then again Ubuntu could be able to leverage on it’s position as the most popular Linux distribution in desktops and servers”. All in all, we hoped for new innovations fueling from these (at least for now minor) competitors.

3. Ubuntu Phone review by a non-geek

The new Ubuntu Phone was clearly in the center of the open source community’s attention in the summer of 2015. A not-that-technical review on the phone became the year’s third most visited blog post. After that the phone has been tested and tried by Seravo geeks too – and has not gained too admirable evaluations…

4. Continuous integration testing for WordPress plugins on Github using Travis CI

If that wasn’t technical enough for you, maybe the article on testing WordPress plugins will do? And if you found youself puzzled after reading it, do not hesitate to contact us. Seravo can help you using PHPUnit, Rspec and Travis in your projects, please feel free to ask us about our WordPress testing via email at wordpress@seravo.fi .

PS. If you identify yourself as a Finn (or for some other odd reason happen to be familiar with Finnish), check out our blog on WP-palvelu.fi that focuses on all things happening within and around WordPress.

5. Fixing black screen after login in Ubuntu 14.04

The fifth most read article in 2015 was our guide to fixing a Ubuntu login problem that raised from some of our customers. Instead of charging for one hour of work for fixing the problem, we provided the instructions for everyone to do it on their own.

 

The all-time favourites

Even though they say that infomation ages faster than ever, it seems that even ancient articles in the Seravo blog are still valid and popular today. The post on how to turn any computer into a wireless access point with Hostapd from August 2014 is actually still the most popular of all seravo.fi pages with over 67 000 page visits. The more recent post on why to migrate to MariaDB has taken the second place with nerly 57 000 visits.

Next in the all-time top five favourites come the following articles:

Free Your Android phone (and upgrade to the latest Android version)!

Optimizing web server performance with Nginx and PHP

Virtualized bridged networking with MacVTap

 

In fact, our front page seravo.fi with approximately ten thousand visits per year only comes as sixth on the list of our most popular pages. This proves us that a blog is a great way to attract attention and to participate in the on-going technological discussion.

Although located in the northern corner of the world, in this distant land of snow and software, Seravo has with its online presence drawn visitors from all over the world: United States, India, Germany and United Kingdom form the top four and our dear own Finland takes the fifth place on this list.

At this point we would like to thank all of our visitors, readers and especially those who have commented on the blog posts! We will keep on posting new topics in 2016 as well, and the comment section is always open for new thoughts and constructive criticism. We are also more than happy to receive ideas for future blog posts from any of you.

Happy new year!

Written by Sanna Saarikangas

January 14th, 2016 at 3:31 am

Posted in technology

The Best of 2015

without comments

 

Another new year has started, and we think it is time to look a bit back and review what became the most popular content in Seravo Blog in 2015.

The five most popular blog posts in Seravo.fi in 2015

1. 10 reasons to migrate to MariaDB (if still using MySQL)

The most popular post last year – and actually the first article written in 2015 – was the one praising MariaDB and providing a list of reasons to start using it as your database. The article also created lively discussion in the comment section!

2. Ubuntu Phone and Unity vs Jolla and SailfishOS

In August we compared two Linux-based mobile operating systems that stand as competitors to Android. As a conclusion we claimed that  “Jolla and SailfishOS would be the technically and usabilitywise superior alternative, but then again Ubuntu could be able to leverage on it’s position as the most popular Linux distribution in desktops and servers”. All in all, we hoped for new innovations fueling from these (at least for now minor) competitors.

3. Ubuntu Phone review by a non-geek

The new Ubuntu Phone was clearly in the center of the open source community’s attention in the summer of 2015. A not-that-technical review on the phone became the year’s third most visited blog post. After that the phone has been tested and tried by Seravo geeks too – and has not gained too admirable evaluations…

4. Continuous integration testing for WordPress plugins on Github using Travis CI

If that wasn’t technical enough for you, maybe the article on testing WordPress plugins will do? And if you found youself puzzled after reading it, do not hesitate to contact us. Seravo can help you using PHPUnit, Rspec and Travis in your projects, please feel free to ask us about our WordPress testing via email at wordpress@seravo.fi .

PS. If you identify yourself as a Finn (or for some other odd reason happen to be familiar with Finnish), check out our blog on WP-palvelu.fi that focuses on all things happening within and around WordPress.

5. Fixing black screen after login in Ubuntu 14.04

The fifth most read article in 2015 was our guide to fixing a Ubuntu login problem that raised from some of our customers. Instead of charging for one hour of work for fixing the problem, we provided the instructions for everyone to do it on their own.

 

The all-time favourites

Even though they say that infomation ages faster than ever, it seems that even ancient articles in the Seravo blog are still valid and popular today. The post on how to turn any computer into a wireless access point with Hostapd from August 2014 is actually still the most popular of all seravo.fi pages with over 67 000 page visits. The more recent post on why to migrate to MariaDB has taken the second place with nerly 57 000 visits.

Next in the all-time top five favourites come the following articles:

Free Your Android phone (and upgrade to the latest Android version)!

Optimizing web server performance with Nginx and PHP

Virtualized bridged networking with MacVTap

 

In fact, our front page seravo.fi with approximately ten thousand visits per year only comes as sixth on the list of our most popular pages. This proves us that a blog is a great way to attract attention and to participate in the on-going technological discussion.

Although located in the northern corner of the world, in this distant land of snow and software, Seravo has with its online presence drawn visitors from all over the world: United States, India, Germany and United Kingdom form the top four and our dear own Finland takes the fifth place on this list.

At this point we would like to thank all of our visitors, readers and especially those who have commented on the blog posts! We will keep on posting new topics in 2016 as well, and the comment section is always open for new thoughts and constructive criticism. We are also more than happy to receive ideas for future blog posts from any of you.

Happy new year!

Written by Sanna Saarikangas

January 14th, 2016 at 3:31 am

Posted in technology

Using RAID with btrfs and recovering from broken disks

without comments

Btrfs logoBtrfs (pronounced Better FS) is a relatively new filesystem that operates on the copy-on-write principle (abbreviated COW, which stems a more friendly pronunciation for btrfs: Butter FS). Btrfs includes a lot of interesting functionality and replaces traditional Linux disk and filesystem tools like LVM (volume manager, disk snapshots) and mdadm (software RAID).

In RAID usage btrfs is much more flexible and space efficient than traditional mdadm, because in btrfs the disks in the RAID array do not conform to any predefined size or count requirements. You can attach any amount of disk of any size to a btrfs RAID array, and btrfs will automatically balance the data across the devices according to the requirements of the selected RAID level. RAID levels 0 and 1 are currently supported while RAID 5 and 6 are under development and will be available as officially supported configurations soon.

Let’s take an example case to see how btrfs RAID works. In traditional mdadm based RAID if you have two 1 TB disks configured to mirror each other in RAID 1 mode, and you want to expand this setup, you basically need to install two more disks, and they need to be of equal size. In btrfs, life is much more flexible.

To expand an array, you can add a disk of any size and btrfs will automatically adjust to the sitation. So if you have two 1 TB disks, and you add a 2 TB disk to the array, then all 4 TB would be utilised and you will have 2 TB of usable storage. Btrfs will make sure that as RAID 1 mandates, each file will exist in two copies and those copies are stored on physically different disks. If you would expand the two 1 TB array with one 1 TB disk, then btrfs would give you 1,5 TB of usable disk space in total while still satisfying the RAID 1 replicate requirements.

Btrfs RAID is flexible also in the sense that you can either create the RAID array by running something like mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1 when you create the partitions, or you can at any later time convert existing partitions to use RAID by running a command like btrfs balance start -dconvert=raid1 -mconvert=raid1 /home

If RAID is active, it will be visible in the btrfs filesystem df command:

$ btrfs fi df /home
Data, RAID1: total=1.59TiB, used=1.59TiB
System, RAID1: total=32.00MiB, used=256.00KiB
Metadata, RAID1: total=10.00GiB, used=8.27GiB
unknown, single: total=512.00MiB, used=0.00

Keep in mind, that the free values shown by the normal df are not reliable, as it is hard to predict how disk usage will behave in a copy-on-write and snapshotting filesystem like btrfs.

Recovering from failed disks in btrfs

If a disk in a btrfs RAID 1 array fails, then btrfs will refuse to mount that filesystem and error messages will be visible in the syslog. If it was the root filesystem, then the system will refuse to boot normally and the system will usually boot to an initramfs console. Luckily all decent systems that support btrfs (like Ubuntu 14.04) will have btrfs tools included in the initramfs environment, so you can run btrfs commands from there and try to recover from the situation without the need to boot the system form an alternative media, like a live CD.

A btrfs volume with a failed (missing) disk will output something like this:

$ btrfs fi show
Label: none  uuid: 4e90ec15-e6f5-470d-96be-677f654a5c79
    Total devices 3 FS bytes used 1.59TiB
    devid    1 size 2.71TiB used 1.60TiB path /dev/sda1
    devid    2 size 1.80TiB used 1.48TiB path 
    devid    3 size 447.14GiB used 121.00GiB path /dev/sdc1/pre>
To force the btrfs volume to mount anyway, the degraded option can be used:
$ mount -t btrfs -o degraded /dev/sda2 /home

The correct thing to do when a disk in an RAID array fails, is to replace it. Once you have a new disk in place notify btrfs about it with the command:

$ btrfs replace start /dev/sdd1 /dev/sdb1 /home

This command reads files both from the original drive (if still accessible) and from other disks in the RAID array, and uses that information to populate the new clean disk.

If needed, the -r flag will prevent the system from trying to read from the outgoing drive if possible. Replacement operations can be canceled, but they cannot be paused. Once the operation is complete, /dev/sdc1 will no longer be a part of the array and can be disposed of.

In some cases one could also run btrfs device delete missing /dev/sdb1 and then add a new drive, but the replace command is the primary command and can be run even if the old drive is completely dead.

Recovering from filesystem corruption in btrfs

In cases where the physical disk has not failed but instead something in the btrfs journal or checksum trees is corrupted and does not match, and the filesystem refuses to mount, this is the recommended procedure to try:

First make a backup the volume.

After that try to mount the volume in the read-only recovery mode:

$ mount -t btrfs -o ro,recovery /dev/sda2 /home

If that fails, look in syslog (or run dmesg) and look for btrfs errors:

[ 74.926506] Btrfs loaded
[ 74.927393] BTRFS: device fsid 4e90ec15-e6f5-470d-96be-677f654a5c79 devid 2 transid 691061 /dev/sdc1
[ 77.439765] BTRFS info (device sdc1): disk space caching is enabled
[ 77.440620] BTRFS: failed to read the system array on sdc1

If there are messages relating to the log tree (not in the example above), then reset the log tree by running:

$ btrfs-zero-log

If syslog shows problems regarding the chunk tree, then btrfs rescue chunk-recover may be of used to replace the chunk blocks with new ones that should work (but may loose some data). Each disk has multiple copies of super blocks, and they are very unlikely to all get corrupted at the same time, but it happens they can be recovered with the rescue command:

$ btrfs rescue super-recover -v /dev/sde1
All Devices:
 Device: id = 3, name = /dev/sdc1
 Device: id = 1, name = /dev/sda2
 Device: id = 2, name = /dev/sde1

Before Recovering:
  [All good supers]:
      device name = /dev/sdc1
      superblock bytenr = 65536

      device name = /dev/sdc1
      superblock bytenr = 67108864

      device name = /dev/sdc1
      superblock bytenr = 274877906944

      device name = /dev/sda2
      superblock bytenr = 65536

      device name = /dev/sda2
      superblock bytenr = 67108864

      device name = /dev/sda2
      superblock bytenr = 274877906944

  [All bad supers]:
      device name = /dev/sde1
      superblock bytenr = 65536

      device name = /dev/sde1
      superblock bytenr = 67108864

      device name = /dev/sde1
      superblock bytenr = 274877906944

After those, try btrfsck, and possibly with options -s1, -s2, -s3. It the volume is still not mountable, then try btrfsck --repair.

The command btrfsck --repair --init-extent-tree may be necessary if the extent tree was corrupted. If there is corruption in the checksums, try –init-csum-tree.

Last resort is to run btrfs check --repair but it’s not recommended because it might write changes to the disk that destroys data.

Generic tools might also be useful. For example the tool testdisk is able to scan disks and find lost partition tables, including ones with btrfs partitions.

Restoring files from a broken btrfs filesystem

If it simply is impossible to mount a btrfs filesystem, it is possible to use the command btrfs restore to fetch files from withing a damaged btrfs partition. The default command will get all files from the root volume.

Sometimes simply restore isn’t enough. For example in Ubuntu, by default the /home directory is a separate btrfs subvolume. To fetch files from there the correct volume root must be defined via the -r option. Also you might not be interested in restoring all possible files, maybe just one particular directory, and for such use a filename filter can be defined with the --path-regex option.

To fetch all files from /home/otto/Kuvat on a system there the @home subvolume object id is 258, the task can be accomplished with the command:

$ btrfs restore -i -vvvv -r 258 --path-regex "^/(otto(|/Kuvat(|/.*)))$" /dev/sdc1 .
...
Restoring ./otto/Kuvat/2015/08/09/IMG_2888.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2889.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2890.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2891.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2892.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2893.JPG
Found objectid=18094703, key=18094702
Done searching /otto/Kuvat/2015/08/09
Found objectid=18094632, key=18094631
Done searching /otto/Kuvat/2015/08
Found objectid=8304076, key=8304075
Done searching /otto/Kuvat/2015
Found objectid=272, key=271
Done searching /otto/Kuvat
Found objectid=258, key=257
Done searching /otto
Found objectid=257, key=256
Done searching

Detecting data corruption

If btrfs detects errors, they will be logged to syslog. Btrfs also maintains error counters, which on normal healthy drives should always list all zeros:

$ btrfs device stats /mnt
[/dev/sda2].write_io_errs   0
[/dev/sda2].read_io_errs    0
[/dev/sda2].flush_io_errs   0
[/dev/sda2].corruption_errs 0
[/dev/sda2].generation_errs 0
[/dev/sdc1].write_io_errs   0
[/dev/sdc1].read_io_errs    0
[/dev/sdc1].flush_io_errs   0
[/dev/sdc1].corruption_errs 0
[/dev/sdc1].generation_errs 0

Btrfs automatically calculates CRC-32C checksums for both data and metadata blocks, and at regular intervals checks if the checksums still match or not. If data corruption is detected, then btrfs will log errors to syslog. If RAID 1 is enabled, btrfs will also automatically fix the corrupted data by overwriting with by the correct duplicate. This process can also be automatically triggered by running btrfs scrub.

Checking disk health

Disks that support the SMART standard are able to report their health status. The Gnome tool ‘disks’ provides a very easy way to access SMART data. Just open Disks, select a device and choose ‘Show SMART Data & Self-Tests’.

Gnome Disks tool Gnome Disks tool SMART data

Alternatively a command line representation of the same data can be fetched with:

$ sudo smartctl -A -H /dev/sda
smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.2.0-90-generic] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0032   098   098   000    Old_age   Always       -       6322
 12 Power_Cycle_Count       0x0032   097   097   000    Old_age   Always       -       2646
175 Program_Fail_Count_Chip 0x0032   100   100   010    Old_age   Always       -       0
176 Erase_Fail_Count_Chip   0x0032   100   100   010    Old_age   Always       -       0
177 Wear_Leveling_Count     0x0013   086   086   010    Pre-fail  Always       -       503
178 Used_Rsvd_Blk_Cnt_Chip  0x0013   094   094   010    Pre-fail  Always       -       184
179 Used_Rsvd_Blk_Cnt_Tot   0x0013   094   094   010    Pre-fail  Always       -       336
180 Unused_Rsvd_Blk_Cnt_Tot 0x0013   094   094   010    Pre-fail  Always       -       6192
181 Program_Fail_Cnt_Total  0x0032   100   100   010    Old_age   Always       -       0
182 Erase_Fail_Count_Total  0x0032   100   100   010    Old_age   Always       -       0
183 Runtime_Bad_Block       0x0013   100   100   010    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
195 Hardware_ECC_Recovered  0x001a   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   253   253   000    Old_age   Always       -       0
232 Available_Reservd_Space 0x0013   094   094   000    Pre-fail  Always       -       3080
241 Total_LBAs_Written      0x0032   099   099   000    Old_age   Always       -       15812437286
242 Total_LBAs_Read         0x0032   099   099   000    Old_age   Always       -       11879385910

Storing data efficiently and dependably are fundamental tasks in computing. Data integrity and data availability are key principles in information security. Btrfs is a promising technology that will help system administrators fulfill these goals and in particular with SSD disks btrfs should be the default filesystem choice for all new systems, and a must for syadmins to read up on.

Written by Otto Kekäläinen

December 9th, 2015 at 7:31 am

Using RAID with btrfs and recovering from broken disks

without comments

Btrfs logoBtrfs (pronounced Better FS) is a relatively new filesystem that operates on the copy-on-write principle (abbreviated COW, which stems a more friendly pronunciation for btrfs: Butter FS). Btrfs includes a lot of interesting functionality and replaces traditional Linux disk and filesystem tools like LVM (volume manager, disk snapshots) and mdadm (software RAID).

In RAID usage btrfs is much more flexible and space efficient than traditional mdadm, because in btrfs the disks in the RAID array do not conform to any predefined size or count requirements. You can attach any amount of disk of any size to a btrfs RAID array, and btrfs will automatically balance the data across the devices according to the requirements of the selected RAID level. RAID levels 0 and 1 are currently supported while RAID 5 and 6 are under development and will be available as officially supported configurations soon.

Let’s take an example case to see how btrfs RAID works. In traditional mdadm based RAID if you have two 1 TB disks configured to mirror each other in RAID 1 mode, and you want to expand this setup, you basically need to install two more disks, and they need to be of equal size. In btrfs, life is much more flexible.

To expand an array, you can add a disk of any size and btrfs will automatically adjust to the sitation. So if you have two 1 TB disks, and you add a 2 TB disk to the array, then all 4 TB would be utilised and you will have 2 TB of usable storage. Btrfs will make sure that as RAID 1 mandates, each file will exist in two copies and those copies are stored on physically different disks. If you would expand the two 1 TB array with one 1 TB disk, then btrfs would give you 1,5 TB of usable disk space in total while still satisfying the RAID 1 replicate requirements.

Btrfs RAID is flexible also in the sense that you can either create the RAID array by running something like mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1 when you create the partitions, or you can at any later time convert existing partitions to use RAID by running a command like btrfs balance start -dconvert=raid1 -mconvert=raid1 /home

If RAID is active, it will be visible in the btrfs filesystem df command:

$ btrfs fi df /home
Data, RAID1: total=1.59TiB, used=1.59TiB
System, RAID1: total=32.00MiB, used=256.00KiB
Metadata, RAID1: total=10.00GiB, used=8.27GiB
unknown, single: total=512.00MiB, used=0.00

Keep in mind, that the free values shown by the normal df are not reliable, as it is hard to predict how disk usage will behave in a copy-on-write and snapshotting filesystem like btrfs.

Recovering from failed disks in btrfs

If a disk in a btrfs RAID 1 array fails, then btrfs will refuse to mount that filesystem and error messages will be visible in the syslog. If it was the root filesystem, then the system will refuse to boot normally and the system will usually boot to an initramfs console. Luckily all decent systems that support btrfs (like Ubuntu 14.04) will have btrfs tools included in the initramfs environment, so you can run btrfs commands from there and try to recover from the situation without the need to boot the system form an alternative media, like a live CD.

A btrfs volume with a failed (missing) disk will output something like this:

$ btrfs fi show
Label: none  uuid: 4e90ec15-e6f5-470d-96be-677f654a5c79
    Total devices 3 FS bytes used 1.59TiB
    devid    1 size 2.71TiB used 1.60TiB path /dev/sda1
    devid    2 size 1.80TiB used 1.48TiB path 
    devid    3 size 447.14GiB used 121.00GiB path /dev/sdc1/pre>
To force the btrfs volume to mount anyway, the degraded option can be used:
$ mount -t btrfs -o degraded /dev/sda2 /home

The correct thing to do when a disk in an RAID array fails, is to replace it. Once you have a new disk in place notify btrfs about it with the command:

$ btrfs replace start /dev/sdd1 /dev/sdb1 /home

This command reads files both from the original drive (if still accessible) and from other disks in the RAID array, and uses that information to populate the new clean disk.

If needed, the -r flag will prevent the system from trying to read from the outgoing drive if possible. Replacement operations can be canceled, but they cannot be paused. Once the operation is complete, /dev/sdc1 will no longer be a part of the array and can be disposed of.

In some cases one could also run btrfs device delete missing /dev/sdb1 and then add a new drive, but the replace command is the primary command and can be run even if the old drive is completely dead.

Recovering from filesystem corruption in btrfs

In cases where the physical disk has not failed but instead something in the btrfs journal or checksum trees is corrupted and does not match, and the filesystem refuses to mount, this is the recommended procedure to try:

First make a backup the volume.

After that try to mount the volume in the read-only recovery mode:

$ mount -t btrfs -o ro,recovery /dev/sda2 /home

If that fails, look in syslog (or run dmesg) and look for btrfs errors:

[ 74.926506] Btrfs loaded
[ 74.927393] BTRFS: device fsid 4e90ec15-e6f5-470d-96be-677f654a5c79 devid 2 transid 691061 /dev/sdc1
[ 77.439765] BTRFS info (device sdc1): disk space caching is enabled
[ 77.440620] BTRFS: failed to read the system array on sdc1

If there are messages relating to the log tree (not in the example above), then reset the log tree by running:

$ btrfs-zero-log

If syslog shows problems regarding the chunk tree, then btrfs rescue chunk-recover may be of used to replace the chunk blocks with new ones that should work (but may loose some data). Each disk has multiple copies of super blocks, and they are very unlikely to all get corrupted at the same time, but it happens they can be recovered with the rescue command:

$ btrfs rescue super-recover -v /dev/sde1
All Devices:
 Device: id = 3, name = /dev/sdc1
 Device: id = 1, name = /dev/sda2
 Device: id = 2, name = /dev/sde1

Before Recovering:
  [All good supers]:
      device name = /dev/sdc1
      superblock bytenr = 65536

      device name = /dev/sdc1
      superblock bytenr = 67108864

      device name = /dev/sdc1
      superblock bytenr = 274877906944

      device name = /dev/sda2
      superblock bytenr = 65536

      device name = /dev/sda2
      superblock bytenr = 67108864

      device name = /dev/sda2
      superblock bytenr = 274877906944

  [All bad supers]:
      device name = /dev/sde1
      superblock bytenr = 65536

      device name = /dev/sde1
      superblock bytenr = 67108864

      device name = /dev/sde1
      superblock bytenr = 274877906944

After those, try btrfsck, and possibly with options -s1, -s2, -s3. It the volume is still not mountable, then try btrfsck --repair.

The command btrfsck --repair --init-extent-tree may be necessary if the extent tree was corrupted. If there is corruption in the checksums, try –init-csum-tree.

Last resort is to run btrfs check --repair but it’s not recommended because it might write changes to the disk that destroys data.

Generic tools might also be useful. For example the tool testdisk is able to scan disks and find lost partition tables, including ones with btrfs partitions.

Restoring files from a broken btrfs filesystem

If it simply is impossible to mount a btrfs filesystem, it is possible to use the command btrfs restore to fetch files from withing a damaged btrfs partition. The default command will get all files from the root volume.

Sometimes simply restore isn’t enough. For example in Ubuntu, by default the /home directory is a separate btrfs subvolume. To fetch files from there the correct volume root must be defined via the -r option. Also you might not be interested in restoring all possible files, maybe just one particular directory, and for such use a filename filter can be defined with the --path-regex option.

To fetch all files from /home/otto/Kuvat on a system there the @home subvolume object id is 258, the task can be accomplished with the command:

$ btrfs restore -i -vvvv -r 258 --path-regex "^/(otto(|/Kuvat(|/.*)))$" /dev/sdc1 .
...
Restoring ./otto/Kuvat/2015/08/09/IMG_2888.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2889.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2890.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2891.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2892.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2893.JPG
Found objectid=18094703, key=18094702
Done searching /otto/Kuvat/2015/08/09
Found objectid=18094632, key=18094631
Done searching /otto/Kuvat/2015/08
Found objectid=8304076, key=8304075
Done searching /otto/Kuvat/2015
Found objectid=272, key=271
Done searching /otto/Kuvat
Found objectid=258, key=257
Done searching /otto
Found objectid=257, key=256
Done searching

Detecting data corruption

If btrfs detects errors, they will be logged to syslog. Btrfs also maintains error counters, which on normal healthy drives should always list all zeros:

$ btrfs device stats /mnt
[/dev/sda2].write_io_errs   0
[/dev/sda2].read_io_errs    0
[/dev/sda2].flush_io_errs   0
[/dev/sda2].corruption_errs 0
[/dev/sda2].generation_errs 0
[/dev/sdc1].write_io_errs   0
[/dev/sdc1].read_io_errs    0
[/dev/sdc1].flush_io_errs   0
[/dev/sdc1].corruption_errs 0
[/dev/sdc1].generation_errs 0

Btrfs automatically calculates CRC-32C checksums for both data and metadata blocks, and at regular intervals checks if the checksums still match or not. If data corruption is detected, then btrfs will log errors to syslog. If RAID 1 is enabled, btrfs will also automatically fix the corrupted data by overwriting with by the correct duplicate. This process can also be automatically triggered by running btrfs scrub.

Checking disk health

Disks that support the SMART standard are able to report their health status. The Gnome tool ‘disks’ provides a very easy way to access SMART data. Just open Disks, select a device and choose ‘Show SMART Data & Self-Tests’.

Gnome Disks tool Gnome Disks tool SMART data

Alternatively a command line representation of the same data can be fetched with:

$ sudo smartctl -A -H /dev/sda
smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.2.0-90-generic] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0032   098   098   000    Old_age   Always       -       6322
 12 Power_Cycle_Count       0x0032   097   097   000    Old_age   Always       -       2646
175 Program_Fail_Count_Chip 0x0032   100   100   010    Old_age   Always       -       0
176 Erase_Fail_Count_Chip   0x0032   100   100   010    Old_age   Always       -       0
177 Wear_Leveling_Count     0x0013   086   086   010    Pre-fail  Always       -       503
178 Used_Rsvd_Blk_Cnt_Chip  0x0013   094   094   010    Pre-fail  Always       -       184
179 Used_Rsvd_Blk_Cnt_Tot   0x0013   094   094   010    Pre-fail  Always       -       336
180 Unused_Rsvd_Blk_Cnt_Tot 0x0013   094   094   010    Pre-fail  Always       -       6192
181 Program_Fail_Cnt_Total  0x0032   100   100   010    Old_age   Always       -       0
182 Erase_Fail_Count_Total  0x0032   100   100   010    Old_age   Always       -       0
183 Runtime_Bad_Block       0x0013   100   100   010    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
195 Hardware_ECC_Recovered  0x001a   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   253   253   000    Old_age   Always       -       0
232 Available_Reservd_Space 0x0013   094   094   000    Pre-fail  Always       -       3080
241 Total_LBAs_Written      0x0032   099   099   000    Old_age   Always       -       15812437286
242 Total_LBAs_Read         0x0032   099   099   000    Old_age   Always       -       11879385910

Storing data efficiently and dependably are fundamental tasks in computing. Data integrity and data availability are key principles in information security. Btrfs is a promising technology that will help system administrators fulfill these goals and in particular with SSD disks btrfs should be the default filesystem choice for all new systems, and a must for syadmins to read up on.

Written by Otto Kekäläinen

December 9th, 2015 at 7:31 am

Using RAID with btrfs and recovering from broken disks

without comments

Btrfs logoBtrfs (pronounced Better FS) is a relatively new filesystem that operates on the copy-on-write principle (abbreviated COW, which stems a more friendly pronunciation for btrfs: Butter FS). Btrfs includes a lot of interesting functionality and replaces traditional Linux disk and filesystem tools like LVM (volume manager, disk snapshots) and mdadm (software RAID).

In RAID usage btrfs is much more flexible and space efficient than traditional mdadm, because in btrfs the disks in the RAID array do not conform to any predefined size or count requirements. You can attach any amount of disk of any size to a btrfs RAID array, and btrfs will automatically balance the data across the devices according to the requirements of the selected RAID level. RAID levels 0 and 1 are currently supported while RAID 5 and 6 are under development and will be available as officially supported configurations soon.

Let’s take an example case to see how btrfs RAID works. In traditional mdadm based RAID if you have two 1 TB disks configured to mirror each other in RAID 1 mode, and you want to expand this setup, you basically need to install two more disks, and they need to be of equal size. In btrfs, life is much more flexible.

To expand an array, you can add a disk of any size and btrfs will automatically adjust to the sitation. So if you have two 1 TB disks, and you add a 2 TB disk to the array, then all 4 TB would be utilised and you will have 2 TB of usable storage. Btrfs will make sure that as RAID 1 mandates, each file will exist in two copies and those copies are stored on physically different disks. If you would expand the two 1 TB array with one 1 TB disk, then btrfs would give you 1,5 TB of usable disk space in total while still satisfying the RAID 1 replicate requirements.

Btrfs RAID is flexible also in the sense that you can either create the RAID array by running something like mkfs.btrfs -d raid1 -m raid1 /dev/sdb1 /dev/sdc1 when you create the partitions, or you can at any later time convert existing partitions to use RAID by running a command like btrfs balance start -dconvert=raid1 -mconvert=raid1 /home

If RAID is active, it will be visible in the btrfs filesystem df command:

$ btrfs fi df /home
Data, RAID1: total=1.59TiB, used=1.59TiB
System, RAID1: total=32.00MiB, used=256.00KiB
Metadata, RAID1: total=10.00GiB, used=8.27GiB
unknown, single: total=512.00MiB, used=0.00

Keep in mind, that the free values shown by the normal df are not reliable, as it is hard to predict how disk usage will behave in a copy-on-write and snapshotting filesystem like btrfs.

Recovering from failed disks in btrfs

If a disk in a btrfs RAID 1 array fails, then btrfs will refuse to mount that filesystem and error messages will be visible in the syslog. If it was the root filesystem, then the system will refuse to boot normally and the system will usually boot to an initramfs console. Luckily all decent systems that support btrfs (like Ubuntu 14.04) will have btrfs tools included in the initramfs environment, so you can run btrfs commands from there and try to recover from the situation without the need to boot the system form an alternative media, like a live CD.

A btrfs volume with a failed (missing) disk will output something like this:

$ btrfs fi show
Label: none  uuid: 4e90ec15-e6f5-470d-96be-677f654a5c79
    Total devices 3 FS bytes used 1.59TiB
    devid    1 size 2.71TiB used 1.60TiB path /dev/sda1
    devid    2 size 1.80TiB used 1.48TiB path 
    devid    3 size 447.14GiB used 121.00GiB path /dev/sdc1/pre>
To force the btrfs volume to mount anyway, the degraded option can be used:
$ mount -t btrfs -o degraded /dev/sda2 /home

The correct thing to do when a disk in an RAID array fails, is to replace it. Once you have a new disk in place notify btrfs about it with the command:

$ btrfs replace start /dev/sdd1 /dev/sdb1 /home

This command reads files both from the original drive (if still accessible) and from other disks in the RAID array, and uses that information to populate the new clean disk.

If needed, the -r flag will prevent the system from trying to read from the outgoing drive if possible. Replacement operations can be canceled, but they cannot be paused. Once the operation is complete, /dev/sdc1 will no longer be a part of the array and can be disposed of.

In some cases one could also run btrfs device delete missing /dev/sdb1 and then add a new drive, but the replace command is the primary command and can be run even if the old drive is completely dead.

Recovering from filesystem corruption in btrfs

In cases where the physical disk has not failed but instead something in the btrfs journal or checksum trees is corrupted and does not match, and the filesystem refuses to mount, this is the recommended procedure to try:

First make a backup the volume.

After that try to mount the volume in the read-only recovery mode:

$ mount -t btrfs -o ro,recovery /dev/sda2 /home

If that fails, look in syslog (or run dmesg) and look for btrfs errors:

[ 74.926506] Btrfs loaded
[ 74.927393] BTRFS: device fsid 4e90ec15-e6f5-470d-96be-677f654a5c79 devid 2 transid 691061 /dev/sdc1
[ 77.439765] BTRFS info (device sdc1): disk space caching is enabled
[ 77.440620] BTRFS: failed to read the system array on sdc1

If there are messages relating to the log tree (not in the example above), then reset the log tree by running:

$ btrfs-zero-log

If syslog shows problems regarding the chunk tree, then btrfs rescue chunk-recover may be of used to replace the chunk blocks with new ones that should work (but may loose some data). Each disk has multiple copies of super blocks, and they are very unlikely to all get corrupted at the same time, but it happens they can be recovered with the rescue command:

$ btrfs rescue super-recover -v /dev/sde1
All Devices:
 Device: id = 3, name = /dev/sdc1
 Device: id = 1, name = /dev/sda2
 Device: id = 2, name = /dev/sde1

Before Recovering:
  [All good supers]:
      device name = /dev/sdc1
      superblock bytenr = 65536

      device name = /dev/sdc1
      superblock bytenr = 67108864

      device name = /dev/sdc1
      superblock bytenr = 274877906944

      device name = /dev/sda2
      superblock bytenr = 65536

      device name = /dev/sda2
      superblock bytenr = 67108864

      device name = /dev/sda2
      superblock bytenr = 274877906944

  [All bad supers]:
      device name = /dev/sde1
      superblock bytenr = 65536

      device name = /dev/sde1
      superblock bytenr = 67108864

      device name = /dev/sde1
      superblock bytenr = 274877906944

After those, try btrfsck, and possibly with options -s1, -s2, -s3. It the volume is still not mountable, then try btrfsck --repair.

The command btrfsck --repair --init-extent-tree may be necessary if the extent tree was corrupted. If there is corruption in the checksums, try –init-csum-tree.

Last resort is to run btrfs check --repair but it’s not recommended because it might write changes to the disk that destroys data.

Generic tools might also be useful. For example the tool testdisk is able to scan disks and find lost partition tables, including ones with btrfs partitions.

Restoring files from a broken btrfs filesystem

If it simply is impossible to mount a btrfs filesystem, it is possible to use the command btrfs restore to fetch files from withing a damaged btrfs partition. The default command will get all files from the root volume.

Sometimes simply restore isn’t enough. For example in Ubuntu, by default the /home directory is a separate btrfs subvolume. To fetch files from there the correct volume root must be defined via the -r option. Also you might not be interested in restoring all possible files, maybe just one particular directory, and for such use a filename filter can be defined with the --path-regex option.

To fetch all files from /home/otto/Kuvat on a system there the @home subvolume object id is 258, the task can be accomplished with the command:

$ btrfs restore -i -vvvv -r 258 --path-regex "^/(otto(|/Kuvat(|/.*)))$" /dev/sdc1 .
...
Restoring ./otto/Kuvat/2015/08/09/IMG_2888.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2889.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2890.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2891.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2892.JPG
Restoring ./otto/Kuvat/2015/08/09/IMG_2893.JPG
Found objectid=18094703, key=18094702
Done searching /otto/Kuvat/2015/08/09
Found objectid=18094632, key=18094631
Done searching /otto/Kuvat/2015/08
Found objectid=8304076, key=8304075
Done searching /otto/Kuvat/2015
Found objectid=272, key=271
Done searching /otto/Kuvat
Found objectid=258, key=257
Done searching /otto
Found objectid=257, key=256
Done searching

Detecting data corruption

If btrfs detects errors, they will be logged to syslog. Btrfs also maintains error counters, which on normal healthy drives should always list all zeros:

$ btrfs device stats /mnt
[/dev/sda2].write_io_errs   0
[/dev/sda2].read_io_errs    0
[/dev/sda2].flush_io_errs   0
[/dev/sda2].corruption_errs 0
[/dev/sda2].generation_errs 0
[/dev/sdc1].write_io_errs   0
[/dev/sdc1].read_io_errs    0
[/dev/sdc1].flush_io_errs   0
[/dev/sdc1].corruption_errs 0
[/dev/sdc1].generation_errs 0

Btrfs automatically calculates CRC-32C checksums for both data and metadata blocks, and at regular intervals checks if the checksums still match or not. If data corruption is detected, then btrfs will log errors to syslog. If RAID 1 is enabled, btrfs will also automatically fix the corrupted data by overwriting with by the correct duplicate. This process can also be automatically triggered by running btrfs scrub.

Checking disk health

Disks that support the SMART standard are able to report their health status. The Gnome tool ‘disks’ provides a very easy way to access SMART data. Just open Disks, select a device and choose ‘Show SMART Data & Self-Tests’.

Gnome Disks tool Gnome Disks tool SMART data

Alternatively a command line representation of the same data can be fetched with:

$ sudo smartctl -A -H /dev/sda
smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.2.0-90-generic] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0032   098   098   000    Old_age   Always       -       6322
 12 Power_Cycle_Count       0x0032   097   097   000    Old_age   Always       -       2646
175 Program_Fail_Count_Chip 0x0032   100   100   010    Old_age   Always       -       0
176 Erase_Fail_Count_Chip   0x0032   100   100   010    Old_age   Always       -       0
177 Wear_Leveling_Count     0x0013   086   086   010    Pre-fail  Always       -       503
178 Used_Rsvd_Blk_Cnt_Chip  0x0013   094   094   010    Pre-fail  Always       -       184
179 Used_Rsvd_Blk_Cnt_Tot   0x0013   094   094   010    Pre-fail  Always       -       336
180 Unused_Rsvd_Blk_Cnt_Tot 0x0013   094   094   010    Pre-fail  Always       -       6192
181 Program_Fail_Cnt_Total  0x0032   100   100   010    Old_age   Always       -       0
182 Erase_Fail_Count_Total  0x0032   100   100   010    Old_age   Always       -       0
183 Runtime_Bad_Block       0x0013   100   100   010    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
195 Hardware_ECC_Recovered  0x001a   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   253   253   000    Old_age   Always       -       0
232 Available_Reservd_Space 0x0013   094   094   000    Pre-fail  Always       -       3080
241 Total_LBAs_Written      0x0032   099   099   000    Old_age   Always       -       15812437286
242 Total_LBAs_Read         0x0032   099   099   000    Old_age   Always       -       11879385910

Storing data efficiently and dependably are fundamental tasks in computing. Data integrity and data availability are key principles in information security. Btrfs is a promising technology that will help system administrators fulfill these goals and in particular with SSD disks btrfs should be the default filesystem choice for all new systems, and a must for syadmins to read up on.

Written by Otto Kekäläinen

December 9th, 2015 at 7:31 am

Android Phone Review: The OnePlus X

without comments

OnePlus is a mobile phone manufacturer famous for selling the OnePlus One with pre-installed CyanogenMod, instead of a bloated custom Android as most manufacturers do. Their newest model OnePlus X was released on November 5th 2015 and after a few days of use, it seems to live up to its promises.

OnePlus X

OnePlus X

Most Chinese brands suffer from the lack of finishing and low quality. OnePlus is something completely different. Everything about OnePlus seems different. It is like a completely new generation. The One Plus website is exiting. Their marketing is based almost entirely on using social media – and in a good way! The device we ordered from China’s Silicon Valley Shenzhen arrived in just a few business days. The packaging had a premium feel to it and what was inside matched the expectations set by the hype on their website.

Packaging includes a silicon case for added protection

Packaging includes a silicon case for added protection.

Excellent craftsmanship with shiny glass-like panels in front and back, connected by a metallic bevel. A vivid display and a responsive and fast Android 5.1.1 experience inside. Dual SIM card capability, great camera and 3 GB of RAM are just some of the high-end technical features. There is no need to repeat those details, as they are already well presented at the original site. All we need to say is that those promises are true and the quality is unexpectedly good. It definitely competes with the other high-end mobile phones in the range of 500-700 euros. And at what price is the OnePlus X available? Only 269 € in Europe, including taxes and tolls.

The only drawbacks we noted are related to problems in Android itself. Everything in what OnePlus has added and customized is done with good judgement and is a step forward.

Very fast and snappy UI

Very fast and snappy UI

 

Inside is Android 5.1.1

Inside is Android 5.1.1

For a Finns like us it was also delightful to notice that their website is also available in Finnish, and that the language is actually flawless and not an amateur translation. Naturally the device operating system Android also has Finnish as an option.

Have any of you already experienced the OnePlus X? Feel free to share your thoughts in the comments!

Written by Otto Kekäläinen

November 11th, 2015 at 12:05 pm

Android Phone Review: The OnePlus X

without comments

OnePlus is a mobile phone manufacturer famous for selling the OnePlus One with pre-installed CyanogenMod, instead of a bloated custom Android as most manufacturers do. Their newest model OnePlus X was released on November 5th 2015 and after a few days of use, it seems to live up to its promises.

OnePlus X

OnePlus X

Most Chinese brands suffer from the lack of finishing and low quality. OnePlus is something completely different. Everything about OnePlus seems different. It is like a completely new generation. The One Plus website is exiting. Their marketing is based almost entirely on using social media – and in a good way! The device we ordered from China’s Silicon Valley Shenzhen arrived in just a few business days. The packaging had a premium feel to it and what was inside matched the expectations set by the hype on their website.

Packaging includes a silicon case for added protection

Packaging includes a silicon case for added protection.

Excellent craftsmanship with shiny glass-like panels in front and back, connected by a metallic bevel. A vivid display and a responsive and fast Android 5.1.1 experience inside. Dual SIM card capability, great camera and 3 GB of RAM are just some of the high-end technical features. There is no need to repeat those details, as they are already well presented at the original site. All we need to say is that those promises are true and the quality is unexpectedly good. It definitely competes with the other high-end mobile phones in the range of 500-700 euros. And at what price is the OnePlus X available? Only 269 € in Europe, including taxes and tolls.

The only drawbacks we noted are related to problems in Android itself. Everything in what OnePlus has added and customized is done with good judgement and is a step forward.

Very fast and snappy UI

Very fast and snappy UI

 

Inside is Android 5.1.1

Inside is Android 5.1.1

For a Finns like us it was also delightful to notice that their website is also available in Finnish, and that the language is actually flawless and not an amateur translation. Naturally the device operating system Android also has Finnish as an option.

Have any of you already experienced the OnePlus X? Feel free to share your thoughts in the comments!

Written by Otto Kekäläinen

November 11th, 2015 at 12:05 pm

Fixing black screen after login in Ubuntu 14.04

without comments

How to fix black screen after login in Ubuntu 14.04?

(Ohje suomeksi lopussa.)

A lot of Linux-support customers have contacted us recently asking to fix their Ubuntu laptops and workstations that suddently stopped working. The symptom is that after entering the username and password in the login screen, they are unable to get in. Instead they see a flickering screen that then goes all black for a while, and then returns back to the login screen.

This problem is caused by an update that didn’t install cleanly and left the graphical desktop environment in a broken state.

The fix is to open a text console by pressing Ctrl+Alt+F1 and then logging in in text mode. Once in, issue these commands to complete the upgrade successfully:

sudo dpkg --configure -a
sudo apt-get update
sudo apt-get upgrade -y

Finally run sudo reboot, Ubuntu restarts, and then you can log in again normally.

Ubuntu_logo_orange

Sisäänkirjautumisen jälkeen näkyvän mustan ruudun korjaaminen Ubuntu 14.04:ssä

Useat Linux-tuki.fi:n asiakkaat ovat viime päivinä ottaneet meihin yhteyttä tilatakseen tukea Ubuntu-läppärin tai -työaseman korjaukseen, kun se yllättäen lakkasi toimimasta oikein. Oire on, että sisäänkirjautumisessa, käyttäjätunnuksen ja salasanan syöttämisen jälkeen ruutu vilkkuu ja on hetken musta. Tämän jälkeen näyttö tulee takaisin kirjautumisnäkymään.

Ongelma johtuu Ubuntun päivityksestä, joka on epäonnistunut ja jättänyt graafisen työpöytäympäristön toimimattomaan tilaan.

Korjauksen voi tehdä itse avaamalla tekstipäätteen painamalla Ctrl+Alt+F1 ja kirjautumalla sisään tekstitilassa. Sen jälkeen voi ajaa päivityksen loppuun onnistuneesti komentamalla:

sudo dpkg --configure -a
sudo apt-get update
sudo apt-get upgrade -y

Lopuksi aja sudo reboot. Ubuntun uudelleenkäynnistymisen jälkeen sisäänkirjautuminen ja käyttö pitäisi onnistua normaalisti.

Written by Sanna Saarikangas

November 5th, 2015 at 2:42 am