LinuxPlanet Casts

Media from the Linux Moguls

Archive for the ‘snapshot’ Category

Ultimate ZFS Overview | TechSNAP 28

without comments

post thumbnail

Coming up on this week’s TechSNAP…

Buckle up and prepare for our Ultimate ZFS overview!

Plus, the next generation of Stuxnet is in the wild, but this time is laying low, collecting data.

All that and more, on this week’s TechSNAP!

Direct Download Links:

HD Video | Large Video | Mobile Video | MP3 Audio | OGG Audio | YouTube

Subscribe via RSS and iTunes:

Show Notes:

Jupiter Broadcasting Gear

  • Coupon Code: SuperDuperShip – Free Shipping on Super Saver, International, and Canadian Airmail orders. No minimums
  • Coupon Code: SuperSave$10 – $10 off orders with a subtotal of $50+
  • Coupon Code: Scary35% – 35% off orders with a subtotal of $100+

Next generation of Stuxnet seen in the wild?

  • Called Duqu, the malware appears to be based on the same concepts as Stuxnet, and likely was written by some of the same people, or someone with access to the Stuxnet source code.
  • The malware is designed to be stealthy and silent, rather than exploiting the system to some gain, like most malware
  • The rootkit loads it self as a validly signed driver. It appears to have been signed by the certificate of a company in Taiwan identified as C-Media Electronics Incorporation. It is possible that their systems were compromised and their private key is being used without their knowledge. The certificate was set to expire on August 2, 2012, but authorities revoked it on Oct. 14
  • The malware is not a worm, as it does it spread, and has no destructive payload
  • It appears to only gather intelligence and act as a espionage agent, collecting data to be used a future attack.
  • Analysts claim it appears to be seeking information on an unidentified industrial control system
  • Duqu appears to have been in operation, undetected for more than a year
  • Symantec has declined to name the countries where the malware was found, or to identify the specific industries infected, other than to say they are in the manufacturing and critical infrastructure sectors
  • Duqu analysis paper

Google switching to SSL for logged in users’ searches

  • Users who do a search while logged in, will do the search over SSL, meaning their search query and the results will be protected from snooping by their ISP, Government, Law Enforcement and WiFi hackers.
  • This is an important step as google works to personalize your search results more and more.
  • An interesting side effect of this is that browsers do not pass referrer headers when you transition from an SSL site. So the sites you visit from the search results page will no longer see what your search query was. Clicks on Adwords and other sponsored links will still pass your search query.
  • The primary impediment to SSL for everything is performance, encrypting all traffic on the web would require a great deal more hardware. This is why Google defaults to a weaker encryption for things like search results, than what online merchants typically use.
  • Another impediment to SSL is the certificate system, typical setups require a unique IP for each SSL certificate (because the name based virtual hosting typically done by web servers relies on an HTTP header, that is not sent until after the encryption session is started). However modern browsers and web servers support ‘SNI’ (Server Name Indication) to allow that information to be passed as part of the initial encryption setup. There are also solutions such as wildcard certificates (ie, * and Unified Communications Certificates (UCC, typically used for MS Exchange servers and the like).
  • Google will also provide website owners with the top 1000 search queries that lead visitors to their site via Google Webmaster Tools.
  • HTTPS Everywhere | Electronic Frontier Foundation


ZFS Segment

  • This week we will be taking a look at ZFS as a storage solution
  • ZFS was originally developed by Sun Microsystems to be able to store a zetta byte of data (A zetta byte is equal to 1 billion tera bytes)
  • ZFS is both the Volume Manager and the File System. This gives it some unique benefits, including the ability to increase the size of the file system on the fly and improves performance for the ‘scrub’ (integrity check all data) and resilver (recover from a failed disk) operations, as only data blocks that are actually in use need to be rewritten, whereas a hardware RAID controller must resilver the entire disk because it is unaware of the file system.
  • ZFS is a ‘Copy-On-Write’ file system, this means that data is not immediately overwritten when it is changed
  • Features
    • Multiple mount points – You can create various mount points from the same storage pool, allowing you to have different settings for different types of files.
    • Passive Integrity Checking (Fletcher Checksum or SHA–2) – As data is read, it is compared against the checksum (or hash, depending on settings). If the data is found to be corrupted, ZFS attempts to recover it (from a mirrored device, RAID Z, or copies). This feature allows ZFS to detect silent corruption that normally goes unnoticed.
    • RAID Z – RAID Z works very similar to RAID 5, except without the requirement for a hardware RAID controller. RAID Z2 provides two parity drives, like RAID 6. Recently, RAID Z3 was also introduced, using 3 drives for parity, providing exceptional fault tolerance.
    • Compression – Allow you to compress the data stored in this mount point (defaults to lzjb for speed, or you can choose a specific level of gzip). This can be great for storing highly compressible information such as log files
    • Deduplication – Since ZFS already knows the hash of your files as it writes them, it can detect that a file with the identical content already exists in your storage pool, and it will simply link the new file to the old one, and because ZFS is copy-on-write, if either file changes, it does not effect the other. ZFS also supports an optional ‘verify’ setting, where even if the checksum/hash matches, it will do a byte-by-byte verification to ensure the files are the same, to avoid a cache collision resulting in data corruption, even though the chances of this happening are around 10^–77. Deduplication uses a lot of ram, so it is recommended that you only use it on datasets where there is a high probability of duplication (It requires 320 bytes per block, meaning 1TB of data in 8kb blocks requires 32GB of ram. ZFS allows blocks up to 128kb). Deduplication will only use up to 25% of ARC memory, after that performance is degraded.
    • Purposeful Duplication (Copies) – Allows you to ask ZFS to maintain more than 1 copy of each file in a mount point. This is in addition to any redundancy provided by mirrors/RAID Z etc. Where possible the additional copies are stored on different physical devices. This allows you to get the benefit of a system like RAID Z but only for a specific set of data, while using regular striping for the rest, to maximize your storage capacity. (The ‘Copies’ system was not designed to protect against entire drives failing, just the loss of specific sectors, also this setting only effects newly created files, so you should set it when you create the mount point)
    • Snapshots – A read only copy of the file system from a specific point in time, great for backups etc.
    • Clones – A writable snapshot. Allows you to create a second copy of the file system that shares all of the same disk space, and any changes to either the original or the clone get saved separately.
    • Dynamic Striping – As you add more disks to your ZFS pool, the strips are automatically adjusted to take advantage of the write performance of all available disks.
    • Space Reservation – Since all mount points share the same pool of free space, you can set reservations to make sure specific mount points always have access to free space, even if another mount point is trying to use all of the space.
  • In summary, ZFS can be a great solution for your home file server, as it allows you the flexibility to add additional storage at any time, deduplicate files, provided limited redundancy without needing RAID and can even provide some Drobo like functionality.
  • If you keep at least one SATA port available in your file server, you can replace smaller devices by attaching the newer drive, and using the ‘zpool replace’ command, to copy all of the data to the new device, then remove the smaller one. You can eventually replace every device in the system this way, and the storage pool sizes up automatically.
  • RAID Z pools cannot currently have devices added to them, although this feature is in the works. If you create a RAID Z (or Z2/Z3) pool, you can still increase it’s storage capacity by replacing each disk one at a time, and waiting for it to resilver (unlike in non-redundant setups, you do not have to connect the new device before removing the old one). Again, because ZFS is both the Volume Manager and the File System, the resilvering process is faster, because only data that is actually in use needs to be written to the new device.

Round Up:

Ultimate Backups | TechSNAP 26

without comments

post thumbnail

We’ll tell you about AT&T leaving Android open to a hack so easy, my two year old son could pull it off. Plus FireFox goes to battle with McAfee and is Bank of America Under attack?

Then – We delve into backups, from the fundamentals to the very best tools!

All that and more, in this week’s TechSNAP!

Direct Download Links:

HD Video | Large Video | Mobile Video | WebM | MP3 Audio | OGG Audio | YouTube

Subscribe via RSS and iTunes:

Show Notes:

Security hole in AT&T Samsung Galaxy S II

  • Bug allows someone to bypass the security lockout screen, accessing the phone without the password
  • The flaw does not exist on the Sprint version of the Samsung Galaxy S , or the Epic Touch 4G
  • By pressing the lock button to wake the phone, and you will be prompted with the unlock screen. Allow the phone to go back to sleep, and immediately tap the lock button again, and you will have access to the phone
  • This feature is likely designed for the situation where you are waiting for some interaction on the phone and it falls asleep, if you press a button to wake it within a few seconds, it doesn’t prompt you to reunlock the phone. This is a useful feature, however, it should be predicated on the fact that you just recently unlocked the phone (don’t make me unlock the phone twice within 90 seconds, or something similar)
  • The flaw only effects phones that have been unlocked once since boot
  • Since the flaw only effects the AT&T version of the phone, it would seem it is based on software added to the phone by AT&T, which appears to cache your response to the unlock screen, and use it to bypass the screen when you re-wake the phone immediately after it goes to sleep.
  • Another example of the vendors messing with the core google product.
  • Users with Microsoft Exchange security policies don’t seem to be affected
  • Users can adjust the settings on their phone by accessing: Settings ->Location and Security->Screen unlock settings->Timeout and setting the value to Immediately, disabling the ‘feature’ that presents the vulnerablity.

Firefox advises users to disable McAfee Plugin

  • Firefox says the McAfee ScanScript plugin causes Stability and Security problems
  • The problem only seems to effect the new Firefox 7, it is likely caused by a compatibility problem with versions of ScanScript designed for older versions of Firefox
  • Firefox has started generating popup warnings to users using versions of McAfee older than 14.4.0 due to an incredibly high volume of crash reports
  • McAfee says it is working with Firefox to solve the issue for the next version of the software
  • McAfee is very popular in corporate environments and is often enforced with a Active Directory Group Policy that makes it nearly impossible for the end user to disable the virus scanner

Bank of America – Unexplained Outages – Is it an attack?

  • The Bank of America website has been degraded, slow, returning errors or down for more than 6 days
  • Bank of America (BofA) said its Web and mobile services have not been hit by hacking or denial-of-service attacks, however they would not disclose what has been causing the online problems.
  • Quote: “I just want to be really clear. Every indication [is that] recent performance issues have not been the result of hacking, malware or denial of service,” said BofA spokeswoman Tara Burke. “We’ve had some intermittent or sporadic slowness. We don’t break out the root cause.”
  • The problems began Friday morning, a day after BofA announced it would charge a $5 monthly fee for account holders using their debit cards
  • Additional Coverage


Continuing our Home Server Segment – This week we are covering backups.
Before we cover some of the solutions, we should look at some of the concepts and obstacles to creating proper backups. There are a number of different ways to back things up, but the most popular involves using multiple ‘levels’ of backup.

  • Full backup

  • This is a backup of every file (or a specific subset, or without specific exclusions) on a system.

  • This is the base of higher level backups, and is also known as a level 0 backup

  • Full backups are the biggest and take the slowest

  • Differential Backup

  • A differential backup is one that includes every file that has changed since the last full backup was started (this is important).

  • >It is very important the higher level backups always be based on the START time of the lower level backup, rather than the last modified, or finish time. During the last backup, if the file changed after it was backed up, but before that backup completed, we want to be sure to include it in the next backup

  • Differential backups require only the most recent full backup to restore

  • Incremental Backup

  • An incremental backup consists of every file that has changed since the start of the last backup of any level

  • Incremental backups are the smallest and fastest

  • Incremental backups can take the longest to restore, and can require access to each of the previous differential backups since last full backup, and that most recent full backup

  • Incremental backups offer the trade off, they take less time and less storage, however they slow the recovery process.

  • Incremental backups, due to their smaller size, make it easier to have ‘point of time’ backups of files, rather than just the most recent.

  • Some backup systems do away with the name designations, and allow even more granularity

  • A level 0 backup is a full backup

  • A level 1 is everything that has changed since the level 0

  • A level n is everything that has changed since the last level n–1 or higher

  • Systems such as the unix ‘dump’ utility, allow up level 9 backups

  • Some backup systems, such as Bacula, support ‘synthetic full backups’

  • A synthetic backup is when you use a full backup, plus more recent differential and incremental backups to create a new, more recent full backup.

  • This can be especially advantageous in remote and off site backup systems, where transferring the full data set over the network can be very slow and costly.

  • rsync

  • Not actually a backup tool, it just creates and synchronizes a copy of the files

  • Copies only the changes to the files, so is faster

  • snapshots

  • A point in time copy of the files in a filesystem (supported by LVM, UFS, ZFS, etc)

  • A good place to take a backup from, resolves issues with open files

  • bacula

  • Designed to backup a large number of machines

  • Quite a bit of setup (Directory, Storage Daemon, SQL Database, File Daemons (Clients))

  • Cross platform

  • Powerful deduplication system, and ‘base backups’

  • Support for Windows Volume Shadow Copy (snapshots of open files)

  • flexbackup

  • simple perl script that creates archives (tar, cpio, etc) with optional compression (gzip, bzip2, etc).

  • Uses the ‘find’ command to create multi-level backups based on modified date

  • backupmypc

  • rsync based

  • Supports FTP, SCP, RCP, & SMB for Windows

  • s very smart about how it handles portable devices that miss backups.

  • It’s magic is it’s de-dupe hard-link mojo that saves tons of space

  • Bit of a nerd project to get going, but is bullet proof once its in

  • TarSnap – BSD Encrypted Cloud Backup

  • Mondo Rescue – GPL disaster recovery solution

  • CrashPlan – Online Backup Software, Disaster Recovery

  • Allan’s article about backups

Round Up:

Jupiter Broadcasting stats

  1. Firefox 42.66%
  2. Chrome 29.73%
  3. Internet Explorer 14.43%