LinuxPlanet Blogs

By Linux Geeks, For Linux Geeks.

Twitter Adds Group Messaging and Native Video

without comments

By Vasudev Ram

Saw this via a tweet.

http://www.programmableweb.com/news/twitter-adds-group-messaging-and-native-video/elsewhere-web/2015/01/27


Should be interesting to check it out once ready. They say the group messaging feature will allow for messaging to small groups of up to 20 at a time. Could be useful for some applications.

- Vasudev Ram - Dancing Bison Enterprises

Signup to hear about my new software products.


Contact Page


Written by Vasudev Ram

January 30th, 2015 at 11:33 am

Twitter Adds Group Messaging and Native Video

without comments

Written by Vasudev Ram

January 30th, 2015 at 6:23 am

Posted in Uncategorized

HTML text to PDF with Beautiful Soup and xtopdf

without comments

By Vasudev Ram



Recently, I thought of getting the text from HTML documents and putting that text to PDF. So I did it :)

Here's how:

"""
HTMLTextToPDF.py
A demo program to show how to convert the text extracted from HTML
content, to PDF. It uses the Beautiful Soup library, v4, to
parse the HTML, and the xtopdf library to generate the PDF output.
Beautiful Soup is at: http://www.crummy.com/software/BeautifulSoup/
xtopdf is at: https://bitbucket.org/vasudevram/xtopdf
Guide to using and installing xtopdf: http://jugad2.blogspot.in/2012/07/guide-to-installing-and-using-xtopdf.html
Author: Vasudev Ram - http://www.dancingbison.com
Copyright 2015 Vasudev Ram
"""

import sys
from bs4 import BeautifulSoup
from PDFWriter import PDFWriter

def usage():
sys.stderr.write("Usage: python " + sys.argv[0] + " html_file pdf_file\n")
sys.stderr.write("which will extract only the text from html_file and\n")
sys.stderr.write("write it to pdf_file\n")

def main():

# Create some HTML for testing conversion of its text to PDF.
html_doc = """
<html>
<head>
<title>
Test file for HTMLTextToPDF
</title>
</head>
<body>
This is text within the body element but outside any paragraph.
<p>
This is a paragraph of text. Hey there, how do you do?
The quick red fox jumped over the slow blue cow.
</p>
<p>
This is another paragraph of text.
Don't mind what it contains.
What is mind? Not matter.
What is matter? Never mind.
</p>
This is also text within the body element but not within any paragraph.
</body>
</html>
"""

pw = PDFWriter("HTMLTextTo.pdf")
pw.setFont("Courier", 10)
pw.setHeader("Conversion of HTML text to PDF")
pw.setFooter("Generated by xtopdf: http://slid.es/vasudevram/xtopdf")

# Use method chaining this time.
for line in BeautifulSoup(html_doc).get_text().split("\n"):
pw.writeLine(line)
pw.savePage()
pw.close()

if __name__ == '__main__':
main()


The program uses the Beautiful Soup library for parsing and extracting information from HTML, and xtopdf, my Python library for PDF generation.
Run it with:
python HTMLTextToPDF.py
and the output will be in the file HTMLTextTo.pdf.
Screenshot below:


- Vasudev Ram - Python training and programming - Dancing Bison Enterprises

Read more of my posts about Python or read posts about xtopdf (latter is subset of former)


Signup to hear about my new software products or services.


Contact Page

Android Version Stats for LQ Mobile (2015)

without comments

With the recent news that Google will not patch the WebView vulnerability in versions of Android <= 4.3, I thought it would be a good time to look at the Android version stats for LQ Mobile. You can see stats from seven months ago here.

Platform Version
Android 4.4 33.14%
Android 4.1 16.82%
Android 4.2 11.18%
Android 4.0.3 – 4.0.4 10.11%
Android 2.3.3-2.3.7 9.69%
Android 5.0 9.44%
Android 4.3 6.96%
Android 2.2 1.82%

So, how has the Android version landscape changed since the last post and what are the implications of the WebView vulnerability in that context? Android 4.4 is still the most common version, with over a third of the market. Versions 4.2 and 4.3 are still common, but less so than previously. Versions 4.0.3/4.0.3 and 2.3.x are both very old and still fairly popular with roughly 10% each. That’s disappointing. Lollipop adoption among LQ Mobile users is significantly higher than Google is seeing generally (still less than .1%) which isn’t surprising given the technical nature of LQ members. Even with that advantage, however, roughly half of LQ Mobile users are using a version of Android that’s vulnerable. Given that data, it’s easy to understand why Google has broken out quite a bit of functionality/code into Google Play Services, which they can update independently of handset manufacturers and carriers

–jeremy


Written by jeremy

January 26th, 2015 at 12:24 pm

PrettyTable to PDF is pretty easy with xtopdf

without comments

By Vasudev Ram




"PrettyTable to PDF is pretty easy with xtopdf."

How's that for some alliteration? :)

PrettyTable is a Python library to help you generate nice tables with ASCII characters as the borders, plus alignment of text within columns, headings, padding, etc.

Excerpt from the site:

[ PrettyTable is a simple Python library designed to make it quick and easy to represent tabular data in visually appealing ASCII tables.
...
PrettyTable lets you control many aspects of the table, like the width of the column padding, the alignment of text within columns, which characters are used to draw the table border, whether you even want a border, and much more. You can control which subsets of the columns and rows are printed, and you can sort the rows by the value of a particular column.

PrettyTable can also generate HTML code with the data in a <table> structure. ]

I came across PrettyTable via this blog post:

11 Python Libraries You Might Not Know.

Then I thought of using it with my PDF creation toolkit, to generate such ASCII tables, but as PDF. Here's a program, PrettyTableToPDF.py, that shows how to do that:
"""
PrettyTableToPDF.py
A demo program to show how to convert the output generated
by the PrettyTable library, to PDF, using the xtopdf toolkit
for PDF creation from other formats.
Author: Vasudev Ram - http://www.dancingbison.com
xtopdf is at: http://slides.com/vasudevram/xtopdf

Copyright 2015 Vasudev Ram
"""

from prettytable import PrettyTable
from PDFWriter import PDFWriter

pt = PrettyTable(["City name", "Area", "Population", "Annual Rainfall"])
pt.align["City name"] = "l" # Left align city names
pt.padding_width = 1 # One space between column edges and contents (default)
pt.add_row(["Adelaide",1295, 1158259, 600.5])
pt.add_row(["Brisbane",5905, 1857594, 1146.4])
pt.add_row(["Darwin", 112, 120900, 1714.7])
pt.add_row(["Hobart", 1357, 205556, 619.5])
pt.add_row(["Sydney", 2058, 4336374, 1214.8])
pt.add_row(["Melbourne", 1566, 3806092, 646.9])
pt.add_row(["Perth", 5386, 1554769, 869.4])
lines = pt.get_string()

pw = PDFWriter('Australia-Rainfall.pdf')
pw.setFont('Courier', 12)
pw.setHeader('Demo of PrettyTable to PDF')
pw.setFooter('Demo of PrettyTable to PDF')
for line in lines.split('\n'):
pw.writeLine(line)
pw.close()

You can run the program with:
$ python PrettyTableToPDF.py
And here is a screenshot of the output PDF in Foxit PDF Reader:


- Enjoy.

--- Posts about Python --- Posts about xtopdf ---

- Vasudev Ram - Dancing Bison Enterprises - Python programming and training

Signup to be informed about my new products or services.

Contact Page

Written by Vasudev Ram

January 22nd, 2015 at 1:54 pm

Bad Voltage Season 1 Episode 34: Hidden Cities

without comments

Bryan Lunduke, Jono Bacon, Stuart Langridge and myself bring you all the Bad Voltage that’s fit to print, including a special feature where three of us record with good microphones and one moron doesn’t:

  • We give the second half of our predictions for where technology will go in 2015: this episode, Jeremy and Jono (2.30)
  • Christian Hergert talks about his crowdfunding campaign to make Builder, a Gnome IDE for building Gnome apps (27.25)
  • Wrong in 60 seconds: Jono on indicating in cars (46.48)
  • Skiplagged, a website for taking advantage of “hidden city” airline tickets through data analysis, is being sued by airlines. We discuss why, and what this indicates for this sort of air-quotes disruptive app (48.33)
  • Jono reviews the Logitech Harmony 650 all-in-one entertainment system remote control (62.35)

Listen to 1×34: Hidden Cities

As mentioned here, Bad Voltage is a new project I’m proud to be a part of. From the Bad Voltage site: Every two weeks Bad Voltage delivers an amusing take on technology, Open Source, politics, music, and anything else we think is interesting, as well as interviews and reviews. Do note that Bad Voltage is in no way related to LinuxQuestions.org, and unlike LQ it will be decidedly NSFW. That said, head over to the Bad Voltage website, take a listen and let us know what you think.

–jeremy


Written by jeremy

January 22nd, 2015 at 9:53 am

Posted in Bad Voltage

Use WhatsApp Web in your web browser | WhatsApp Web Interface

without comments

Hello, This Article will show you how to use WhatsApp Web from your Computer / Web Browser Google Chrome. Prerequisites: 1. Latest Version of WhatsApp 2. Google Chrome Web Browser. You have to match above two requirements to use WhatsApp Web from Computer/Google Chrome. Steps to Use: 1. Open Google Chrome Web Browser. 2. Open link http://web.whatsapp.com […]

Use WhatsApp Web in your web browser | WhatsApp Web Interface

without comments

Hello, This Article will show you how to use WhatsApp Web from your Computer / Web Browser Google Chrome. Prerequisites: 1. Latest Version of WhatsApp 2. Google Chrome Web Browser. You have to match above two requirements to use WhatsApp Web from Computer/Google Chrome. Steps to Use: 1. Open Google Chrome Web Browser. 2. Open link http://web.whatsapp.com […]

Music video: Dire Straits – Walk of Life

without comments

By Vasudev Ram



An old favorite, listened to it again today. The video is good too.

Dire Straits

Dire Straits - Walk of Life:



- Vasudev Ram - Dancing Bison Enterprises

Written by Vasudev Ram

January 19th, 2015 at 12:06 pm

AWS Cloudformation: Defining your stack

without comments

AWS Cloudformation is a service that allows you to define your infrastructure on AWS in a template. This template it’s just a json file where you can define all the resources you need to create on your infrastructure, this is really useful to keep track of all your changes on the infrastructure under a version control of your choice, rollback changes and replicate your environment in other places in question of minutes.

When you define a template, you can think about the definition of one stack where is a set of logical resources you’ll need to provide a service. For example imagine a typical architecture for a web application, is composed basically by a web layer and database layer. Depending on the size of the project, we’ll need more than one web server to serve the content to the clients so we’ll need a load balancer to distribute the load to the web servers. The web server layer can be setup under an auto scaling groups to scale up or scale down the number of servers depending on the load of our web servers. As far we’ve our basic web stack defined:

– Web server instances.
– Database server instance.
– Auto scaling group for web servers.
– Load balancer.

aws-CloudFormation

So based on the example of the web application, cloudformation allows us to define all these resources in a json file creating a stack and cloudformation will be responsible to create automatically all the resources for us. After the creation of the stack you can update, add or delete more resources modifying the template and updating our stack, it’s possible to protect some resources to be modified or deleted if they are critical for our service creating a stack policy. Now let’s see the basic anatomy of a cloudformation template:

{
"AWSTemplateFormatVersion" : "version date",

"Description" : "JSON string",

"Parameters" : {
set of parameters
},

"Mappings" : {
set of mappings
},

"Conditions" : {
set of conditions
},

"Resources" : {
set of resources
},

"Outputs" : {
set of outputs
}
}

AWSTemplateFormatVersion: Cloudformation template format used, most of the time is used the last one “2010-09-09″.
Description: A small explanation about our stack and all the resources.
Parameters: All the parameters passed to our resources at the creation time of the stack, for example the administrator user and password of the database instance, the number of initial instances to launch, and elastic ip to associate to an ec2 instance, etc…
Mappings: It’s a kind of lookup table where you can store definitions of key:value and retrieve the value using the internal function Fn::FindInMap. This is useful for example in cases we need to launch ec2 instances using different AMIs based on the region the stack is created.
Conditions: Includes statements to conditionally create or associate resources in our stack. For example imagine we’ve a stack definition for a test and a production environment and conditionally creates t2.small ec2 instances for our testing environment or m3.xlarge size for the production environment.
Resources: This is the central part of our template, here are defined all the resources of the stack such as s3 buckets, ec2 instances, load balancers, etc…
Outputs: The values returned by the different resources created, for example the URL of a S3 bucket, the dns record of a load balancer, the elastic ip address associated to an EC2 instance, etc…

The documentation of cloudformation it’s well documented, so understanding the anatomy of a template and following the documentation and some examples is enough to start working with cloudformation. I’ll leave link to my github repository where I’ve defined a small stack for my environment: https://github.com/opentodonet/cloudformation-templates/blob/master/WebServer-Stack.json

Basically this stack creates an EC2 instance with an elastic IP associated, a RDS database instance, two S3 buckets to store backups and logs with a lifecycle, a couple of security groups associated to the web server instance and the rds instance and an IAM role including policies to grant access to EC2, S3 buckets and cloudformation resources to the new EC2 instance. Let’s see with a bit more detail the different resources defined on this stack:

AWS::EC2::SecurityGroup: Creates two security groups, one is for the EC2 instance to give access to http, https and ssh services and the security group for the RDS with access to the entire internal subnet to the port 3306. See how parameters are referenced using the internal function {“Ref” : “VpcId”}, where associates the security groups to the VPC id passed.

  "WebServerSecurityGroup" : {
      "Type" : "AWS::EC2::SecurityGroup",
      "Properties" : {
        "GroupDescription" : "Enable HTTP, HTTPS and SSH access",
        "VpcId" : {"Ref" : "VpcId"},
        "SecurityGroupIngress" : [
          {"IpProtocol" : "tcp", "FromPort" : "80", "ToPort" : "80", "CidrIp" : "0.0.0.0/0"},
          {"IpProtocol" : "tcp", "FromPort" : "443", "ToPort" : "443", "CidrIp" : "0.0.0.0/0"},
          {"IpProtocol" : "tcp", "FromPort" : "22", "ToPort" : "22", "CidrIp" : "0.0.0.0/0"}
        ]
      }
    },
    "DBEC2SecurityGroup": {
      "Type": "AWS::EC2::SecurityGroup",
      "Properties" : {
        "GroupDescription" : "Frontend Access",
        "VpcId"            : {"Ref" : "VpcId"},
        "SecurityGroupIngress" : [{
          "IpProtocol" : "tcp",
          "FromPort"   : { "Ref" : "DBPort" },
          "ToPort"     : { "Ref" : "DBPort" },
          "CidrIp"     : "172.16.0.0/16"
        }]
      }
    },

AWS::S3::Bucket: Here are defined the two buckets for logs and backups. The DeletionPolicy ensures if the stack is deleted the s3 buckets will be preserved. AccessControl defines the ACL to access on this bucket, in that case both are private. LifecycleConfiguration allows you to create a lifecycle policy to apply on the bucket, in that case both buckets will remove the files older than 15 or 30 days, but here you can setup to archive the files to AWS Glacier for example.

  "S3BackupBucket" : {
      "Type" : "AWS::S3::Bucket",
      "DeletionPolicy" : "Retain",
      "Properties" : {
        "AccessControl" : "Private",
        "BucketName" : "opentodo-backups",
        "LifecycleConfiguration" : {
          "Rules" : [ {
            "ExpirationInDays" : 15,
            "Status" : "Enabled"
          } ]
        }
      }
    },
    "S3LogBucket" : {
      "Type" : "AWS::S3::Bucket",
      "DeletionPolicy" : "Retain",
      "Properties" : {
        "AccessControl" : "Private",
        "BucketName" : "opentodo-logs",
        "LifecycleConfiguration" : {
          "Rules" : [ {
            "ExpirationInDays" : 30,
            "Status" : "Enabled"
          } ]
        }
      }
    }

AWS::IAM::Role: Allows to make API requests to AWS services without using an access and secret keys, using Temporary Security Credentials. This role creates different policies to give access to S3 buckets backups and logs, ec2 access and cloudformation resources.

  "WebServerRole": {
      "Type": "AWS::IAM::Role",
      "Properties": {
        "AssumeRolePolicyDocument": {
          "Version" : "2012-10-17",
          "Statement": [ {
            "Effect": "Allow",
            "Principal": {
              "Service": [ "ec2.amazonaws.com" ]
             },
             "Action": [ "sts:AssumeRole" ]
          } ]
        },
        "Path": "/",
        "Policies": [
          { "PolicyName": "EC2Access",
            "PolicyDocument": {
              "Version" : "2012-10-17",
              "Statement": [ {
                "Effect": "Allow",
                "Action": ["ec2:*","autoscaling:*"],
                "Resource": "*"
              } ]
            }
          },
          { "PolicyName": "S3Access",
            "PolicyDocument": {
              "Version" : "2012-10-17",
              "Statement": [ {
                "Effect": "Allow",
                "Action": "s3:*",
                "Resource": ["arn:aws:s3:::opentodo-backups","arn:aws:s3:::opentodo-backups/*","arn:aws:s3:::opentodo-logs","arn:aws:s3:::opentodo-logs/*"]
              } ]
            }
          },
          { "PolicyName": "CfnAccess",
            "PolicyDocument": {
              "Version" : "2012-10-17",
              "Statement": [ {
                "Effect": "Allow",
                "Action": ["cloudformation:DescribeStackResource"],
                "Resource": "*"
              } ]
            }
          }
        ]
      }
    },

AWS::IAM::InstanceProfile: references to the IAM role, this is just a container for the IAM role and this allows to assign the role to an EC2 instance.

  "WebServerInstanceProfile": {
      "Type": "AWS::IAM::InstanceProfile",
      "Properties": {
        "Path": "/",
        "Roles": [ {
        "Ref": "WebServerRole"
         } ]
      }
    },

AWS::EC2::Instance: Creates the EC2 instance using the AMI id ami-df1696a8, and assigns the InstanceProfile defined before, in the subnet id subnet-7d59d518 and the instance size and key pairs passed as parameters. The property UserData allows to setup scripts to run in the startup process. This commands passed on the user-data are run by the cloud-init service, which is included on the public AMIs provided by AWS. This user-data setup here installs the package python-setuptools and installs CloudFormation Helper Scripts, which are a set of python scripts to install packages, run commands, create files or start services as part of the cloudformation stack on the EC2 instances. The cfn-init command gets the cloudformation metadata to check what tasks has to run the instance (that’s why we include the policy access to cloudformation:DescribeStackResource on the IAM role before). The cloudformation metadata is setup on the AWS::CloudFormation::Init key, where basically installs some packages including the awscli tool and creates a couple of files, the /root/.my.cnf to access to the RDS instance which is filled using the attributes got after create the RDS instance, and the file /etc/bash_completion.d/awscli for the awscli auto completion. The cfn-signal command on user-data is used to indicate if the EC2 instance have been successfully created or updated, which is handled by the CreationPolicy attribute to wait until the cf-init command has finished, with a timeout of 5 minutes.

  "WebServerEc2Instance" : {
      "Type" : "AWS::EC2::Instance",
        "Metadata" : {
        "AWS::CloudFormation::Init" : {
          "config" : {
            "packages" : {
              "apt" : {
                "nginx" : [],
                "php5-fpm" : [],
                "git" : [],
                "etckeeper" : [],
                "fail2ban" : [],
                "mysql-client" : []
              },
              "python" : {
                "awscli" : []
              }
            },
            "files" : {
              "/root/.my.cnf" : {
                "content" : { "Fn::Join" : ["", [
                  "[client]n",
                  "user=", { "Ref" : "DBUser" }, "n",
                  "password=", { "Ref" : "DBPassword" }, "n",
                  "host=", { "Fn::GetAtt" : [ "DBInstance", "Endpoint.Address" ] }, "n",
                  "port=", { "Fn::GetAtt" : [ "DBInstance", "Endpoint.Port" ] }, "n"
                ] ] },
                "mode"  : "000600",
                "owner" : "root",
                "group" : "root"
              },
              "/etc/bash_completion.d/awscli" : {
                "content" : { "Fn::Join" : ["", [
                  "complete -C aws_completer awsn"
                ] ] },
                "mode"  : "000644",
                "owner" : "root",
                "group" : "root"
              }
            }
          }
        }
      },
      "Properties" : {
        "ImageId" : "ami-df1696a8",
        "InstanceType"   : { "Ref" : "InstanceType" },
        "SecurityGroupIds" : [ {"Ref" : "WebServerSecurityGroup"} ],
        "KeyName"        : { "Ref" : "KeyPair" },
        "IamInstanceProfile" : { "Ref" : "WebServerInstanceProfile" },
        "SubnetId" : "subnet-7d59d518",
        "UserData": {
          "Fn::Base64": {
            "Fn::Join": [
              "",
              [
                "#!/bin/bashn",
                "aptitude updaten",
                "aptitude -y install python-setuptoolsn",
                "easy_install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gzn",
                "# Install the files and packages from the metadatan",
                "cfn-init --stack ", { "Ref" : "AWS::StackName" }," --resource WebServerEc2Instance --region ", { "Ref" : "AWS::Region" }, "n",
                "# Signal the status from cfn-initn",
                "cfn-signal -e $? ","--stack ", { "Ref" : "AWS::StackName" }," --resource WebServerEc2Instance --region ", { "Ref" : "AWS::Region" }, "n"
              ]
            ]
          }
        }
      },
      "CreationPolicy" : {
        "ResourceSignal" : {
          "Timeout" : "PT5M"
        }
      }
    }

AWS::EC2::EIPAssociation: Associates the elastic IP passed as parameter to the EC2 instance. The elastic IP must be allocated before on AWS.

  "EIPAssociation" : {
      "Type" : "AWS::EC2::EIPAssociation",
      "Properties" : {
        "InstanceId" : {"Ref" : "WebServerEc2Instance"},
        "EIP" : {"Ref" : "ElasticIP"}
      }
    },

AWS::RDS::DBSubnetGroup: Creates a DB subnet group using the subnet ids defined where the RDS instance will be setup.

  "DBSubnetGroup" : {
      "Type" : "AWS::RDS::DBSubnetGroup",
      "Properties" : {
        "DBSubnetGroupDescription" : "WebServer DB subnet group",
        "SubnetIds" : [ "subnet-058c0560", "subnet-2072c457" ]
      }
    },

AWS::RDS::DBInstance: Creates the RDS instance on the subnet group created before with some properties passed as parameter.

  "DBInstance" : {
      "Type": "AWS::RDS::DBInstance",
      "Properties": {
        "DBInstanceIdentifier" : "WebServerRDS",
        "Engine"            : "MySQL",
        "MultiAZ"           : { "Ref": "MultiAZDatabase" },
        "MasterUsername"    : { "Ref" : "DBUser" },
        "MasterUserPassword": { "Ref" : "DBPassword" },
        "DBInstanceClass"   : { "Ref" : "DBClass" },
        "AllocatedStorage"  : { "Ref" : "DBAllocatedStorage" },
        "DBSubnetGroupName" : { "Ref" : "DBSubnetGroup" },
        "Port"              : { "Ref" : "DBPort" },
        "StorageType" : "gp2",
        "AutoMinorVersionUpgrade" : "true",
        "BackupRetentionPeriod" : 5,
        "PreferredBackupWindow" : "02:30-03:30",
	"PreferredMaintenanceWindow" : "sun:04:30-sun:05:30",
        "VPCSecurityGroups": [ { "Fn::GetAtt": [ "DBEC2SecurityGroup", "GroupId" ] } ]
      }
    },

As I said AWS has a very well documentation, so all that you need you can find on his doc pages and find very useful examples:

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-sample-templates.html

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-init.html

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-init.html


Github repository with the full template: https://github.com/opentodonet/cloudformation-templates/

Written by ivanmp91

January 18th, 2015 at 2:11 pm