Code: Python

Showing posts with label Python. Show all posts

Convert CSV to JSON

Sometimes I find myself converting google doc spreadsheets (CSV) to JSON format. I would later on use the JSON string to insert it into a JavaScript file.
So here is a simple script to convert a CSV file to JSON format.

Python

import sys, csv, json

if len(sys.argv) == 1:
    print "1 argument for filename required"
    sys.exit()

gdoc = csv.reader(open(sys.argv[1]))

# Get the 1st line, assuming it contains the column titles
fieldnames = gdoc.next() 

# Get the total number of columns
fieldnames_len = len(fieldnames)

data = [] # Empty list
i = 0

for row in gdoc:
    
    # Add an empty dict to the list
    data.append({})
    
    for j in range(0, len(row)):
        data[i][fieldnames[j]] = row[j]
    
    # What if the last few cells are empty ? There may not be commas
    for j in range(len(row), fieldnames_len):
        data[i][fieldnames[j]] = ""
    
    i = i + 1

print json.dumps(data)
sys.exit()

Python 2.7.2

Vanakkam !

Python2's String = Python3's Text Vs. Data

A significant change from Python 2 to Python 3 is the way strings are dealt with.
Python 3 doesnt always return a string when expected.
For example, the return type of read() in version 2 has always been a string. But in version 3, its very often a "bytes" string.
When you print a "bytes" string, you'll see every character in its byte format, special characters as escape secquences (newline as \n) and other unicode characters as escape sequences.
This is because Python 3 differentiates between text (string) and data ("bytes" string) as oppossed to Unicode vs 8-bit string. (Text Vs. Data Instead Of Unicode Vs. 8-bit)

My localhost/index.html contains just this :
<html><body><h1>It works!. stärke gläser</h1></body></html>

Python 2.x

import urllib

url = "http://localhost"
fp = urllib.urlopen(url)
data = fp.read()
print "%s, %s" % (type(data), type(data).__name__)
print data

Python 2.6.4

Python 2.x

~$ python
Python 2.6.4 (r264:75706, Nov  2 2009, 14:44:17)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib
>>> url = "http://localhost"
>>> fp = urllib.urlopen(url)
>>> data = fp.read()
>>> print "%s, %s" % (type(data), type(data).__name__)
<type 'str'>, str
>>> print data
<html><body><h1>It works!. stärke gläser</h1></body></html>

>>>

Python 2.6.4

In Python 3, we need to need to explicitly convert it to string format via the str() function and specify the encoding-type.
If you are getting errors using Python 3.0, you may want to update to atleast Python 3.0.1 - many have reported possible Unciode encoding/decoding bugs in 3.0.

Python 3.x

import urllib.request

url = "http://localhost"
fp = urllib.request.urlopen(url)
data = fp.read()
print ("%s, %s" % (type(data), type(data).__name__))
print (data)
data = str(data,'utf-8') # convert a byte datatype to string datatype using utf-8 encoding. For ASCII, data = str(data,'ascii')
print (data)

Python 3.1.1

Python 3.x

~$ python3
Python 3.1.1+ (r311:74480, Oct 12 2009, 02:14:03)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.request
>>> url = "http://localhost"
>>> fp = urllib.request.urlopen(url)
>>> data = fp.read()
>>> print ("%s, %s" % (type(data), type(data).__name__))
<class 'bytes'>, bytes
>>> print (data)
b'<html><body><h1>It works!. st\xc3\xa4rke gl\xc3\xa4ser</h1></body></html>\n'
>>> data = str(data,'utf-8') # convert a byte datatype to string datatype using utf-8 encoding. For ASCII, data = str(data,'ascii')
>>> print (data)
<html><body><h1>It works!. stärke gläser</h1></body></html>

>>>

Python 3.1.1

Vanakkam !

Python's Classmethods

Lets say I have an abstract class called Vehicle and 3 classes that descend from it are Bike, Car & Truck.
I use a static variable in Vehicle called total to keep track of the total number of Vehicles. But I really don't want to keep track of the total number general Vehicles. What I really want is to keep track of the total number of Bikes, Cars & Trucks individually. This is easy - just declare total in Bike, Car & Truck classes.

Now arises a situation where we need a function println in Vehicle that accesses total. We'll also include a function called set in Vehicle to explicitly set the value of total (instead of creating 10 instances to prove a point).

There are two scenarios to this, both of which are not possible :

Declare a static variable total in Vehicle but different values persist in Bike::total, Car::total & Truck::total. This is impossible because total is Vehicle's static variable which is common to all. The following will output 2 2 2.

PHP

<?php
abstract class Vehicle
 {
        protected static $total;

        public static function println()
         {
                echo self::$total."\n";
         }

        public static function set($value)
         {
                self::$total = $value;
         }
 }

class Bike extends Vehicle
 {
 }

class Car extends Vehicle
 {
 }

class Truck extends Vehicle
 {
 }

Bike::set(3);
Car::set(5);
Truck::set(2);

$b = new Bike();
$c = new Car();
$t = new Truck();

Bike::println()
Car::println()
Truck::println()

// Is there any way for Bike's static variable to hold 3 and Car's static to hold 5 & Truck's static to hold 2 ?
?>

PHP 5.2.5

Declare the static variable total in each of Vehicle's subclasses, Bike, Car & Truck. But in this case, Vehicle's println needs to access the descendant class static variable like child::total which is not possible in most(all) languages.

PHP

<?php
abstract class Vehicle
 {
        public static function println()
         {
                echo child::$total."\n";
         }

        public static function set($value)
         {
                child::$total = $value;
         }
 }

class Bike extends Vehicle
 {
        public static $total; # Should've been protected, but then parent wouldn't able to access
 }

class Car extends Vehicle
 {
        public static $total;
 }

class Truck extends Vehicle
 {
        public static $total;
 }

Bike::set(3);
Car::set(5);
Truck::set(2);

$b = new Bike();
$c = new Car();
$t = new Truck();

Bike::println();
Car::println();
Truck::println();
?>

PHP 5.2.5

It is possible to overcome this problem by setting and getting the value in the subclasses and use $this->childMethod() in the parent class, Vehicle.

PHP

<?php
abstract class Vehicle
 {
        public function set($value)
         {
                $this->setChildValue($value);
         }

        public function println()
         {
                echo $this->getChildValue()."\n";
         }
 }

class Bike extends Vehicle
 {
        protected static $total;

        public function getChildValue()
         {
                return self::$total;
         }

        public function setChildValue($value)
         {
                self::$total = $value;
         }
 }

class Car extends Vehicle
 {
        protected static $total;

        public function getChildValue()
         {
                return self::$total;
         }

        public function setChildValue($value)
         {
                self::$total = $value;
         }
 }

class Truck extends Vehicle
 {
        protected static $total;

        public function getChildValue()
         {
                return self::$total;
         }

        public function setChildValue($value)
         {
                self::$total = $value;
         }
 }

$b = new Bike();  $b->set(3);
$c = new Car();   $c->set(5);
$t = new Truck(); $t->set(2);

$b->println();
$c->println();
$t->println();
?>

PHP 5.2.5

But the two methods, getChildValue and setChildValue must be defined properly in all subclasses.
__CLASS__ returns the class in which its called from and get_class($this) returns the class of the current instance.
Example : echo __CLASS__; in a method in Vehicle will always output Vehicle, but get_class($this) will output the classname of the object (In this case, Bike, Car or Truck).
If we could do get_class($this)::$total (Bike::$total), then it could've been easily solved.

PHP

<?php
abstract class Vehicle
 {
        public function set($value)
         {
                get_class($this)::$total = $value;
         }

        public function println()
         {
                echo get_class($this)::$total."\n";
         }
 }

class Bike extends Vehicle
 {
        public static $total;
 }

class Car extends Vehicle
 {
        public static $total;
 }

class Truck extends Vehicle
 {
        public static $total;
 }

$b = new Bike();  $b->set(3);
$c = new Car();   $c->set(5);
$t = new Truck(); $t->set(2);

$b->println();
$c->println();
$t->println();
?>

PHP 5.2.5

The last method is possible in Python in two ways - self.__class__.total & classmethod. Python has a way to access class member of the caller's class, not just the class members in which its being accessed. Here each of the subclasses have a static member called total which gets created and assigned in its parent, Vehicle

Python : Using __class__

class Vehicle:

      def println(self):
          print self.__class__.total

      def set(self, value):
          self.__class__.total = value

class Bike(Vehicle):
          pass

class Car(Vehicle):
          pass

class Truck(Vehicle):
          pass

Bike().set(3)   # Bike.total = 3
Car().set(5)    # Car.total = 5
Truck().set(2)  # Truck.total = 2

b = Bike()
c = Car()
t = Truck()

b.println()
c.println()
t.println()

Python 2.5.1

Python : Using classmethod

class Vehicle:

      @classmethod
      def println(cls):
          print cls.total

      @classmethod
      def set(cls, value):
          cls.total = value

class Bike(Vehicle):
          pass

class Car(Vehicle):
          pass

class Truck(Vehicle):
          pass

Bike.set(3)   # Bike.total = 3
Car.set(5)    # Car.total = 5
Truck.set(2)  # Truck.total = 2

b = Bike()
c = Car()
t = Truck()

b.println()
c.println()
t.println()

Bike.println()
Car.println()
Truck.println()

Python 2.5.1

self.__class__.total points to the static member total of the object's (this) class and not of the class Vehicle.
cls.total references the same thing as the first argument is actually the classname which is not passed in the parentheses, but by using the classname preceding the dot.

Vanakkam !

Generating a Random String

PHP

<?php
class RandomText
 {
        const RAND_NUMBERS       = 1;
        const RAND_SMALL_LETTERS = 2;
        const RAND_CAP_LETTERS   = 4;
        const RAND_ALL           = 7;

        /**
         * @access public
         * @param Type (INT) One of the random constants RAND_*
         * @NoOfChars [INT] Optional - No of characters to generate
         * @return (string) random string
         * @example echo RandomText::Random_String(RandomText::RAND_NUMBERS + RandomText::RAND_CAP_LETTERS, 25);
         */
 	public static function Random_String($Type = self::RAND_NUMBERS, $NoOfChars = 6)
 	 {
 	 	$strList = array();

 	 	// Convert to binary format and find out what combination of random characters are required
 	 	$binType = base_convert($Type, 10, 2);

 	 	// Loop through the bits from Right To Left
 	 	for ($i = strlen($binType) - 1; $i >= 0; $i--)
 	 	 {
 	 	 	if ($binType[$i])
 	 	 	 {
 	 	 	 	switch (pow(2, strlen($binType) - $i - 1))
 	 	 	 	 {
 	 	 	 	 	default:

 	 	 	 	 	case self::RAND_NUMBERS:
 	 	 	 	 	 $Ascii_Range = array(48, 57);  // Numbers - Characters 0 to 9
 	 	 	 	 	break;

 	 	 	 	 	case self::RAND_SMALL_LETTERS:
   	 	 	 	 	 $Ascii_Range = array(97, 122); // Small Letters - Characters a to z
 	 	 	 	 	break;

 	 	 	 	 	case self::RAND_CAP_LETTERS:
 	 	 	 	 	 $Ascii_Range = array(65, 90);  // Capital Letters - Characters A to Z
 	 	 	 	 	break;
 	 	 	 	 }

 	 	 	 	for ($j = $Ascii_Range[0]; $j <= $Ascii_Range[1]; $j++)
 	 	 	 	 $strList[] = chr($j);
 	 	 	 }
 	 	 }

 	 	$RndString = "";
 	 	for ($i = 0; $i < $NoOfChars; $i++)
 	 	 $RndString .= $strList[rand(0, count($strList) - 1)];

 	 	return $RndString;
 	 }
 }

echo RandomText::Random_String(RandomText::RAND_NUMBERS + RandomText::RAND_CAP_LETTERS, 25);
?>

PHP 5.2.3

using System;
using System.Text;

public class RandomText
 {
        public const int RAND_NUMBERS       = 1;
        public const int RAND_SMALL_LETTERS = 2;
        public const int RAND_CAP_LETTERS   = 4;
        public const int RAND_ALL           = 7;

        public static string Random_String()
         {
                return Random_String(RAND_NUMBERS, 6);
         }

        public static string Random_String(int Type)
         {
                return Random_String(Type, 6);
         }

        public static string Random_String(int Type, int NoOfChars)
         {
                StringBuilder strList = new StringBuilder();
                int[] Ascii_Range;
                int i;

                // Convert to binary format and find out what combination of random characters are required
                string binType = Convert.ToString(Type, 2);

                // Loop through the bits from Right To Left
                for (i = binType.Length - 1; i >= 0; i--)
                 {
                        if (binType[i] == '1')
                         {
                                switch ((int)Math.Pow(2, binType.Length - i - 1))
                                 {
                                        default:

                                        case RAND_NUMBERS:
                                         Ascii_Range = new int[] {48, 57};  // Numbers - Characters 0 to 9
                                        break;

                                        case RAND_SMALL_LETTERS:
                                         Ascii_Range = new int[] {97, 122};  // Small Letters - Characters a to z
                                        break;

                                        case RAND_CAP_LETTERS:
                                         Ascii_Range = new int[] {65, 90};  // Capital Letters - Characters A to Z
                                        break;
                                 }

                                for (int j = Ascii_Range[0]; j <= Ascii_Range[1]; j++)
                                 strList.Append((char)j);
                         }
                 }

 	 	StringBuilder RndString = new StringBuilder();
 	 	Random rnd = new Random();
 	 	
 	 	for (i = 0; i < NoOfChars; i++)
 	 	 RndString.Append(strList[rnd.Next(strList.Length - 1)]);

 	 	return RndString;
         }

        public static void Main(string[] args)
         {
                Console.WriteLine(RandomText.Random_String(RandomText.RAND_NUMBERS + RandomText.RAND_CAP_LETTERS, 25));
         }
 }

.NET 2.0

Java

import java.io.*;
import java.util.Random;

public class RandomText
 {
        public static final int RAND_NUMBERS       = 1;
        public static final int RAND_SMALL_LETTERS = 2;
        public static final int RAND_CAP_LETTERS   = 4;
        public static final int RAND_ALL           = 7;

        public static String Random_String()
         {
                return Random_String(RAND_NUMBERS, 6);
         }

        public static String Random_String(int Type)
         {
                return Random_String(Type, 6);
         }

        public static String Random_String(int Type, int NoOfChars)
         {
                StringBuilder strList = new StringBuilder();
                int[] Ascii_Range;
                int i;

                // Convert to binary format and find out what combination of random characters are required
                String binType = Integer.toString(Type, 2); // Type.toString(2);

                // Loop through the bits from Right To Left
                for (i = binType.length() - 1; i >= 0; i--)
                 {
                        if (binType.charAt(i) == '1')
                         {
                                switch ((int)Math.pow(2, binType.length() - i - 1))
                                 {
                                        default:

                                        case RAND_NUMBERS:
                                         Ascii_Range = new int[] {48, 57};  // Numbers - Characters 0 to 9
                                        break;

                                        case RAND_SMALL_LETTERS:
                                         Ascii_Range = new int[] {97, 122};  // Small Letters - Characters a to z
                                        break;

                                        case RAND_CAP_LETTERS:
                                         Ascii_Range = new int[] {65, 90};  // Capital Letters - Characters A to Z
                                        break;
                                 }

                                for (int j = Ascii_Range[0]; j <= Ascii_Range[1]; j++)
                                 strList.append((char)j);
                         }
                 }

                StringBuilder RndString = new StringBuilder();
                Random rnd = new Random();

                for (i = 0; i < NoOfChars; i++)
                 RndString.append(strList.charAt(rnd.nextInt(strList.length() - 1)));

                return RndString.toString();
         }

        public static void main(String[] args) throws Exception
         {
                System.out.println(RandomText.Random_String(RandomText.RAND_NUMBERS + RandomText.RAND_CAP_LETTERS, 25));
         }
 }

JDK 6.0

JavaScript

function RandomText()
 {
        this.RAND_NUMBERS       = 1;
        this.RAND_SMALL_LETTERS = 2;
        this.RAND_CAP_LETTERS   = 4;
        this.RAND_ALL           = 7;

        this.Random_String = function(Type, NoOfChars)
         {
                var strList = "";
                var Ascii_Range;
                var i;

                // Convert to binary format and find out what combination of random characters are required
                binType = Type.toString(2);

                // Loop through the bits from Right To Left
                for (i = binType.length - 1; i >= 0; i--)
                 {
                        if (binType[i] == '1')
                         {
                                switch (Math.pow(2, binType.length - i - 1))
                                 {
                                        default:

                                        case this.RAND_NUMBERS:
                                         Ascii_Range = new Array(48, 57);  // Numbers - Characters 0 to 9
                                        break;

                                        case this.RAND_SMALL_LETTERS:
                                         Ascii_Range = new Array(97, 122); // Small Letters - Characters a to z
                                        break;

                                        case this.RAND_CAP_LETTERS:
                                         Ascii_Range = new Array(65, 90);  // Capital Letters - Characters A to Z
                                        break;
                                 }

                                for (var j = Ascii_Range[0]; j <= Ascii_Range[1]; j++)
                                 strList += String.fromCharCode(j);
                         }
                 }

                var RndString = "";
                for (i = 0; i < NoOfChars; i++)
                 RndString += strList[Math.floor(Math.random() * strList.length)];

                return RndString;
         }
 }

var oRandomText = new RandomText();
alert(oRandomText.Random_String(oRandomText.RAND_NUMBERS + oRandomText.RAND_CAP_LETTERS, 25));

JavaScript 1.5

Python

import array
import random

def int2bin(n):
    "Convert an integer to binary - no built-in function"
    bStr = ''
    while n > 0:
          bStr = str(n % 2) + bStr
          n = n >> 1
    return bStr

class RandomText:

      RAND_NUMBERS       = 1;
      RAND_SMALL_LETTERS = 2;
      RAND_CAP_LETTERS   = 4;
      RAND_ALL           = 7;

      def Random_String(Type = RAND_NUMBERS, NoOfChars = 6):
          strList = array.array('c', '')

          # Convert to binary format and find out what combination of random characters are required
          binType = int2bin(Type)

          # Loop through the bits from Right To Left
          for i in range(len(binType) - 1, -1, -1):

              if binType[i] == '1':

                 x = 2 ** (len(binType) - i - 1)

                 if x == RandomText.RAND_NUMBERS:
                    Ascii_Range = [48, 57]   # Numbers - Characters 0 to 9
                 elif x == RandomText.RAND_SMALL_LETTERS:
                    Ascii_Range = [97, 122]  # Small Letters - Characters a to z
                 elif x == RandomText.RAND_CAP_LETTERS:
                    Ascii_Range = [65, 90]   # Capital Letters - Characters A to Z
                 else:
                    Ascii_Range = [48, 57]   # Numbers - Characters 0 to 9

                 for j in range(Ascii_Range[0], Ascii_Range[1] + 1):
                    strList.append(chr(j))

              RndString = array.array('c', '')

              for i in range(0, NoOfChars):
                  RndString.append(strList[random.randint(0, len(strList) - 1)])

          return RndString.tostring()

      Random_String = staticmethod(Random_String)


print RandomText.Random_String(RandomText.RAND_NUMBERS + RandomText.RAND_CAP_LETTERS, 25)

Python 2.5

Vanakkam !