Why I love Perl already?

Since I have only started learning Perl I’m not yet qualified to talk about the programming aspects of the languages.I also don’t have much experience with lot of other languages so wouldn’t try even comparing them. Here I’m only trying to state a few traits of the general Perl community and programmers, which I already started loving.

1) Perl people are passionate.
Have you ever talked to a Perl programmer? The chances are that you will start programming in Perl too. Actually I too was brought in like that by a friend. Perl Programmers love Perl more than anything in the world and will do anything to popularize it. If you are in doubt check the community.

2) The gurus are there to help you
The irc channel contains knowledgeable guys and they are always ready to help a newbie. Whenever I had doubt I visited the #perl channel in freenode and my doubts were cleared within minutes. Yesterday I saw a senior member in irc stating “:) Ha I didn’t kill a newbie”, after explaining a seemingly easy doubt for a newbie and the newbie reporting to check into the matter further. The emotion embedded in the statement was how much the Perl community want new people to come in. I don’t know if this is normal with all the programing language channels, but in #perl you are sure to be in company of prominent and great guys. It is possible that the author of the book that you following explaining your doubts.

3) Great documentation
Good documentation is available online, for download and most of the time pre installed in linux.
Documentation is rather exhaustive and you get to learn every aspect of the language from this alone. However there are also lot of printed material to assist. The Llama, Camel and Alpaca are the premium source of education in the field of Perl. The documenation got a new and nice face lift recently and is rather god looking and easily navigable.

4) Highly expressive and less restrictive language
The language is really expressive and less restrictive. There are more than one way to do things, and actually it is the motto of the language. At first a newbie may find it hard to remember all of these different techniques and also some of the language constructs may seem confusing and unnecessary. But a few days in Perl and you will start enjoying all of these and it begins to seem second language to you. For a sample of what I was saying look into the last post. For another example you will get confused (I got) that a subroutine can be called with and without a ‘&’ and you doubt why? But a couple of pages more into the Llama would tell you the reasons. And once you know the reasons, you will appreciate it, and sometimes you even wonder how in the world Larry even visualized this feature. Another thing I found in my small experience so far are two new constructs, unless and until. I haven’t seen any of these in any of the languages I have used so far.You may argue by saying it is same is made possible by adding a ‘!’ in front of ‘if’ and ‘while’. Yeah, it is true, but it is great to have those and it makes you more expressive and I’m sure Perl have more of them in stock for me.

5) Nice and humble people.
There are more but I’m stopping with this one. This is by far the most important thing I have noticed. Perl people are humble. They love their language and are passionate about them but would never belittle another languages. Most of them are well versed in many languages. Actually all of them would tell you to learn other languages too and use one which suits you. I think the confidence is from the fact that once you start coding in Perl you are unlikely to move into others. Perl is pretty addictive.

add to del.icio.us : Add to Blinkslist : add to furl : Digg it : Stumble It! : add to simpy : seed the vine : : : TailRank : post to facebook


Explantion on the use of chop() and chomp() functions.

In an effort to learn Perl I was going through the Llama book. When I came to the second chapter there was an exercise where we need to enter two numbers and the program will return their product. I did the program and it worked well except for a small glitch. Here is my code and the output.

print "Enter the first number : ";
$num1 = <STDIN>;
print "Enter the second number : " ;
$num2 = <STDIN>;
$prd = $num1 * $num2;
print "The product of $num1 and $num2 is $prd\n";

The code worked well and outputted 20 if we enter 4 and 5 but it printed output like this.

Enter the first number : 4
Enter the second number : 5
The product of 4
and 5

Now it was annoying. Having only started programming in Perl and my previous experience in C or Java or even PHP never made me expect something like this. I expected it to print the whole sentence in one single line. What could be wrong. Then I recalled Llama telling about <STDIN> adding a ‘\n’ to the input and the need for using chomp() to get rid of this.

So I modified code and added two more lines


This made it correct. But still I found it odd and thought this feature of <STDIN> a nuisance. So I went into some knowledge hunting. I asked in #perl channel at freenode. So far I was wary and wasn’t talking and was merely listening. I was afraid even while asking this of getting some serious ‘RTFM’. But the folk in there responded nicely. Thanks for <claes_>. He gave the apt reply. Here is what I learned from him.

There is a special variable in Perl named $/ which is initialzed to be ‘\n’. This variable is termed the input record separator. This variable is used to separate the various lines from your inut. Since $/ contains ‘\n’ it assumes your input has been ended once you key in the return key.(Note: Though the ‘\n’ is not a part of the intended input, since you entered it it will also be included as a part of your input). That’s why we get the newline when printing it back. When we use chomp() it removes these ‘\n’. If we explicitly assign say, ‘f’ to $/ then <STDIN> enters whatever up to the first ‘f’ it sees. Also the functionality of chomp() changes. It will now remove the last ‘f’ from your input (Once again though ‘f’ wasn’t in your intended input, since you entered it it would be there, Please don’t do so. Leave ‘$/’ to the ‘/n’, or be ready to bear the dire after effects). So $/ variable helps in entering user input from standard input.

Another option is to use in the above example is the chop() method [I earlier called it a better option, but I taking that away as per the info by ‘Andrew’]. When chomp() removes whatever declared in the $/ variable, chop() removes the last character from the input.

So in the below snippet
$str = "Christy;
print $str;

the output is ‘Christ’;

I think I was successful in explaining the issue and let this save another newbie someday.

[Revision: Thursday 3 Sep, 2009, There was some confusion in the explanation of input record separator. Thank to Chas Owen. I corrected it.}

add to del.icio.us : Add to Blinkslist : add to furl : Digg itStumble It! : add to simpy : seed the vine : : : TailRank : post to facebook

Stepping into the world of Perl.

Alan Haggai Alavi a friend of mine and another Perl hacker introduced me to Perl a few days back. He has been doing steady Perl coding for a while and has submitted some modules in CPAN and is a known name among some elitist coders in the Perl community. I am happy to have him as my friend.

Much to the anger of the die hard Perl hackers, I was also an ignorant fellow who confused Perl to be synonymous wih cgi. But Haggai and some reading on the matter got rid of the misinformation. Haggai showed me some samples of a CMS he was developing in Perl. It is using catalyst as the frame work. He showed me how to do a Unit Test, though most of it was Latin to me, since I haven’t yet done Test driven development (I know for a programmer with more than 2 years of experience it is a shame, What to do, all the work I have done so far didn’t demand one. I’m pretty beginner.). However this got me excited and I started looked into Perl.

There are so much myths regarding Perl and I’m yet to gain experience to believe or dispel one. From what I heard from people who actually code Perl is that Perl is robust, powerful and a less restrictive language. So I’m going to learn it.

One more thing I love about Perl is the passion among Perl coders. They will Eat, Drink and Sleep Perl and will do anything and everything for the language. I haven’t seen such a kind of emotion in no other language community. Also the Perl coders are humble. They don’t belittle other languages. They even tell people to learn more languages to experience the difference and gain wisdom. This is some kind of emotion which isn’t seen elsewhere. For an experience, ask a python programmer what he feels about Perl. I think this cool behavior of Perl programmers must have been inherited from Larry Wall (I don’t know him personally. But I read he is a very religious guy. So it implies he is humble).

Haggai also introduced me to Ohloh and Stack Overflow. Since I haven’t developed any open source software and isn’t yet a genius my rating there is very poor. But I believe I can improve in the coming days.

I also decided to join the Iron man competition.Maybe I’m vain, but I have a dream. One day I too will a known name in the open source community. Like Matt S Trout, Shlomi Fish, Randal Schwartz, brian d foy, Larry Wall (Oh, God! I’m really vain)  one day Perl community will know me too.

What is actually unicode and how Java implements it?

In an attempt to move into the field of enterprise application development I started refreshing my Java recently. I was going through a well known book when I stumbled upon the implementation of  Strings in Java. I have high esteem for the developers at Sun, but I really could not digest the fact that Sun engineers thought 2 bytes would be enough for characters. It was kind of  Y2kish. Now that the UTF has grown above the usual number 16 bits can represent I was eager to find how Sun tackled this problem. The book touched the matter vaguely but since that didn’t completely clear my doubts I decided to investigate. And these are my findings.

Before we dive into the Java implementation of the standard, we should understand what UTF is. At least some of us might have seen it somewhere. May be those of us who have the creepy behavior of going through the source of an HTML page might have seen the following,
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />.
And we may have a vague idea on what it does. Don’t worry let’s understand what it really is.

Unicode is an internationally accepted standard to define character sets and corresponding encoding. And the above piece of code is informing the browser that the document should be interpreted using the utf-8 character set.  So what exactly is UTF or Uniform Transformation Format?

In order to completely know about Unicode, we have to traverse a few decades back when people thought the earth was the center of  the universe and it was indeed flat. Oh sorry, we have to traverse a few decades back when the majority of the software was written by English speaking people. It was only quite natural they thought the only set of characters ever to be  encountered in the realm of programming would be from English alphabets, numerical digits and a couple of other prominent characters. So it was logical to use 28 or 256 bits to represent the set of characters. It was enough for the normally used characters back then and also space were left for the inclusion of characters in the future. The problem started when different people started encoding different characters in the free space. What evolved was chaos. Also when the internet happened, all the people around the globe  started using technology and started tweaking programs to their likes and in their own native languages. It became impossible to fit all the characters into the tiny space of 256 bits. Soon the encoding system known as ASCII ran out of space and a need for a better encoding system evolved. To make a long story short, thus evolved UTF. More on this can be read from here and here.

In UTF characters are represented by code points. It is usually a hex number preceded by U+. For example U+2122 means TM, the trademark symbol. The prominent UTF encodings are UTF-8, UTF-16 and the latest one being UTF-32. These three are methods to represent the UTF character set using 8 bit, 16 bit and 32 bit respectively. To get a more detailed and exclusive idea on Unicode please read Joel Spolsky’s post.

In Java from beginning characters were represented by 16 bits and for some time it was enough to represent all the characters. But since the characters included into UTF overgrew the 16 bit realm, Java was faced with a dilemma, either to change the char representation into 32 bit or use some other methods.. It is not of much issue since most of the characters outside the 16 bit representation is rarely used. But since Java is a language which believes a in portability very much and the engineers in Sun are much more intelligent than the average developers like us, they found a way to circumvent this issue. Java is now equipped to represent all the characters in the 32 bit realm also. So how does Java tackle the supplementary characters out side the 16 bits? What Sun employed to get out of this mess was UTF-16 encoding. So what is UTF-16?

To quote Wikipedia UTF- 18 is a variable length character encoding for Unicode, capable of encoding the entire Unicode repertoire. The encoding maps character to a sequence of 16 bit words. Characters are known as code points and 16 bit words are known as code units. The basic characters from the Basic Multilingual Plane’ can be represented using the 16 bits. For characters outside this we need to use a pair of 16 bit words called as the surrogate pair. Thus all the characters that can be encoded by 232 or U+0000 through U+10FFFF , except for U+D800–U+DFFF (These are not assigned any characters) can be specified using UTF-16. Why are these numbers not assigned any characters? It is an intelligent choice made by the Unicode community to design the UTF-16 encoding scheme.

The characters outside the BMP (those from U+10000 through U+10FFFF) are represented using a pair of 16 bit words as I said before. These pair is known as surrogate pair. Now 1000016 is subtracted from the original code point to make it a 20 bit number. Now it is divided to two 10 bit numbers each of which is loaded into a surrogate with the higher order bits in the first. the two surrogates will be in the range 0xD800–0xDBFF and 0xDC00-0xDFFF. Thus since we have left out those region unassigned we can be sure it isn’t a character but need processing before the original code point is found out.  You can read the UTF-16 specification from Sun here.

add to del.icio.us : Add to Blinkslist : add to furl : Digg it : Stumble It! : add to simpy : seed the vine : : : TailRank : post to facebook

Links to other good tutorials in internet

I believe in the idea of recycling and reuse. It is much more efficient to reuse someone else’s effort than reinventing the wheel. So I’m taking the opportunity in this post to point you to some of the best tutorials and fixes for various issues we may have while using a computer. It’s better this way than just plagiarizing their efforts.

1. Installing Mplayer with GUI and browser plugin.

The big crash! How I learned form it.

If you remember I was telling you how my system was slow on Fedora 11and how it will alarm me of a kernel error every now and then. I never cared. But it turned out to be a big mistake. Once I rebooted the system, it went crazy and the GUI crashed and some geeky error messages came on the screen, As a newbie I was sure there is nothing much to do other than reinstalling.

The scenario wasn’t different even after reinstalling. The system crashed again at the second boot. Enough was enough, I decided, Either the distro isn’t stable or my system isn’t mature enough to handle an advanced operating system. Any how I wanted my system back in working condition. My mind begged for the stability of my old OpenSUSE system. Alas! The copy I had has gone corrupted. But so strong was my will and highly demanding was the situation that I somehow managed to get hands on one by the end of the day. An old friend from college had a copy.

I was sure about everything obout the distro other than the feasibility of the old internet trick. The installation technique in SUSE differed very much from Fedora. But I believed I could somehow make it work, And my belief was justified.

Kppp wasn’t present in my installation. So I had to install it later. But it was an easy and seamless job. YaST did a wonderful job and it is an innovation of a kind. If ever you are going to work with SUSE you are going to appreciate the cool software management tool. Installation of Kppp took me less than 3 minutes. I configured it like earlier and was soon browsing.

But a new challenge arouse. Once I logged out of root privilege Kppp wasn’t letting me log in. I later found out it is a design flaw of t Kppp (or is it a feature) and there are methods to overcome it. I followed the steps given on this page. Please check it. I think it is a solution for an older version of OpenSUSE. Though the former method didn’t work for me I was able to implement the graphical solution.

Even though the steps may differ, there exists workarounds for the above issue in other distros too. Please use Google and you will find them.

Using a wireless USB modem in Linux.

As I have said earlier, I’m writing the methods to use the USB modem in Linux. In this tutorial I have to taken the privilege to believe you are using a recent distro with KDE as the Windowing Manger. In my case, it was Fedora 11 and I’m trying to use the EVDO from BSNL. I also used kppp as the dialer.

I believe by this time you have connected the modem to one of the vacant USB ports. If not, do so. Now our task consists of making the OS detect it and then inputting the correct parameters for the dialer,

The first is pretty straight forward job and you would be able to do this pretty easily. I’m not suggesting the other part is difficult, but without enough documentation and guidelines you are likely wander in Neverland, let alone get in. I did until I tried all the possibilities and dedicated some time to read the manuals. Also I got loads of help from other bloggers. So here I’m providing the little I learned from the experience. I hope it helps you.

First login as the super user. You need admin powers to execute certain commands.

1. Find the the vendor and product id of the modem.
You can do this in two ways
a) cat /proc/duc/usb/devices
This will give a list of the USB devices. Find the make of yours from the list. Most probably it will output something similar to

P: Vendor=05c6 ProdID=6000 Rev= 0.00
S: Manufacturer=ZTE, Incorporated
S: Product=ZTE CDMA Tech

b) You may also execute lsusb command
It will give another list and your device may be listed something like
Bus 001 Device 002: ID 05c6:6000 Qualcomm, Inc.

Note that even though the make of your device is ZTE, it is shown as Qualcomm here. It doesn’t matter. What we are really after are the hex codes for the product code and vendor id. We understand that the vendor is 05c6 and product id is 6000 in our case.

2. Now we need to detect the modem using modprobe command. Please input the following in the terminal.
modprobe usbdserial vendor=0x05c6 product=0x6000
This will detect the modem for you. Please make sure the ids are in hex (ie, there is 0x in front of the codes). You can automate this process by adding ‘usbserial vendor=0x05c6 product=0x6000‘ to file “/etc/modules“. This will execute the command each time the system starts up.

3. You may check whether it was successful by running dmesg command. If successful it will give output with lines similar to the following.
usbserial_generic 1-3:1.0: generic converter detected
usb 1-3: generic converter now attached to ttyUSB0
usbserial_generic 1-3:1.1: generic converter detected
usb 1-3: generic converter now attached to ttyUSB1
usbserial_generic 1-3:1.2: generic converter detected
usb 1-3: generic converter now attached to ttyUSB2

Now it means our modem has been detected and it is called ttyUSB0 by Linux. Now that we have finished the first part we are now into configuring the dialler. Here also you can follow two methods. Either by configuring kppp, the pppd dialler in KDE or editing the wvdial.conf file and using the terminal to dial in. I will explain both, though I prefer the former (If you Desktop manager is Gnome, you may not able to use kppp, but may be able to follow the wvdial method. I’m not sure, Please try and let me know).

Configuring kppp.

Please launch kppp (Where you find it under the start menu may differ for different distros. Please consult your distro’s manual for finding it). Yo will get an interface with a username and password option. At first it will be blank. You need to configure it before using it. Please click the ‘Configure’ button to open a new panel.

1. Under the many tabs, please make sure the ‘Accounts‘ tab is active and click the ‘New‘ button. Check the ‘Manual Setup‘ button in the new window.
2.A new window is opened with more tabs out of which the first one is ‘Dial‘. Add whatever name you like in the ‘Connection Name‘ field. Add new phone number. In BSNL’s case it is #777 or *99***1#.
3. Make sure PAP is selected under the Authentication field. The default is PAP/CHAP. Since BSNL gave ulogin credentials we need to authenticate ourselves with the server. PAP ensures it. Since the BSNL server doesn’t need to authenticate itself to our system, we can let go off channel handshaking and thus the CHAP protocol. This is atmost important whose failure may cause kppp to quit with an error code 19.
4. No need to bother with any other tabs. Let them remain as it is. Confirm the changes to come back to the initial window.
5. Now access the ‘Modems‘ tab. Click ‘New‘ to open a new window.
6. Enter a name for the modem and select ‘/dev/ttyUSB0‘ from the ‘Modem Device’ drop down in the ‘Device’ tab.
7. Now access the ‘Modem’ tab and uncheck the ‘Wait for dialtone before dialling‘ option. Confirm the changes to go back to the initial screen. Enter the username and password in the fields and we are 80% finished.

Now comes two of the most important configuration without which you are as good as you were in the beginning. If we try to dial in now, we will see that the dialer quits with an error 19. You will be searching in dark as you are sure there is no issues with the login credentials and you have entered the correct. Even re entering them won’t help. you will still stumble at the same block.

So what is the real issue here. You need someone to point out or extreme patience to read the manuals to find out. The kppp help is extremely good. It together with some online help ( rebooting OS’s alternatively to check the net and try the findings can be tiresome but is necessary at times like these) can eventually lead you to the solution. But don’t worry I had done all the hard work for you. All you need to do is do a couple more of fixes.

1. The “pppd died – The remote system is required to authenticate itself …” or the code no 19 error is caused due to two reasons.
Either the /etc/ppp/options contains the auth option or your system already has a default route. In this case recent versions of pppd will behave as if auth had been specified. To ward off the first issue add a ‘#’ symbol in front of it and to override the second option you may add noauth to the file.

This enables your modem to connect to the internet. But this doesn’t fully solve the issue. You will notice the browser is still not loading the pages. This time you will be in more despair as there isn’t a good error message along with it. But nothing is impossible. You are indeed destined to browse net from your Linux box. Just perform the next fix.

2. Add the following lines in the to the /etc/resolv.conf file

These are the ip addresses of OpenDNS servers. It’s secure, fast and trusted service. It’s better than the ISP’s servers ad I suggest you use them. You can learn more about them at the company site.

Voila! you are done and is now you are eligible for the ‘Geek of the Hour’ award. Happy browsing.

Editing wvdial.conf
Now for the command line method add a new file wvdial.conf in the /etc folder in addition to former files of resolv.conf and ppp/options file. Most probably the wvdial may be present you may just need to edit it. If not you may need to install wvdial. Check with the OS manual for the method.
Add the following to the file

[Dialer Defaults]
Modem = /dev/ttyUSB0
Dail Command = ATDT
init1 = ATZ
init2 = AT+CRM = 1
Flow Control = Hardware (CRTSCTS)
Username = <substitute yours>
Password = <substitute yours>
Phone = #777
Stupid Mode = 1
Auto DNS = 1

Save the file and execute wvdial from terminal and you are connected to the internet. Never close the window until you are done browsing.

There is one more issue we need to address. Kppp wouldn’t work normally if you log out of the root privilege. For a workaround see the end of this post.
I wish this serves as a good tutorial to using USB modem in Linux. Wish you happy browsing.

add to del.icio.us : Add to Blinkslist : add to furl : Digg it : Stumble It! : add to simpy : seed the vine : : : TailRank : post to facebook