Latest Entries »

In an attempt to move into the field of enterprise application development I started refreshing my Java recently. I was going through a well known book when I stumbled upon the implementation of  Strings in Java. I have high esteem for the developers at Sun, but I really could not digest the fact that Sun engineers thought 2 bytes would be enough for characters. It was kind of  Y2kish. Now that the UTF has grown above the usual number 16 bits can represent I was eager to find how Sun tackled this problem. The book touched the matter vaguely but since that didn’t completely clear my doubts I decided to investigate. And these are my findings.

Before we dive into the Java implementation of the standard, we should understand what UTF is. At least some of us might have seen it somewhere. May be those of us who have the creepy behavior of going through the source of an HTML page might have seen the following,
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />.
And we may have a vague idea on what it does. Don’t worry let’s understand what it really is.

Unicode is an internationally accepted standard to define character sets and corresponding encoding. And the above piece of code is informing the browser that the document should be interpreted using the utf-8 character set.  So what exactly is UTF or Uniform Transformation Format?

In order to completely know about Unicode, we have to traverse a few decades back when people thought the earth was the center of  the universe and it was indeed flat. Oh sorry, we have to traverse a few decades back when the majority of the software was written by English speaking people. It was only quite natural they thought the only set of characters ever to be  encountered in the realm of programming would be from English alphabets, numerical digits and a couple of other prominent characters. So it was logical to use 28 or 256 bits to represent the set of characters. It was enough for the normally used characters back then and also space were left for the inclusion of characters in the future. The problem started when different people started encoding different characters in the free space. What evolved was chaos. Also when the internet happened, all the people around the globe  started using technology and started tweaking programs to their likes and in their own native languages. It became impossible to fit all the characters into the tiny space of 256 bits. Soon the encoding system known as ASCII ran out of space and a need for a better encoding system evolved. To make a long story short, thus evolved UTF. More on this can be read from here and here.

In UTF characters are represented by code points. It is usually a hex number preceded by U+. For example U+2122 means TM, the trademark symbol. The prominent UTF encodings are UTF-8, UTF-16 and the latest one being UTF-32. These three are methods to represent the UTF character set using 8 bit, 16 bit and 32 bit respectively. To get a more detailed and exclusive idea on Unicode please read Joel Spolsky’s post.

In Java from beginning characters were represented by 16 bits and for some time it was enough to represent all the characters. But since the characters included into UTF overgrew the 16 bit realm, Java was faced with a dilemma, either to change the char representation into 32 bit or use some other methods.. It is not of much issue since most of the characters outside the 16 bit representation is rarely used. But since Java is a language which believes a in portability very much and the engineers in Sun are much more intelligent than the average developers like us, they found a way to circumvent this issue. Java is now equipped to represent all the characters in the 32 bit realm also. So how does Java tackle the supplementary characters out side the 16 bits? What Sun employed to get out of this mess was UTF-16 encoding. So what is UTF-16?

To quote Wikipedia UTF- 18 is a variable length character encoding for Unicode, capable of encoding the entire Unicode repertoire. The encoding maps character to a sequence of 16 bit words. Characters are known as code points and 16 bit words are known as code units. The basic characters from the Basic Multilingual Plane’ can be represented using the 16 bits. For characters outside this we need to use a pair of 16 bit words called as the surrogate pair. Thus all the characters that can be encoded by 232 or U+0000 through U+10FFFF , except for U+D800–U+DFFF (These are not assigned any characters) can be specified using UTF-16. Why are these numbers not assigned any characters? It is an intelligent choice made by the Unicode community to design the UTF-16 encoding scheme.

The characters outside the BMP (those from U+10000 through U+10FFFF) are represented using a pair of 16 bit words as I said before. These pair is known as surrogate pair. Now 1000016 is subtracted from the original code point to make it a 20 bit number. Now it is divided to two 10 bit numbers each of which is loaded into a surrogate with the higher order bits in the first. the two surrogates will be in the range 0xD800–0xDBFF and 0xDC00-0xDFFF. Thus since we have left out those region unassigned we can be sure it isn’t a character but need processing before the original code point is found out.  You can read the UTF-16 specification from Sun here.

add to : Add to Blinkslist : add to furl : Digg it : Stumble It! : add to simpy : seed the vine : : : TailRank : post to facebook

I believe in the idea of recycling and reuse. It is much more efficient to reuse someone else’s effort than reinventing the wheel. So I’m taking the opportunity in this post to point you to some of the best tutorials and fixes for various issues we may have while using a computer. It’s better this way than just plagiarizing their efforts.

1. Installing Mplayer with GUI and browser plugin.

If you remember I was telling you how my system was slow on Fedora 11and how it will alarm me of a kernel error every now and then. I never cared. But it turned out to be a big mistake. Once I rebooted the system, it went crazy and the GUI crashed and some geeky error messages came on the screen, As a newbie I was sure there is nothing much to do other than reinstalling.

The scenario wasn’t different even after reinstalling. The system crashed again at the second boot. Enough was enough, I decided, Either the distro isn’t stable or my system isn’t mature enough to handle an advanced operating system. Any how I wanted my system back in working condition. My mind begged for the stability of my old OpenSUSE system. Alas! The copy I had has gone corrupted. But so strong was my will and highly demanding was the situation that I somehow managed to get hands on one by the end of the day. An old friend from college had a copy.

I was sure about everything obout the distro other than the feasibility of the old internet trick. The installation technique in SUSE differed very much from Fedora. But I believed I could somehow make it work, And my belief was justified.

Kppp wasn’t present in my installation. So I had to install it later. But it was an easy and seamless job. YaST did a wonderful job and it is an innovation of a kind. If ever you are going to work with SUSE you are going to appreciate the cool software management tool. Installation of Kppp took me less than 3 minutes. I configured it like earlier and was soon browsing.

But a new challenge arouse. Once I logged out of root privilege Kppp wasn’t letting me log in. I later found out it is a design flaw of t Kppp (or is it a feature) and there are methods to overcome it. I followed the steps given on this page. Please check it. I think it is a solution for an older version of OpenSUSE. Though the former method didn’t work for me I was able to implement the graphical solution.

Even though the steps may differ, there exists workarounds for the above issue in other distros too. Please use Google and you will find them.

As I have said earlier, I’m writing the methods to use the USB modem in Linux. In this tutorial I have to taken the privilege to believe you are using a recent distro with KDE as the Windowing Manger. In my case, it was Fedora 11 and I’m trying to use the EVDO from BSNL. I also used kppp as the dialer.

I believe by this time you have connected the modem to one of the vacant USB ports. If not, do so. Now our task consists of making the OS detect it and then inputting the correct parameters for the dialer,

The first is pretty straight forward job and you would be able to do this pretty easily. I’m not suggesting the other part is difficult, but without enough documentation and guidelines you are likely wander in Neverland, let alone get in. I did until I tried all the possibilities and dedicated some time to read the manuals. Also I got loads of help from other bloggers. So here I’m providing the little I learned from the experience. I hope it helps you.

First login as the super user. You need admin powers to execute certain commands.

1. Find the the vendor and product id of the modem.
You can do this in two ways
a) cat /proc/duc/usb/devices
This will give a list of the USB devices. Find the make of yours from the list. Most probably it will output something similar to

P: Vendor=05c6 ProdID=6000 Rev= 0.00
S: Manufacturer=ZTE, Incorporated
S: Product=ZTE CDMA Tech

b) You may also execute lsusb command
It will give another list and your device may be listed something like
Bus 001 Device 002: ID 05c6:6000 Qualcomm, Inc.

Note that even though the make of your device is ZTE, it is shown as Qualcomm here. It doesn’t matter. What we are really after are the hex codes for the product code and vendor id. We understand that the vendor is 05c6 and product id is 6000 in our case.

2. Now we need to detect the modem using modprobe command. Please input the following in the terminal.
modprobe usbdserial vendor=0x05c6 product=0x6000
This will detect the modem for you. Please make sure the ids are in hex (ie, there is 0x in front of the codes). You can automate this process by adding ‘usbserial vendor=0x05c6 product=0x6000‘ to file “/etc/modules“. This will execute the command each time the system starts up.

3. You may check whether it was successful by running dmesg command. If successful it will give output with lines similar to the following.
usbserial_generic 1-3:1.0: generic converter detected
usb 1-3: generic converter now attached to ttyUSB0
usbserial_generic 1-3:1.1: generic converter detected
usb 1-3: generic converter now attached to ttyUSB1
usbserial_generic 1-3:1.2: generic converter detected
usb 1-3: generic converter now attached to ttyUSB2

Now it means our modem has been detected and it is called ttyUSB0 by Linux. Now that we have finished the first part we are now into configuring the dialler. Here also you can follow two methods. Either by configuring kppp, the pppd dialler in KDE or editing the wvdial.conf file and using the terminal to dial in. I will explain both, though I prefer the former (If you Desktop manager is Gnome, you may not able to use kppp, but may be able to follow the wvdial method. I’m not sure, Please try and let me know).

Configuring kppp.

Please launch kppp (Where you find it under the start menu may differ for different distros. Please consult your distro’s manual for finding it). Yo will get an interface with a username and password option. At first it will be blank. You need to configure it before using it. Please click the ‘Configure’ button to open a new panel.

1. Under the many tabs, please make sure the ‘Accounts‘ tab is active and click the ‘New‘ button. Check the ‘Manual Setup‘ button in the new window.
2.A new window is opened with more tabs out of which the first one is ‘Dial‘. Add whatever name you like in the ‘Connection Name‘ field. Add new phone number. In BSNL’s case it is #777 or *99***1#.
3. Make sure PAP is selected under the Authentication field. The default is PAP/CHAP. Since BSNL gave ulogin credentials we need to authenticate ourselves with the server. PAP ensures it. Since the BSNL server doesn’t need to authenticate itself to our system, we can let go off channel handshaking and thus the CHAP protocol. This is atmost important whose failure may cause kppp to quit with an error code 19.
4. No need to bother with any other tabs. Let them remain as it is. Confirm the changes to come back to the initial window.
5. Now access the ‘Modems‘ tab. Click ‘New‘ to open a new window.
6. Enter a name for the modem and select ‘/dev/ttyUSB0‘ from the ‘Modem Device’ drop down in the ‘Device’ tab.
7. Now access the ‘Modem’ tab and uncheck the ‘Wait for dialtone before dialling‘ option. Confirm the changes to go back to the initial screen. Enter the username and password in the fields and we are 80% finished.

Now comes two of the most important configuration without which you are as good as you were in the beginning. If we try to dial in now, we will see that the dialer quits with an error 19. You will be searching in dark as you are sure there is no issues with the login credentials and you have entered the correct. Even re entering them won’t help. you will still stumble at the same block.

So what is the real issue here. You need someone to point out or extreme patience to read the manuals to find out. The kppp help is extremely good. It together with some online help ( rebooting OS’s alternatively to check the net and try the findings can be tiresome but is necessary at times like these) can eventually lead you to the solution. But don’t worry I had done all the hard work for you. All you need to do is do a couple more of fixes.

1. The “pppd died – The remote system is required to authenticate itself …” or the code no 19 error is caused due to two reasons.
Either the /etc/ppp/options contains the auth option or your system already has a default route. In this case recent versions of pppd will behave as if auth had been specified. To ward off the first issue add a ‘#’ symbol in front of it and to override the second option you may add noauth to the file.

This enables your modem to connect to the internet. But this doesn’t fully solve the issue. You will notice the browser is still not loading the pages. This time you will be in more despair as there isn’t a good error message along with it. But nothing is impossible. You are indeed destined to browse net from your Linux box. Just perform the next fix.

2. Add the following lines in the to the /etc/resolv.conf file

These are the ip addresses of OpenDNS servers. It’s secure, fast and trusted service. It’s better than the ISP’s servers ad I suggest you use them. You can learn more about them at the company site.

Voila! you are done and is now you are eligible for the ‘Geek of the Hour’ award. Happy browsing.

Editing wvdial.conf
Now for the command line method add a new file wvdial.conf in the /etc folder in addition to former files of resolv.conf and ppp/options file. Most probably the wvdial may be present you may just need to edit it. If not you may need to install wvdial. Check with the OS manual for the method.
Add the following to the file

[Dialer Defaults]
Modem = /dev/ttyUSB0
Dail Command = ATDT
init1 = ATZ
init2 = AT+CRM = 1
Flow Control = Hardware (CRTSCTS)
Username = <substitute yours>
Password = <substitute yours>
Phone = #777
Stupid Mode = 1
Auto DNS = 1

Save the file and execute wvdial from terminal and you are connected to the internet. Never close the window until you are done browsing.

There is one more issue we need to address. Kppp wouldn’t work normally if you log out of the root privilege. For a workaround see the end of this post.
I wish this serves as a good tutorial to using USB modem in Linux. Wish you happy browsing.

add to : Add to Blinkslist : add to furl : Digg it : Stumble It! : add to simpy : seed the vine : : : TailRank : post to facebook

I was always a Linux aspirant. Actually my fascination towards it started when one of seniors at college gave me a copy of Redhat Linux 9. I think it was back in 2002,way before the dawn of Fedora. After that I tried many flavours to various degrees of satisfaction and success, but never evolved into hardcore user. Though in my heart I wanted to be one, the factors that worked against it were the truth that neither I was given the brain of a geek, nor the phenomenon called internet which equipped the noobies like me to evolve into one. I was using a P3 desktop with a win modem back then about which I sadly came to know after long sleepless nights reading resources from various Linux help sites . Also I never understood the man pages. I still find it hard, but back then I was pathetic. So I usually uninstalled them after the initial phase of fascination worn out.

The distro which holds the record for most consecutive days in my system is OpenSuse 10.2 which clocked an amazing 2 and a half months. The experience was good. When this happened, I had already moved from my beatup P3 to a brand new Acer Aspire 3680 laptop, which is fast reaching it’s doomsday but still enjoys the privileges of employment. Though the processor degraded to Celeron from P3 it’s speed improved. So did the primary memory.

Anyway my experience with OpenSUSE was good. It automatically identified all the drivers which was indeed great relief. SUSE also showed great reliability and I was also able to connect it to net. But I couldn’t utilise it to upgrade my Linux gyan as I never had internet connection at home and the office internet wasn’t meant for personal use(The lamer always has excuses). So after remaining in my system idly for some time It had to make way for the higher demand of disk space. So I unistalled it and never ever thought of going back to it, until recently.

Now a days I am home doing nothing but refreshing my computer fundas. I had quit my job and is seriously preparing for GATE 2010. Most of the time I am in front of huge stack of books or video lectures. Wherever I look I see computer stalwarts only these days. And mostly it’s during these kind of times I go crazy and start believing that I am not meant to be a low life and I am destined to do big things in the field of computing. That”s when I erase Windows and power up my PC with Linux. This time I was not fully on high. I decided I give Linux one more chance. So I made some space in computer and looked around for some distros.

My Suse DVD was with me, but sadly my laptop decide not to recognize it. Now a days it won’t recognize most of the DVDs. Even if the disk has the smallest of scratches, the drive would start to cough and sneeze. I searched deeper in my cupboard. Out came Ubuntu (Breezy Badger) and Fedora 2 (Wow, it awoke pleasent memories. The cover still featured the “Future is Open” slogan penned down by my friend).

I have this big thing with me. If the nerds preferred a way I too would follow it even though I am a noob and there are other ways a noob should follow before aping the masters. So most of the time my choice of the distro depended on the opinions of the who is who of the linux community. So I thought a Debian based distro should be a wise man’s choice and opted for Ubuntu. But the badger crashed on me with a broken X Windows.

Though the true giants preferred the console, I was realistic about my ability and decide this is not going to work for me and I needed a GUI. Left with no other choice I loaded the Fedora. It was functional, but the resolution sucked. It reminded me of the XP before the drivers were loaded. But it was working so I was happy it would serve my purpose. The next step was getting internet connection. This time I was pretty determined to hook my linux to the net and hack around with it and ain something out of it.I am using the wireless USB contraption from BSNL. ZTE is the manufacturer and the box only contained drivers and manual for Windows. So it meant I’m left in a paradox. I need internet to get into internet. As I said I was adamant this time I rebooted most of the time to my XP to browse net and check for a solution. It was pretty hard. But I remained motivated. I was determined to succeed. But all my efforts left me with no solution. My box refused to cooperate.

At last I decided I need to upgrade. Most of the tips I found on net catered towards Fedora 8 or high. So I decided it was the fault of the version of the distro(Little did I know It wasn’t the version but the window manager which was at fault). I looked into the Fedora site. I was surprised to see Fedora has grown so fast. The latest version was 11. I was aware of the scene till Fedora 6, after which I lost track. It was like Leopold walking down the lanes of modern London. I booted up my torrent and after 38 hours, state of the art, off the mill Fedora was in my hands. I decided for the CD version, since I was sure my DVD drive is gonna go berserk if I insert another DVD.

Installation was smooth, except for the couple of times when the check utility complained about the bad bad checksums. But that turned out to be a false alarm and was rectified on a second try. I opted for Gnome for the window manager as it was default. I didn’t knew the pros and cons of the two windowing systems. So I never cared and it costed me big. The finished product was good. The looks were great and now I needed to check the functionality. After some minutes I felt the distro isn’t fast. I believe it’s my RAM and I think if I could plug in more I could get better performance. So no grudges on that grounds. But the issue that really made concern was a report of broken kernel now and then. But I decided it to let it go until something really broke.

Next was hooking into internet. This process alone took me full two days, except for the 6 hour sleep in between. I did all the known tricks of the trade but my system refused to detect the modem. I also couldn’t find the PPP client. So after trying out all the prominent techniques I was still left standing in the rain. Also after rebooting, the system decided it isn’t going to cooperate with me any more and started showing random errors, the ones which I obviously didn’t want at this juncture of time. Some of the errors were Latin to me but conveyed a feeling that the kernel is now a real mass of junk and he system needed a reinstall.

I was also fed up with all the numerous failures and needed time to rewind. So I placed the first cd in tray and rebooted. This time I decided to go with KDE and I thank for the moment. Because now my system would let me get into the internet and draft a blog from it. But this also wasn’t a piece of cake. It required lot of effort. But the difficulty can never be attributed to any of the features of the OS or KDE but only to my lack of knowledge. The difficulty I felt was normal for a first time user configuring the PPP. KDE was indeed a revelation. Commands which surmounted to nothing in Gnome were giving positive output in KDE. My modem was detected, the KPPP was configured and I was able to dial. But still I wasn’t able to get in. KPPP crashed with an Exit status : 19 message. I was sure my credentials were correct but it was saying I have a wrong username or password and I cannot be authenticated.

I once again rebooted into XP and browsed the web for reasons and found that I am not alone. Many respondents said that it was normal and BSNL responded like this at times. Some even said I need to get back to the exchange and get the modem reconfigured. But I knew this wasn’t true. If I could get it working in XP it should work in Linux too unless it is only meant for Windows. But that wasn’t the case and the web had several examples of EVDO users from Linux. So it was surely something I missed. So I decided o dig deeper. After so many rebootings I decided I am gonna read the friendly manual. So I read the help and the man pages and tried several more wrong things before I hit the jackpot. The authentication method in KPPP was the culprit. Once I fixed that I was able to log in to the net.

Though it was a long try I found the end result so rewarding. I now know what Einstein might have felt when he invented the electric bulb. Now that I am able to log into internet. I am going to use Linux for all of my computing unless the situation demands. I wish In a year I can become proficient in Linux. Till then it is lot of hard work and joys of sharing the end results.

The method to configure the Wireless USB modem in Fedora is given in the next post.