Take a closer look at the Popular Tags section.. now do you see it? Here is a higher resolution image of it.
For example, in the following free form text from Yahoo! News
John McCain's presidential campaign on Wednesday released a withering television ad comparing Barack Obama to Britney Spears and Paris Hilton, suggesting the Democratic contender is little more than a vapid but widely recognized media concoction
Knowing that entities like John McCain, Barack Obama, Britney Spears and Paris Hilton are names of people helps us understand the text better. It adds more semantic information to text mining engines. Also, knowing other entites such as Wednesday is a day could potentially be useful in building systems that can say generate an automatic summary of the political season in the United States.
Now, I was recently writing up a short paper for an upcoming conference and happened to paste some text from MS Word into a HTML text box. And guess what? MS Word seems to have atleast a basic named entity recognition engine.
Consider this example
Mountian View, CA is at the heart of Silicon Valley and is home to a lot of internet companies and startups such as Google, Meebo.com and SmugMug.com.
The Microsoft XML generated
<?xml:namespace prefix = st1 /><st1:City w:st="on">− <?xml:namespace prefix = w /><w:r><w:t>Mountain View</w:t> </w:r></st1:City>− <w:r><w:t>, </w:t></w:r>− <st1:State w:st="on">− <w:r><w:t>CA</w:t> </w:r></st1:State>− <w:r><w:t>is the heart of </w:t></w:r>− <st1:place w:st="on">− <w:r><w:t>Silicon Valley</w:t> </w:r></st1:place>− <w:r>− <w:t>and is home to many internet companies such as Google, Meebo.com and SmugMug.com. </w:t></w:r>
As one can see, even though it is not able to recognize Google, Meebo and SmugMug as organizations, the fact that it can handle places accurately seems to suggest that its probably nothing more than a simple regex. Nevertheless, that Word 2003 can do these things and is atleast five years old, when most of this technology was pretty much in its infancy is pretty darn good. I'm not sure how advanced this peice of technology is in the latest version of Word but seeing NLP techniques in places you don't expect to see is a good sign that we are moving to more intelligent software.
Signing Off,
Vishnu Vyas
]]>Moving 350 miles north and it feels like I'm in a totally different country. The laid back and the mild traffic conditions of the Bay Area are just a deceptive façade to buzzing business activity that lies underneath. This is a place that seems so placid and yet so active. Exactly, opposite to the high flying and colorful
One thing that I find amazing is that I get to walk past and "hang out" so to speak in the same areas across some of the new internet companies and websites that I use and love - be it meebo or SmugMug or Facebook. It is amazing to be walking past offices of internet properties with multi-billion dollar valuations and look at their tiny offices and wonder - "really, thats how big a multi billion dollar company looks?"
All this entrepreneurial activity has infected me in a small way and even for someone who is as risk averse as I am, has led me to ask the question - why not me? But that is for annother day, but till then ..
Signing Off
Vishnu Vyas
The story is set in a medieval fantasy world which is inhabited by humans and sentient demonic begins called yoma which feed on human entrails. Normal humans who stand no chance against these demonic beings depend on half-human half-yoma hybrids created by a mysterious and creepy organization, referred to within the series as 'The Organization' who send out female warriors wielding large swords (thus the name Claymore). The story starts out with showing these female warriors , the claymores, as beautiful and deadly but rather pale and without any humanity to them. They are feared and ostracized by the people who depend on the claymores to protect themselves from the demonic beings.
As the story progresses, we see another more human side to the claymores and learn that the organizations intentions aren't so noble after all. The storyline mainly revolves around a single low ranked claymore - Claire, her journey and her growth - both emotionally and as a stronger warrior as she picks up friends, comrades and battles a few powerful adversaries all on the way to finding out much more sinister secrets.
Even though the series focuses on Claire, the protagonist, Other characters are well thought out and are fully developed as the series progresses; they end up playing crucial roles alongside Claire and are important facets of the storyline. A complete and an entertaining anime overall that is completely worth the read!
Signing Off,
Vishnu Vyas
]]>
Unlike the world of C++, where the standard library is essentially small and can for the most part be held in your head, Java has a massive standard library which includes features for everything under the sun. Keeping that in your head is not for the light of heart. Thats where a good IDE like Eclipse comes in, especially the content assist features can make your life a lot easier. Also, the IDE can take a lot of pain out of managing your projects and libraries (because Java has a preferred conventions for writing and using libraries). One more amazing thing with Eclipse is that its got a nifty Java debugger which lets you inspect among other things - the threads currently running in your program. If you think eclipse is massive bloat or don't have access to a powerful system for doing your development, (some times I do, where I have to ssh into some machine and change, recompile, rinse, repeat if I am running experiments) then get JDEE (an Emacs plug in).
Ant is a simple Java build system based on XML (yeah.. yeah, you've to write XML, stop complaining). In fact, you can use their tutorial build file and start from there. It works with very little modifications, unless you have a special project needs (In which case, Internet is your friend). Say good bye to make files and man up and write some XML.
Java has threading built in, even though there are some pains with the thread synchronization, Java makes things a bit easier and the best part is on any Linux distribution with NPTL (which is almost every distribution out there), the Java threading model is essentially 1-1, which means you can use all those cores to do the hard work. This is great if you are running experiments under different conditions, in which case, each run of the experiment is essentially independent of others. This usually gives you a performance boost of more that 20-25% (well, but then, I have access to a machine with 8cores and 8gigs of RAM, so YMMV). Here is a really nice tutorial to get started on Java threads
For those who mostly work with input output formats from text files Scanner is one of your best friends. It essentially encapsulates the work of reading a line, splitting it into various parts and converting them into your favorite data types. And for all your input requirements learn to use the Console class, which provides a nice interactive console. If you the simple readLine method does not satisfy your needs, Java has a complete regex library, and again, the internet is your friend.
And for the last tip of the day, learn to use jconsole a monitoring tool for java applications which lets you inspect the amount of memory usage and active threads. If you are using this tool locally, then for the most part there is no configuration required - type jconsole at a command prompt and you should see a window with the list of process that are open and their PIDs. connect to one that you want to monitor and you are ready to track your java process. Things get a bit tricky with remote process and security issues - so my advice is setup a VPN or use ssh tunneling. More information on using jconsole is available here. So, thats all for now.
Signing off,
Vishnu Vyas.
]]>And sometime in the fast moving internet space is 3 days, and on the third day.. I check my logs and I see lots and lots random people from all over the world trying to hack my damn server. Well this story is not about them. So I decide to setup iptables - thats a pretty darn good idea, you might say.. except for one thing.. I don't know anything about iptables. So, after browsing for almost an hour on tutorials, howtos, message boards, google groups..(has anyone noticed the search in google groups sucks?) I still couldn't get anywhere.
Every tutorial out there seems to want to teach me what a TCP packet is or what link layer protocols are or the history of the whole IPTables filtering. Many would say thats great, you learn from the basics, you get your concepts straight. And to them I say "F*#$ you". I just want to secure my damn server, not take the RHCE. Finally after three more hours of digging and reading the various "subtleties" of the IP protocol, I finally maanged to figure out what to do to secure my server.
Write 2 lines. Yeah, just 2 lines - the result for spending 4 fucking hours is not enlightenment, just getting to write two lines. For those who are using mod_proxy and don't have linux networking guru to service you at your every beck and call, here are those two lines :
/sbin/iptables -A INPUT -p tcp -m tcp -s "yourhostname/ip/trusted subnet" --dport:"application server port" -j ACCEPT
/sbin/iptables -A INPUT -p tcp -m tcp --dport:"application serverport" -j DROP
Where "your-hostname/ip/trusted subnet" should usually refer to the machine on which apache is running, In my case, the same machine. The "application server port" is the port on which CherryPy listens, by default i think its 8080. If you have multiple instances of CherryPy running, you would need to add similar rules for each instance (note : add the ACCEPT rules first, before you do the DROP rules).
Signing off,
Vishnu Vyas
]]>Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.
-Zawinski's Law
Now that emacs has fulfilled its divine destiny via RMail, GNUS and various other add ons, the next step for emacs should obviously be something that most people have wished for - Multithreading. Emacs needs multithreading so badly that its almost impossible to do anything of significance especially with tramp comming included with emacs, I don't have to wait till a file gets saved over the network. I might just want to start making more edits right away. Inferior process are fine, but emacs and particularly elisp is still in the days of DOS. With multicore cpu's becoming commodity hardware, I think the biggest priority is to implement multithreading for emacs.
Thats all for nowSigning off, Vishnu Vyas.
]]>
Take a closer look at the Popular Tags section.. now do you see it? Here is a higher resolution image of it.
If you look at it, not only does it let you type tamizh in a phonetic sense, it also does away with the problems most phonetic typing schemes have when they are dealing with languages like tamil which have more liquids than I care to count, by simply using a well tested UI idiom - pop up suggestions. Its amazing that they do this in realtime in a web interface.
This is precisely what every one wants. No one, ever actually uses a tamizh all the time. Most of modern Tamizh's vocabulary is approximately 20% English. And, people can 'scrap' (orkut's equivalent to facebook's wall). each other just the way they would talk in real life. And all it takes to switch between the two is a simple keystroke.
So, Kudos to Google.
Signing Off,
Vishnu Vyas]]>
My thoughts when I initially wrote the previous post, where much unclear – a bit foggy and nebulous. Now, I believe that I have a more concrete version, or at least one that is a little less foggy. My study of the fundamentals of computer science has grounded my faith in the fact that this is as fundamental a science, as much as physics or mathematics, if not more. Computer Science studies things that can be computed by beings of finite resources – be they humans made of meat and bone or be automatons constructed of metal and electricity.
It is the study of what is possible in a finite universe. The nature of truths privy to us – beings of finite resources. What can be efficiently computed and what cannot. These are the fundamental questions of computer science. How hard is really hard? Are there easy ways to get around the hardness of problems? These questions are what a people who work on the frontiers of the body of theory that make up computer science ponder about. And unlike the esoterica of mathematics or the alienated theories of physics - The questions that computer science endeavors to answer have bearing on the real world.
From improving the efficiencies of every day corporations to fundamental questions about the market - questions that have plagued economists for decades, with the tools of computer science are within our sights. Our approximations to what cannot be answered precisely can be given limits of error. It gives us the tools to tell, if we are not at the truth, at least how far are we from it. From the fundamental building blocks of all life - DNA to questions about the most refined form of human intellect, Language. All are now under the sights of computation. What nature does and what can nature do? The ultimate limits of reality and our perception. All answers to these questions have their answers inside of computer science.
This article would require an entire book to be written, but for now, I shall contend my self with writing small expository examples via posts here, of the tall claims that I've made in the previous paragraph.
Till Then, Signing Off, Vishnu Vyas]]>