Tip to self: Check if a process is slowing my mac's internet connectivity

For intermittently slow internet connectivity, use nettop(1) in OS X to figure out if a process is choking the internet bandwidth. It is pre-installed.

$ nettop
Then hit 'd' and look for entries with consistently large values in the 'bytes in' or 'bytes out' column. The 'd' instructs nettop to only show differences in each screen refresh.

Practical usage notes:
If you don't recognize the process name, Google it.

If you don't want the process around, get the pid (the number next to the process name in nettop), and kill it with kill -9 <pid>. If that doesn't solve it, find out if you can politely tell the process to stop (for e.g. I logged out of AppStore, iBooks, iTunes and killed storeagent again).

If you're interested in what the process is doing over the network, use the client port number for a particular connection (e.g. '53133' from 10.0.0.1:53133->74.125.68.100:80) to run a tcpdump(1) on to see what packets are being exchanged: sudo tcpdump -nnvvXSs 1514 port 53133

If you're interested in who the process is talking to, but the destination IP address doesn't have a reverse DNS hostname (like in the previous example), then try visiting that IP address in a web browser as https://IP, click the broken lock icon in the address bar, and view the certificate details to find out which domain is served there (*.google.com in this case). This won't work if SSL port 443 isn't open on the destination.

On Hobby Startups and such..

You know, an odd thing has been happening to me recently. I've been drifting away from hobby programming, and drifting into hobby startups. For folks who've known me long, most projects I do (outside of work) are hobby projects. Programming and tinkering around has been a dominant theme.

But this is different, it feels different, and I'm suddenly getting yanked back to the unsureness I had back when I was a teenage programmer. I'm now a total newbie and beginner in a world of suits and businessmen, and wondering what I'm doing here. For starters, I'm currently totally immersed in this - I've taken a break from Amazon and am learning how to start a company. It sounds quite loopy (probably is), but in the university of life that's the learning I'm seeking. And as usual, I prefer the practical approach of doing it and make up/learn theory along the way, and iterate.

A good number of people start companies to promote their vision of the world, which I've heard is the right way to start a company. But the hobbyist/tinkerer itch I've had for a while is to actually learn how to start a company, just for the sake of that. And learning to do so much like how I learnt how to program - by just programming a lot (really, a lot). It was through lots of fun projects, painful hours of debugging and figuring things out, years of constantly realizing what I was doing wrong and in the process learning better things. It was fun, it led to a brief career as a programmer solving somewhat interesting challenges, and a countless series of hobby projects I loved.

Now this new branch of learning - of building a company to "program" your vision is turning out to be quite fascinating. This involves newer aspects of learning, where I'm learning (through repeated reality checks) the simple yet complex interplay between a customer, his/her problem, his/her solution to the problem through the purchase of your product, which in turn is your solution to how you understood the customer's problem, and improving the overlap in the two. There are other equally interesting aspects like keeping the company afloat, and dealing with the new, unexpected reactions from the people you interact with. If you aren't confused yet, good, because you either got it or your eyes glazed over.

But yeah, it does feel at a fundamental level like programming, except you're dealing with an extremely complicated analog system, and I've never programmed analog systems. Like my post above, logical contradictions are part of the whole deal. And you need patience, tonnes of it. There's no point getting frustrated with a computer, and there's no point getting frustrated with humans.

I'll try to post more on this as it goes, but I'm usually quite lazy in matters of posting :)

Cheers
~Shashank
PS: The thumbnail pic is the Antikythera mechanism, the world's oldest analog computer dating back to ~150 BC.

Oculus Rift at the Makerfaire 2014 in Seattle

I had good fun demoing the Oculus Rift Virtual Reality glasses to visitors at the Makerfaire 2014 held at the Amazon campus on Sat, 14th June. I walked into the Makerfaire at 9am in the morning with my demo gear, and can only describe the feeling as a kid walking into a candy store :) All around me were 3D printers, space elevators, robotic arms, quadcopters, a WallE prototype that mowed your lawn, a theremin, and other fun hobby projects. Yes, these were all DIY hobby projects made by the presenters.

Having failed to get my demo virtual reality game running in time for the fair, I decided to just show the visitors the amazing device the Oculus Rift is. Most of them had never experienced immersive Virtual Reality before, so they were definitely in for a visual and sensory treat :) There are a few good demo programs around - I used the standard Tuscany demo, a hungry Giganotosaurus dinosaur looming over you, Eden River: an ethereal glide over a river, the infamous Malfunction, and a video of a roller coaster ride. My hardware setup was a MacBook Pro connected to the Oculus Rift Dev Kit 1. Many visitors wore glasses, so I was glad I was well-practiced at changing the lens in the Rift quickly (the kit comes with 3 pairs of lenses of increasing power).

I was blown away by the sheer number of people eager to try the Oculus Rift, everybody it seems had heard of it but had never tried one. Most had never even tried a Virtual Reality glass before, so I wasn't kidding when I told them they were going to experience a new dimension soon, technologically and philosophically. My rough guess is that about 300 people tried a few minutes of demo each, most with their family and kids. The fair lasted about 6.5 hours, and it was fun to see people try to physically run back when a gigantic dinosaur tried to bite their heads off, or pensively stare at distant mountains off a balcony in a sunny Italian villa. Many parents lined up particularly to request the dinosaur demo for their kids :)

I was very happy to see many kids not older than 13 asking me detailed technical questions about how the
device hardware worked, the programming involved, the math behind the code etc. And they got it, way more than most adults did. It was also interesting that kids were immediately comfortable with using the headset and roaming the virtual world freely, while most adults looked around very hesitantly until I told them they could actually turn their heads and body around.

I really wish I could talk more about the other prototypes at the fair, but my booth kept me busy all through and I couldn't see most other demos, which sucked. I did manage to make a quick round to nearby demos, and was particularly impressed by a homebuilt hexacopter with a clever contraption to keep the onboard camera always looking straight ahead regardless of the turning of the 'copter. The gyroscope wasn't the interesting bit, it was how he had hooked together his feedback-correction motors. We also chatted about how he could attach a stereoscopic camera to his 'copter and look-through-the-eyes using the Oculus Rift while flying, similar to this guy. That's really trippy.

All in all, I'm really glad I attended the fair, and I'm excited about new interesting uses of the Oculus Rift, and hope the Virtual Reality party continues worldwide.

Until next time

I'm back

It's been a while since I last wrote about what I'm working on. My work at Amazon has been taking most of my time, especially over the past year. Other than that, I branched out into mind-hacking for a while which was fun, albeit a bit dangerous at times if you don't "mark the trees" to retrace your path back.

I've been really getting back into programming in C and server-side Javascript, which happen to be my two favorite languages, so I'm happy to get back the focus of being able to sit for, say, 6 straight hours and just write code and design things. You lose that a bit working at big companies where there's a lot of randomization, and less opportunities to write multiple versions of the same thing to pick the best approach, due to shipping dates committed to by a much larger group of people. But you accomplish way more because you can move faster without necessarily sacrificing quality. My biggest learning from Amazon as a developer I'd say would be the use of libraries for not writing everything from scratch. BTW on the topic of server-side Javascript - NodeJS, npm and CommonJS have really matured a great deal since I last lurked in the CommonJS mailing lists in 2009, and that's a great step ahead for Javascript becoming a language of preferred choice.

In terms of personal projects, I've been playing around with writing a neural network (to use the term loosely) that spans multiple EC2 instances, where nodes run as individual processes that can probabilistically fork and communicate with each other. I stumbled upon the Oculus Rift while working on that, since I needed a set of VR glasses to visualize the neural network in 3D, and it was insane! I haven't used VR glasses before, but if you look past the lens-magnified pixels and blur, the feeling of being present elsewhere was very very good. At $300 for the developer kit, I think it is well worth the money. I don't care too much for the Facebook acquisition, since I highly doubt Mark Zuckerberg sees Facebook as just a social network and would be surprised if the social aspects get kludged into a VR headset. The raw number of possibilities with the Rift is so immense that there is interesting stuff for everyone, beyond just repurposing a social network to work in 3D with head-tracking. Good protection against Sony's muscle, is my best guess.

I've restarted my study of Physics, and I hope to research alternatives to rocket propulsion this year in my spare time. I still don't have the money required to make this more than a hobby, but I'm an optimist and hope I can fund this interest at some point in my life. I'm currently taking classes for Muay Thai and Boxing, which is the hardest I've pushed my body and is really fun. I think I should learn driving at some point too, which is ironic for a guy with dreams of traveling to space. I spent many months last year teaching myself improvisational guitar playing, especially with scales and modes for playing solos, and combining different influences from a really wide range of bands from the world. It's a really good way to relax, and I continue to believe that everyone in the world should learn an instrument to play music on.

I hope to post more frequently in the coming months with what I'm working on.

Until next time

Sun TechDay at VIT University

Sun Microsystems recently came to our college to conduct a TechDay (also called: "SunDay", which co-incidentally was held in our college on a sunday too!) on March 1st, 2009. The event comprised of four workshops - on J2ME, Solaris, and JavaFX. We had to conduct two parallel sessions of J2ME instead of one (so J2ME counts for 2) because the response was so great -- we got double the no. of registrations we had expected! The expected nos. for each session was 70 which works out to 210 in total for the three sessions, but we registered 162 for J2ME, 85 for Solaris and 124 for JavaFX = 371 registrations!

The engineers who conducted the sessions were excellent, the sessions were really informative, and we're already receiving requests for another TechDay from those folks who missed out on the registrations as well as those who attended (for sessions on advanced topics). We're planning another version of this somewhere at the start of the next semester.

The task of organizing the entire show fell upon the Sun Open Source Community that me and Aditya had kicked off in December last year, and the team of 9 core members and 23 associates that was selected did an exceptional job! Frankly I gave a pretty conservative estimate to the Sun folks when asked initially about the participation, keeping in mind the previous events, and other issues like 1st being a sunday and the SEDS India National Conference happening at the same time. But the members of the Sun Core and the Sun Associates did a fabulous job in getting so many registrations, ensuring that all the ground-operations on 1st went smoothly and basically that everyone had a good time. Kudos to them!

Cheers

Google v2

Been a long time since I wrote, and as comments to my last post prove that I have a fan following of critics now, waiting for new posts, here it is:

The previous month has been really hectic, as we're working very hard (and at great speed) to complete our project for the finals of Google Product Prodigy '09. BTW, I should first inform you that we guys got selected to the finals of the Google Product Prodigy contest (again, for me) from the entire Asia-Pacific region! People who've read my previous blog (at awardspace.com, which seems to be dead now) would recall the post I made describing the fantastic experience I had at GooglePlex Bangalore last year. We're pretty excited at being able to go through that again!

The other interesting project me and my friends are working on presently are an Adaptive Web Search engine, that adapts to a user's search-usage pattern over a period of time, so that after a while, the search engine starts delivering results tailored to the trends the person has set over weeks.

We're also about less than a month now to the launch of Flare (our Google project), which will be launched as a web service! It will initially be invite-only. :-)

Cheers!
Shashank

LSJ: The Blue Screen of Megadeath

Shudder. For those of you who have used Microsoft Windows XP, be afraid, be very afraid. Because the British Navy has just upgraded its nuclear (powered and deterrent) submarines to the state-of-the-art control technology -- Microsoft Windows XP. A stripped down version of it, if that be any consolation, which it is not.

Check out the Linux Journal article: Blue Screen of Megadeath

"We can't speak for anyone else, but the thought that somewhere floating out there are two hundred nuclear warheads under the control of Windows XP scares the living daylights out of us. If anyone needs us, we'll be building a bunker in the back yard before one of the boom-boom boats throws a blue screen of death and blows us all to kingdom come."

Boo!

~Shashank

Mounting ext2/ext3 partition in Windows

Having no Linux disc at hand in order to live-boot and see my "Back2Basics" root filesystem files in X Windows (I removed X Windows as well, remember?), I found this interesting tool for accessing the Linux ext2/ext3 partitions from inside Windows. It works with almost all versions of Windows, but doesn't do reiserfs, and others.

Check this post out: Post

Download the tool from here: Download

Cheers

Shashank

Real-time color scheme generator

Here's a very interesting and ingenious tool I found on the Internet: A real-time color scheme generator that does a Yahoo Image search for the term you search and returns the most prominent colors in the images returned.

Here's what they say:

"We take 5 related images from Yahoo Image Search and then show the most prominent 6 colors from each image. Each column of colors is taken from a single image."

Check it out: link

Cheers

Shashank

PS: Why the hell is Google displaying Visual Ads now? It is just going the same way as the other old internet-advertising agencies went - ugly, intrusive and heavy picture ads, and some of them even Flash-based! So much for light, elegant and unobtrusive text ads.

Day two: Back2Basics Linux

Having vowed not to spend my 10-day winter vacation doing anything serious, I decided to play a bit with Linux, apart from reading the novel "She" by H. Rider Haggard (of Allan Quartermain fame) and revisiting the Beatles' album collection I got hold of just before setting off from college.

Anyhow, one of the first things I did when I got home was to strip off /everything/ from my ancient Ubuntu installation in the PC at my home, and essentially removed ALL the packages other than the important core and base and essential ones. This brought me down to the beautiful command line at the next reboot, and then I to do a little tweaking with the /etc/init.d/rc to display some messages and request for user login, instead of asking for root login (it was configured to spawn /sbin/sulogin, instead of login) and such. All this led to a cool "Back2Basics" Linux, which still takes around 800 MB because I haven't removed a LOT of unneccesary libraries incase I break something (I've done that once before). So I'll proceed to do the stripping work soon (hehe). I got hold of this O'Reilly book on Building Embedded Linux Systems sometime back and it lists this minimum required directory structure for the root (/) filesystem, and another book on Embedded Linux (whose publisher I cannot recall) which listed the bare essential binaries and the corresponding libraries required. So I'm planning to base the stripping-down on the guidelines in these books, and of course using the ldd command to find the dependencies of the desirable binaries incase I want something extra. I replaced the GNU utils with Busybox yesterday. My target is to just have the Kernel Image, Busybox, Perl, and Java. Lets see what comes of it. ;-)

Also, I'm working on a new categorized layout for this blog, something I'd developed two years back but never implemented. Its going to automatically pick the most popular categories and show posts filed under those in square boxes, and the more popular categories will be higher up in the boxed, 3-column layout.

Cheers
Shashank

Network scanner script

One of the more useless handiwork of a wasted 2 hour lab session:

A shell script for scanning the network with Class C IP Addresses from 192.168.0.1 to 192.168.255.255 and listing the active IP addresses. It writes it to a file called IP_list as well as echoes it.

ip_scan.sh:
-------------------------------------------------------------------

#!/bin/bash

# A Useless network scanner
# by Shashank Shekhar
# - Scans the Class C network from 192.168.0.1 to 192.168.255.255
# - Note: Ping doesn't always return the true state of a network node
##

for ((i = 1; i < 256; i++))
do
 for ((j = 0; j < 256; j++))
  do
   ping -c 1 -w 1 192.168.$j.$i > opFile
   cnt=$(grep -c -i " 0% packet loss" opFile)
   if test $cnt -eq 1
   then
     echo "192.168.$j.$i is there" >> IP_list
     echo "192.168.$j.$i is there"
   fi
 done
done

Cheers
~Shashank
PS: I was so bored.

Edit: Evil text-garbling spirits, Stay Away!

Life in other places. What is Life?

The question of whether there is life in other places in the Universe requires a fundamental question to be answered: What is the definition of life?

This is not a spiritual discussion on Life and its meaning, but for those who thought it was, do read on. Fresh viewpoints are always stimulating for spirituality.

People tend to have very varied and often biased views with regards to their definition of Life. Most would think of a grotesque humanoid form with a head, two arms, two legs as an instant reaction to the question, while others might take more colorful forms: octopuses, crabs, giant ants etc. These have been covered extensively by countless science-fiction material, books, movies etc. These view points are flawed at several places:

a) These replies do not even answer the question: "What is the definition of life". These just provide the answer to the specific question: "What would living beings look like in some alien planet". A sea of difference.

b) They all assume that the physical and environmental conditions required for life are similar to that of the Earth. The fact that life could exist at 12,000 degree centigrade is just absurd, in these viewpoints. But living cells have been found on Earth in the most extreme conditions [1].

c) They all assume that "Life" is Carbon-based, and in-fact, material. What I mean is, they assume that the dependence of Living beings on Water, Oxygen and Nitrogen is universal. This "life" that people talked about involved Amino acids and nitrogenous compounds forming the basic genetic material and cell structures that coordinated to form the diverse life-forms that we know of. But this entire set of known life-forms are mostly based on Nitrogenous compounds with dependence on Hydrogen, Oxygen and Carbon. Assuming that life could evolve only from these is a precarious one.

I do not claim to have the correct answer. The purpose of this article is just to try to broaden the scope of thought and opinion of what is living and what is not.

What is Life? The property that makes entities living. Thus if a set of rules determine that the entity in question is "living", then it has "Life". Hence the question narrows down to determining the set of rules that declare an entity living or non-living.

This is where things get fuzzy. I'm afraid there will never be a conclusive, definitive and "correct" answer, but just our viewpoints. Just like our viewpoints of what is "right" or "wrong" determines laws and so-called morality.

I think an expanded and relatively broad-minded checklist for a single "living entity" would be:

a) Should be responsive to external environments and stimuli
b) Should be able to self-sustain for as long as possible in the environment
c) Should be able to change itself and adapt in order to continue existence

Is reproduction a necessity to certifying something "living"? Probably not. Because a single male or female of the human species, or any sexually reproducing species, cannot reproduce without involving two entities. Thus a single man would not be able to reproduce alone, and would hence be declared non-living by following that logic.

Note: Do not read the following lines if you have had enough mind-expansion for the day.

This expanded view brings us to a series of interesting ideas. Could life forms exist in electromagnetic waves? Why make the assumption that life needs to have material composition? Electromagnetic waves are doomed (in a way) to keep propagating ahead in space until their energy is absorbed by any entity they collide with. But what prevents us from calling a man who keeps running non-stop until he vaporizes by running into a blast furnace as living? You'd say he's living even though he's constantly running non-stop because he can change himself, he's interacting with the environment and external stimuli, and because he is... well a man!

But similar is the case of an electromagnetic wave. Its a bundle of energy, like every man or for that matter, "matter". If some quirks of nature in some part of the Universe would cause these bundles of energy to self-sustain on loss of energy, and suit themselves to the surroundings, all while running along at the speed of light constantly, would they be called Living? I believe they would, and should.

Cheers
~Shashank
PS: Image: X-ray of the left hand of a ten year old boy with polydactyly.

The Curse of the Gravel Machine

Trudging through college life, we come across several obnoxious myths, stories and legends belonging to our college. One such legend I came across is that my college is a cursed one. The curse is that the moment it stops constructing something new, building a new annexe block, pulling down a perfectly nice place and building another... the moment it stops constructing... It will cease to exist! Just like that. God's WILL.

Obviously its obnoxious and untrue, or so we'd hope as long as we're here. But the construction bug has bitten my college since before the guards were toddlers (or so they say), and you're bound to collect different proportions of sand, cement and pebbles in your slippers depending on which path you take across the hostel campus.

Oh and I'm on a break from blogging, which is an odd statement to make since I blog in intervals of decades anyway. I'm also on a break from nosy-technical-fun-&-games since there are a couple of exciting new projects I and my friends have embarked upon which is taking up all my time.

Cheers
Shashank

Some sad news

Been busy with exams, and work on the latest project which we're submitting for this year's Google Product Prodigy. But this post is not about that.

This post is about the tragic passing of my friend, Ishan Goel, who passed away a few days back. He was just 20. He was a colleague in the Sun Ambassador programme, i.e. he was my colleague at the Sun Open Source club we were founding here at college, and worked very hard with me in making the Software Freedom Day (September 2008) a success, and also a member of our last year's Google Product Prodigy team (which I covered in great detail in a series of blog posts previously), which we worked for close to 6 months together. A brilliant and immensely creative fellow, and a "Linux Helpline". This is not an obituary, or any formal note of sort. It's just hard to absorb this. My hand still tries to dail his number whenever a problem with my Linux system occurs.

~Shashank

Hello World! iPodLinux - Java style!

October's been a good month for me, and better for my iPod. iPodLinux.org is back online and they've released a couple of pretty neat tools, and updated builds. Loader 2 (the new bootloader) is great - no more messing around with complicated wheel/key combinations to select the boot OS. Loader 2 gives you a GRUB-like list of installed OS images on the iPod disk, and can read kernel images from ext2 and ext3 partitions, as well as FAT32 (W95). So this effectively removes the need to re-create the firmware each time a kernel update takes place. All you need to do is use ipodpatcher (which originally was used for installing Rockbox) and install Loader 2, which will now get merged with the Apple Firmware.

The Podzilla interface has been improved too, and lots of annoying little problems have been ironed out mostly. The text-input is (or atleast, feels) WAY better now, and I can actually input something meaningful in a meaningful amount of time. Here is the text file I "typed" (with awkward scrolling and tapping) on my iPodLinux's PodWrite with a "Scroll with Prediction" text-input mode:

first.txt : "This isn't bad at all. The prediction algo is pretty good as far as this text input session is concerned. Took me 10 minutes to input this on the iPod, but I'll improve I guess."

Then I finally got my first Java program running on the iPod! After encountering wierd errors and battling nasty iPod text-input mechanisms for months, things have finally come to a closure. The latest build of podzilla, makes things easier, and thus enabled me to run Java programs using the K Virtual Machine (kvm) by Sun Microsystems. I got hold of the kvm binary from some place, but I don't think its available on the Sun website anymore, which is pretty odd. But this is how I got my "Hello world" in Java working on the iPod:

1. Install iPodLinux by using the latest Podzilla and loader.
2. Download kvm from here. Put the kvm binary into your iPod's /bin folder and make sure it has execute permission.
3. Write your java code, but keep it simple. KVM just supports basic I/O, data types, Vector, Stack and a couple of other simple things. No graphics, gui etc. For e.g.:

Test.java:
public class Test {
public static void main(String[] args) {
System.out.println("Hello World!");
}
}

4. Then compile the code to the JDK 1.4 format:

#javac -source 1.4 -target 1.4 Test.java

5. Finally, create a script file in gedit or vim called, say, Test.sh:

Test.sh:
kvm Test

6. Save the script file in the same folder as the Test.class file, and give both Test.sh and Test.class execute permissions:

#chmod +x Test.sh
#chmod +x Test.class

7. Put all these files in some folder in the iPod's Linux partition. I put it in "/home/shashank/java" (on the iPod partition, not your own system! Also avoid spaces in your folder path.). Then boot into iPodLinux and open the folder where you placed the files using the File Browser, and scroll to "Test.sh", and keep the "Select" button of the iPod pressed until the context menu shows up. Then select "Execute" and select "Read output". Voila! You should get the two words: "Hello World!" on your screen!

No cross-compilation, no recompiling of gcc to work with ARM7 processors (most iPods run on ARM7 processors, thus C code compiled on your PC will not work on the iPod). Life is easier with this if you want to do simple programming. Maybe if someone wrote an extended Virtual Machine, with more libraries, then it would make this a pretty solid platform.

BTW, the above steps have been written assuming you have a Linux/UNIX machine. If you have Windows, a VMWare or xVM VirtualBox image of some Linux distro on your Windows machine would make life tonnes easier compared to trying to use Windows' limited functionality with everything in life in general.

Cheers
Shashank

Possible loophole with dd command in multi-user Linux servers

Possible loophole with dd command in multi-user Linux servers

I didn't think this would be a big deal, but I noticed that several remote logging Linux servers permit users to have an unlimited disk quota, which can easily be exploited using a command like:

~$dd if=/dev/zero of=null.bin

It doesn't require the user to 'sudo'. I checked it out in one of the remote access servers that need to be logged in to using Telnet, and I executed this command and was surprised to see it being permitted and having a free and unchecked run. Effectively, /dev/zero is a virtual disk that can generate a series of 'zero' bytes, and the 'dd' command will copy from the input file (i.e. /dev/zero) to the output file (null.bin here) and will not terminate until it reaches the End of File of /dev/zero. And because I didn't give it any count or bytes limit (which can be done using the bs=xxx and count=xx parameter), and since it never reaches the end when reading from /dev/zero, it will effectively create a massively huge file within seconds, and fill up an entire normal-sized hard-disk in about a minute and a half! I tried it out and in less than 7 seconds, I had created a 962 MB null.bin in my home folder. And then I tried on a virtual machine (Ubuntu 8.04 Hardy Heron running in VMWare on Windows) I had created, with a fixed disk-space of 3 GB, and in about 10-15 seconds my 'dd' command died with a "No space left on disk" error message.

I'm not sure if its really a loophole or not, but it certainly made my home directory expand to 1.2 GB unchecked within seconds in a remote telnet login to a Redhat Server. If it is a loophole, I think it could be solved by assigning a strict disk-quota to each user, encompassing the total areas where he can write to.

~Shashank

How to boot from USB in xVM VirtualBox

I've looked around the internet a lot and mostly didn't find any working solution apart from trying to make an image/virtualDisk from a raw physical disk (USB). I was basically experimenting with writing simple bootloaders in assembly language, and wanted to rapidly test and make changes to the code without rebooting my system again and again to try booting. The obvious solution is to use a Virtualization tool like VMWare or xVM VirtualBox by Sun (the latter is free), but the trouble is that neither of them support booting from pen-drives and I have long forgotten the existence of floppy drives, and have thus no way of directly writing to a disk and virtual booting straight-away, without creating an new image of the disk each time I make changes to it. So I decided to write to my USB pen-drive directly and try booting from it. It works when I boot the entire computer with it, but VMWare and VirtualBox do not have the option of booting from a USB device.

So messing around xVM VirtualBox 2.0.2 36488 for Windows yesterday, I was pretty surprised to find my USB Pen-Drive listed under the field of Floppies! That is,

a) Create a new Virtual "Machine"
b) Click on the "Floppy" link in the main screen (right panel)
c) Enable "Mount Floppy Drive", and from the drop-down list in "Host Floppy Device", select your USB drive's Drive Letter (haven't tried it in Linux, I guess you'll have to select the correct device like /dev/sdxx)

That's it. Try starting the Virtual Machine now.

Its quite wierd, and I'm still wondering why my USB drive gets listed under floppy, but it works for me! Neither VMWare Workstation, nor xVM VirtualBox permit booting directly from the USB otherwise.

Cheers
Shashank

Of the Elusive Breakfast Syndrome

I really have a problem, or rather had one. It is an uninteresting topic for anyone but me, but I haven't managed to have breakfast on any weekend this ENTIRE semester i.e. in the past 4 months. I have never managed to wake up before the closing hours (9 am) of the hostel mess (often I directly wake up for evening snacks on weekends, let alone breakfast and lunch). I have even attempted, several times, to stay awake the entire night, till the golden moment when the clock struck 6:45 am, and each time some odd conspiracy of Nature conspired against me. Once I watched a movie till 6:35 am, and then pondered over how to pass the intermediate 10 minutes, and then continued pondering as I next regained conciousness at 10:30 am. Then once I managed to sleep off on the floor of my room, beside my door, while trying to open it to leave for the mess, again at 6:35 am.

Today, however, I grabbed Fate by the wrist and bent it to my will! I left my room at 6:30 am, as I was sure I wouldn't manage to survive any longer after watching Lord of the Rings III (for the 5th time), and also figured that I probably wouldn't sleep off in the corridor beside the mess, though I'm not sure I'd put that past my capabilities. Anyhow, here I was - unshaven, unkempt, with a haggardly-desperate look, sitting outside the stairs of the mess waiting for it to open. Then I tried walking around the area a bit, and I discovered I didn't like the cold air too much, and again returned to my patient look-out post. And then, the Gates of Destiny opened! And I had wonderful breakfast, and Tea! One of the greatest battles of will and destiny in my life had been fought, and I had emerged the victor - a fed victor at that. :-)

Shashank
PS: I haven't gone loopy. Food breaks the best of men, I'm just a mere, humble mortal.

DMOZ Local Directories and BatchDMake

Today morning I finally settled in to perform a long-pending weekend house-cleaning job, except that the cleaning work was electronic, i.e. my computer. It is a common scenario to find your files so badly scattered around the hard-disk, inspite of some organization scheme that we all implement. Years ago, I saw a nice scheme Roshan, a friend of mine, implemented. He created directories and sub-directories based on topics and categories, like "Science", "Computers", etc, and Science would contain "Physics", "Chemistry" ... while Computers would contain "Programming", "Internet" and so on. It was a complete hierarchial structure, and I've been using it on my systems too ever since I saw it there.

But, its hard to maintain my directory, as creating such a structure is a rather tough task. You could either create an entire directory structure before-hand, with empty folders waiting to be filled, or you could create new folders as and when you require, in the correct category of course. The latter option obviously seems more efficient, as you don't have an empty mammoth directory structure, but a dynamic and to-your-taste structure. But after several years of trying out both, I have found that the first approach is much better. When you are, say, saving a file from the Internet, you often don't have the patience to ponder over the hierarchial relationship between "Geese Hunting" and "Duck Hunting", and whether you should create a new directory for each, or just club them into one single directory called "Bird Hunting". This inherent impatience of people when using files, as understandable and natural it is, is THE reason why files are scattered and chaotic. Thus I have concluded that people need an existing order to follow, that is as extensive and accomodating as it can be, but also flexible in order to allow people to delete and modify and create new topics and directories.

With this long discourse on the philosophy of file management in mind, I sat about thinking! :-D

This is what I came up with: Why not use an existing MASSIVE topic-wise directory structure, which has been developed and nurtured for about a decade by tens of thousands of people across the world? That directory structure is the DMOZ Open Directory Project (http://www.dmoz.org) by the Mozilla Foundation (of Firefox fame). It forms the backbone of Google Directory and several other places on the internet, and is a collaborative and open effort at building efficient web directories. My interest, however, is not in the links to websites that is provides, but the topic-based organizational structure. And as luck has it, they offer a downloadable plain-text file containing the ENTIRE folder structure. Perfect. Its available at:
http://rdf.dmoz.org/rdf/categories.txt (57.4 MB)

Then I wrote a simple tool in Java called "BatchDMake", which will read a given text-file line-by-line and create the folders listed in it. Each folder path is listed in a separate line. For example, this is from the DMOZ dump:

Arts
Arts/Movies
Arts/Music
Computers
Computers/Artificial Intelligence
Computers/Games

The syntax for operating it is:

java BatchDMake [filename] [root folder]

The filename parameter is obvious, the root folder is the folder name relative to the current working directory in which it should create the folders listed in the text file. For e.g. using the downloaded DMOZ categories file:

java BatchDMake categories.txt dmozRoot

The source code for BatchDMake is available here, and the executable .class file is here.

I just created the ENTIRE folder structure of DMOZ on my hard-disk, it has about 770,000 directories & sub directories. A lot of them are useless ones, so I'll go about pruning the tree to come up with the most useful structure for normal users. Will post that one soon. In the meanwhile, you could try the DMOZ structure and use BatchDMake to create it on your system!

Cheers!
Shashank
PS: I even found the place I'm currently in!
root\Regional\Asia\India\Tamil_Nadu\Districts\Vellore
This is like exploring an uncharted world in your hard-disk! :-P

Software Freedom Week 08 - Phew!

Note: This is a blog post I'd written in a state of half-sleep at 3:38 AM, 27th September, 2008. I slept off before managing to send it and forgot all about it, so here it is. :-)

Sitting at 3:38 AM, while browsing the Internet for some interesting articles, I found I had received a new mail. Looking into my Inbox, I discovered that the mail had been entered into my "Friend Networks" label, and it turned out to be an insistent invite (he had sent it thrice) from a friend in a social networking site, and the invite reads: "You have been invited by ____ to join the community: I LOVE FRUITS". Need I ask, "Huh?"

This was meant to be a blog on the "Software Freedom Week", which more or less got concluded today. Technically, it finishes tomorrow, but the only event we're having tomorrow is the passive collection of Abstracts for the Product Engineering contest and a party! So no crazy real-world events that require fighting with the "bureaucrazy" and stuff. The last one week has been a very hectic period, which reminded me of the days of our "IEEE Resonance", the two-day National Level Technical Fest, held in March earlier this year. That event was gigantic and I, as one of the 3 main organizers, probably lost several years of my lifespan due to the stress and workload that went into organizing that event.

This Software Freedom Week was organized together in a VERY short span of time. Really. It sets a sort of personal record, as the only Large Scale event I've organized within a span of 9 days. The other large events have me and the core team starting work atleast 3 months in advance. The entire idea that we Sun Campus Ambassadors have to organize a Software Freedom Day in our colleges by 27th September became clear only once we had done our Induction & Training programme at Noida from 5th - 8th September.

Our SFW went off pretty well! We managed to conduct several successful installfests, and installed and distributed openSolaris DVDs to atleast 30 folks! The other guys installed some distributions of Linux, like Ubuntu, Fedora and openSUSE. Then our Linux & Unix demos were quite successful with a lot of interested people turning up who managed to get an intro to the basics of the file structure and philosophy of the Unix operating system, as well as some common tools and commands. Our Programming challenge paper was very well appreciated by all the students as being "damn interesting to solve". The biggest success was the "Mobile Application development using NetBeans" workshop which I conducted yesterday evening (26th). The turnout was HUGE and the guys were amazed at the ease of developing MIDlet apps using the Visual Editor. There was a pretty strong request for conducting an advanced session for them soon as they wanted to learn more about this. :-) And just to recall that I myself had been trained at this for the first time 3 weeks ago in Noida!

Organizing large events at a very short notice leads to nightmarish organizational problems, and can easily spin out of control to lead to horrible mis-management and utter chaos. We came close to it during the hiccup that happened on the first day with the venue conflict with another department's research class, and it took us a great effort to continue with full speed throughout the week inspite of the regular, minor-but-painful hiccups that occured. We always managed to solve them somehow, but the fact that they appeared shows that a lot more work could have gone into the planning process. Yet, I do not regret anything, and even though I am so totally physically and mentally exhausted, I can still say that the SFW was one of the most challenging and awesome things I and my friends have attempted. And I enjoyed it and the great response it generated. But now we're planning a FAR larger thing in December. A sort of a Open Source Mega Fest at the top levels.

More soon, I'm too sleepy!

Cheers
Shashank
PS: No technical work has been possible over the past one week
PPS: Metallica released their new album called "Death Magnetic". Most of the songs are just average in my scale. I like "Cyanide" and the intro of "That was just your life". And Kirk Hammet is finally playing guitar solos again. His solos were cut out of the last album: "St. Anger", which was a welcome return to Thrash Metal. In my opinion, all the songs on the St. Anger album sounded like an army of tin-soldiers sprinting over a tin bridge, and a song playing in the distant background. Heh!