Archive for March, 2010

Into the Baltics

After doing a longish stopover in Helsinki for three and a half days, I took the Linda Line Cruise into Tallin, Estonia – A distance of just 80 kms across the Baltic Sea. While planning the Europe trip, the Baltics were never part of the plan. Estonia, Latvia and Lithuania were part of the USSR before it disintegrated in 1991. As a young kid I saw this on TV, as the USSR broke into independent states. At that point, I was saddened (Remember Misha – the Russian Kids Magazine ? ) as it probably meant I would not get any of the cheap Russian books that I had grown accustomed to. Over the years, I had been following the news about new-born states as some fell into chaos and dictatorship and some prospered as they broke the shackles of communism and the USSR and adopted free-market policies. This was a great chance to visit them and see for myself. As I started reading about them, I got more and more interested and decided to visit them all for a brief while.

I traveled through the Baltics from the north to south before flying to the North of Norway (Tromso – north of the Arctic Circle). I traveled through the cities of Tallinn and Parnu (Estonia), Riga, Salaspils, Liepaja and Cesis (Latvia) and Vilnius and Trakai (Lithuania). Though the Baltic countries have shared history and close ties they are different in several ways from each other.

Estonia is a confident country that has wholeheartedly adopted new technologies and free market policies. I met several people in Estonia who were planning to start their own businesses or already running them. The adoption of mobile, wireless technologies and Internet was high. Random factoid – The founders of Skype were from Estonia. It also helps that Finland is just across the Baltic seas so there is quite a bit of expertise and investment coming from across the border. Economic activity was buzzing and people were cheerful, friendly and optimistic about the future. They were well informed about the environment – An example was an internet mobilized cleanup of the forests – one of the largest in the world. Also most people were atheists (A survey suggested that Estonia had the highest percentage of atheists). Estonia has just 1.3 million people but for a small country it has achieved a lot in the last 2 decades since independence.

Latvia has the most fabulous (and most populous) capital – Riga – amongst the Baltic states. When I was there, it’s economy was going through a severe recession as the real estate bubble (fueled somewhat by money from the UK) had burst and the GDP had fallen a staggering 20+ percentage points. About half of Latvia’s population lives in Riga – which is also considered as one of the unofficial Capital of the Baltics. Riga is relatively modern. Riga is a photographer’s delight with wonderful architecture – right from the medieval ages to the bland Soviet style buildings to the swanky steel and glass towers of the 21st Century. Some of the local economy is fueled by the tourism from the UK (Notoriously young Britons who come to the Riga for sex and booze). Undeniably, Riga has a dark edgy underside which it tries to hide. However the people are friendly (almost everyone speaks English) though the older generation seems to be somewhat nostalgic (and conflicted) about the Soviet Era.

Lithuania seems more influenced by Eastern Europe (It shares a border with Poland in the South) than the Scandinavian countries. It’s beautiful capital Vilnius is rich in culture but seems more provincial and backward when compared to rest of the Baltics. Vilnius had a sizable population of Jews before World War II and was called Jerusalem of the North. Few people speak English and getting around is not very easy. Like Latvia, Lithuania was in the midst of a severe recession and it was apparent even in the city. I passed several factories that had closed down and a car factory that had it’s huge temporary parking space full of cars as people could no longer afford to buy them. There were dilapidated buildings around the outskirts (Where tourists typically never visit) and people still lived in cramped soviet style apartment blocks. Lithuania is still dependent on Russia for large amount of trade and there is a sizable Russian population in the country which is also represented in Parliament.

UNESCO World Heritage Site – Soumenlinna

King's Grave

Soumenlinna (literally Finland’s Castle in Finnish) or Sveborg (Fortress of Svea in Sweden) is a UNESCO World Heritage site built on an a group of islands south-east of Helsinki in the Baltic sea. It is accessible by the water ferries that ply across the sea from Helsinki. The ramparts of the forts are star-shaped and consists of many fortifications made of large mounds of sand. (called Skansen in Swedish – literally meanings fortifications or fort walls).

Bunkers / Storage / Skansen Soumenlinna - Sea Fortress

The fortress itself is inhabited and there are a number of cafes and restaurants to cater to the gaggle of picnicking Helsinkians (Not Helsinkers !! however tempted you are to use that word !!). The fortress also has Finland’s smallest official beach just a few metres across (see below). Suomenlinna was used during the Second World War as one of Helsinki’s air surveillance centres and served as a garrison until 1972. Today it houses the Finnish Naval War Academy. It also houses the Helsinki open prison. A substantial part of the repairs to the walls, ramparts and buildings are carried out by convicts.

Beach Quaint house

Interestingly, Soumenlinna was built by a Swedish King (when Finland was still a part of Sweden 250 years ago) to guard against Russia whose increasing maritime power in the Gulf of Finland made the Swedes uneasy. Soumenlinna contains a shipbuilding yard that is now defunct. The Soumenlinna Church was built originally in Greek Orthodox Style with five onion domes. Later it was converted into Lutheran Church (Most Finnish are Lutheran Christians). Today, the main dome of the church doubles up as a lighthouse. The light in the Lighthouse signals the morse code for the letter ‘H’ (for Helsinki) and is one of the first landmark for vessels approaching Helsinki.

The Shipyard Bell

The Sibelius Monument and the Olympics Stadium

Sibelius Monument

The Sibelius Monument (more info) is one of the must-see places in Helsinki. It is an interesting Abstract Art sculpture made by Eila Hiltunen to commemorate Jean Sibelius – A celebrated Finnish composer whose work was instrumental in the formation of the Finnish Identity. The Finlandia Concert Hall built by Alvar Alto is named after a one of his most famous compositions. The sculpture itself is made of acid-proof stainless steel pipes that are welded together individually to form the distinctive abstract shape.

Sibelius Monument Sibelius Monument

The Helsinki Harbour

The Helsinki harbour is beautiful by night. The photo below (of a bridge and the power station) was taken while going to a Madonna concert that night.

Electric Power plant

Finland and Russia have a troubled past though relations are very cordial right now. Finland was a part of Russia for more than 100 years before it was annexed by Swedish Empire. Finland was attacked by Russia during World War II and some symbols of Russia in Helsinki (see photo below – the two-headed eagle in the Helsinki Harbour) still remind of this uneasy past. Russian Ships were docked in the harbour and heavily guarded when I visited it.

Russian ships Double Headed Eagles

The Olympic Stadium

The Olympic Stadium in Helsinki (and the tall tower beside the stadium) are iconic. The stadium was built for hosting the 1940 Olympics which was canceled due to World War II but hosted the 1952 Olympics. The tall tower beside the stadium offers great views of the entire city. The Finnish football team was practicing when I visited it. On the board inside the stadium,there are quite a few recognizable names amongst those who have set records here – Marion Jones, Justin Gatlin (both Sprinters), Moses Kiptanui and Haile Gebrselassie (both have held several long distance running records). Outside the stadium, there is a statue of the “Flying Finn” – Paavo Nurmi – who has won 12 Olympic medals and is considered one of the greatest track and field athletes of all time.

The running track Paavo Nuurmi

Helsinki Cathedral

The Helsinki cathedral (or St. Nicholas’ Church) is the distinctive landmark in Helsinki with it’s green domes and and bright white exterior. It looks stunning standing on a top of a hill with stairs leading to it and is considered as the unofficial symbol of the Helsinki city.

The Famous Cathedral

Cloudcamp Bangalore 2010 and Hadoop Summit

The 2nd CloudCamp Bangalore was held at Dayanand sagar College of Engineering. It was co-located with the First Hadoop summit in India. The Hadoop summit was interesting and more relevant to me as I am using a Hadoop cluster for Analytics at Inmobi. Dave kicked off Cloudcamp with signature “unPanel”. I was on the Unpanel this time and answered some questions on mobiles, netbooks and smartphones as access devices for the cloud and the on impact of Google patent on MapReduce.

The corridor discussions with a bunch of Hadoop committers were insightful. I also found out more about Mahout. Mahout is a Apache project to build scalable machine learning libraries. It is not restricted to Hadoop implementations, but much of the current activity seems to be around Hadoop.

Notes and embedded slides from the sessions I attended follow:

Hadoop summit Keynote

Data Management on Grid

Notes:

  • Y! uses a HDFS replication factor of 3 (the hadoop default) in most cases. Exceptions are big clusters with large number of applications running simultaneously.
  • Y! does not use Avro yet due to large amount of legacy data. Twitter uses Avro.
  • Data ingestion layer uses MapReduce for heavy lifting and format conversion for storage.
  • LZO is used for compression. gzip (not ideal due to non-block-level indexing) and bzip2 is also used. There are problems with slowness of bzip2 decompression but bzip2 delivers better compression ratios.
  • Data ingestion layer also oversees policy for data retention and purging.
  • Underlying filesystems is rarely a bottleneck for Hadoop. Mostly the synchronization semantics of HDFS is a bottleneck. A file operation is not successful until all the replicas are in sync.

Machine Learning using Hadoop

Notes:

  • There are clear differences between data mining and machine learning.
  • ML is harder to implement efficiently on Hadoop. Improving efficiency is still a research problem.
  • Hadoop creates one map job / block creating too many empty files and also many reducers.

Optimizing and Benchmarking Hadoop

Notes:

  • As a thumb rule, adding as much memory as money can buy is a a good idea for Hadoop
  • Consider Network connections as shuffle stage does heavy network I/O
  • Solid state disks might make sense at certain price/performance ratios. They are also more power efficient.

Tuning Hadoop To Deliver Performance To Your Application

Notes:

  • Several parameters to tune Hadoop but must be used in conjunction with each other.
  • Set number of map jobs slightly more than number of cores to ensure better utilization. Makes sure that data is processed in waves. Also better network utilization (as shuffle phase happens parallely with Map phase) along with CPU scheduling
  • Choosing a good HDFS block size is important. Number of HDFS blocks is directly proportional to number of Map tasks generated

Links to all presentations

Headstart 2010 – Hyderabad

We recently had our flagship event Headstart 2010, Hyderabad . This was the fourth edition of Headstart and the first one in Hyderabad. The previous editions were held in Bangalore and Mumbai. The event was co-located with ITsAP (HYSEA) at the HICC Hotel in Hi-Tech City. We had a good response from the startups.

This time we had a similar format as the previous Headstart. The on-stage demo and demo-pit were provided free of cost to the startups (We only charged for registration of the startups). The startups made good use of the demo-pit space. A few startups generated quite a bit of buzz including Notion-ink (makers of the Adam Tablet PC), Robosoft Systems (Makers of duct cleaning robot), Sysblitz (on-demand Business applications), Youpid (Tapping into the marriage market using social networks), Go-live gaming (Educational products using gaming) and Innoz (A student startup from Kerala that makes mobile applications for filesharing).

The complete list of selected startups

Two of the panels “Unconventional Sources of funding” and “Business Plan mistakes” were well received by packed halls. The questions posed to the panelists and the conversations with the attendees during the conference pointed to the fact that the startup scene in Hyderabad is still nascent and needs to mature as compared to Mumbai and Bangalore. We were happy to help connect many people in Hyderabad through the Headstart event and our regular Startup Saturdays there.

More coverage

Get Adobe Flash playerPlugin by wpburn.com wordpress themes